Guest column: Another perspective on ranking ‘valuable’ schools

By Jeff Padden/Public Policy Associates 

Bridge Magazine published an analysis of the Value-Added Matrix scores for 560 school districts in Michigan, including 52 charter schools and 508 traditional public schools. The VAM scores attempt to correct for differences among the student bodies in schools by looking at the socioeconomic condition of the district, a factor known to affect student achievement.

While there is substantial controversy about the method, the goal is worthy: We should not always judge the outcomes of a poor district by the same absolute standards as we do a school in a wealthy district. Instead, we should look at whether schools meet, exceed or fall short of their expected results.

State Champs: See how your district ranks.

This, of course, includes a trap for students. A school with very low expected results may achieve precisely those results, but, while parents and students might feel good about that, the graduates will still not be able to succeed in higher education or in competing for jobs. Godwin Heights Public Schools, cited in the Bridge coverage, is a clear example.

While the Bridge article stated that “only 6 percent of its juniors were considered ‘college ready’ on their ACTs in 2012,” it was the top ranked traditional district in the state. That, however, is an issue for another day. For now, it is enough to say that all simplistic means of ranking or rating school performance have significant limitations.

What is worth looking at in the VAM rankings is how schools are distributed throughout the list. Much has been made of the fact that three of the top five schools are charters. Yet, seven of the top 10 are traditional public schools. And, four of the bottom five are charters, but six of the bottom 10 are traditionals.

So, which type is really doing better by this VAM measure?

That can be answered by looking at how the two types of schools are distributed throughout the list. One way to do so is to divide the list into five equal segments from top to bottom. If the two types performed equally well, we would expect them to be proportionally represented in each of those five groups, with about 102 traditionals and 10 charters in each group. The actual distribution is quite different, especially for charters, as the table shows.

To correct for the much larger number of traditionals, the bar chart at right shows how the actual results are different from a proportional distribution. A value of 100 percent means that a district is represented in the group exactly as expected; 50 percent means that it is half represented.

The chart vividly shows that, while charters and traditionals are almost equally represented in the top group, charters are much less likely to show up in the middle. Most surprising is that they are more than twice as likely to be in the bottom group than would be expected.

Keep in mind, this is after the rankings have been corrected for the socioeconomic status of the district; poor districts are pushed upward by the VAM ranking method. So, a low-ranked poor school must perform exceptionally badly to land in the bottom group. The fact is that 39 percent of charters in the bottom group have more than 70 percent low socioeconomic status students, while only 16 percent of traditionals have poverty rates that high.

This should be a matter of grave concern to policy-makers and charter school authorizers.

This distribution of school performance is consistent with research by my firm, Public Policy Associates, Inc., from a dozen years ago. At that time, our national study of employer-linked charter schools concluded that the range of performance was wider than for traditional public schools; even then, charters were present at both extremes.

As a product of public schools, the father of children in public schools and as a board member of a charter operating company and former charter school board member, my perspective covers both sides of this street. Most Michiganians, including me, want to see the best possible results for our kids. Too often beliefs, rather than good analysis, drive that debate. We should all hope that the VAM rankings and other measurement tools can be looked at objectively rather than being skewed by either side.

Jeff Padden is the founder and president of Public Policy Associates, a Lansing-based research firm.

Bridge welcomes guest columns from a diverse range of people on issues relating to Michigan and its future. The views and assertions of these writers do not necessarily reflect those of Bridge or The Center for Michigan.

If you are interested in submitting a guest commentary, please contact Monica WilliamsClick here for details and submission guidelines.

Has this story impacted or informed you about Michigan? Please support our work.

No other news outlet is dedicated to providing the same level of in-depth, data-driven coverage of Michigan’s issues as Bridge Magazine. Any donation between now and December 31, will be matched dollar-for-dollar, thanks to our generous partners. Become a Bridge Club member and help our reporters get the resources they need to ramp up coverage during a critical election year. Join the Bridge team today.

Pay with VISA Pay with MasterCard Pay with American Express Donate now

Comment Form

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Comments

Gary Naeyaert
Sun, 02/03/2013 - 10:11am
It's a little disturbing that Jeffrey Padden, a respected policy professional, would use an analysis that included data from only 52 charter public schools (less than 20% of all charter schools in the state) to make such broad statements as "This should be a matter of grave concern to policy-makers and charter school authorizers."