As I posted yesterday, the tweets going around comparing Wisconsin’s SAT/ACT scores to five states where teachers have no right to unionize are
based on bad data — it’s not true that Wisconsin’s SAT/ACT ranking is second in the nation, and that Texas, Virginia, Georgia, North Carolina, and South Carolina are all clustered at the bottom of the pile. The chart that says otherwise is based on outdated statistics and improper statistical analysis.
So what’s the truth? What would
good data tell us about this question?
Well, it turns out that that’s kind of a complicated question. I can answer it, but you’ll have to bear with me for more than 140 characters.
It’s hard to measure SAT/ACT performance, because different numbers of students take the tests in every state, and comparing the strongest students from one state with a much bigger sample from another doesn’t tell us much that’s interesting. A 2000 study in the Harvard Educational Review, in fact, found that 85% of the difference in states’ performance on those tests is due to variation in participation rates.
Having said that, though, it’s clear from the numbers in my last post that once you’ve controlled for participation Wisconsin remains near the top of the country on SAT/ACT scores, Virginia is near the middle, and the rest of the no-union states from the tweet are near the bottom. High school graduation rates — the subject of another popular Wisconsin tweet meme in recent days —
tell a similar story. It’s not as dramatic as best vs. worst, but it’s still dramatic.
Wisconsin does well on a third measure of student performance, too. Its scores on the National Assessment of Educational Progress (NAEP) in 2009 were
above the national average in three of four measures (fourth grade math and eighth grade math and reading) and at the national average in the other (fourth grade reading). Of the ten states in the US without teachers’ unions, only one — Virginia — had NAEP results above the national average, and four — Arizona, Alabama, Louisiana, and Mississippi — were in the bottom quintile. (One scholar, in fact, found that the states with the strongest teachers unions tended to out-perform states with weaker unions too.)