Dana Goldstein grabs my attention with this sentence about a new study showing that teachers with high “value-added” ratings have a genuine impact on the life outcomes of their students:
In a rare instance of edu-wonk consensus, both friends and skeptics of standardized tests are praising the study as reliable and groundbreaking.
The cynic in me wants to say that it’s precisely when everyone agrees on something in the education arena that we should be double-plus skeptical, but I’ll restrain my worst instincts and go along with this. After all, common sense suggests that good teachers are better than bad teachers, and I don’t 
have a hard time believing that success on standardized tests provides at least some indication of who’s better and who’s worse. Still, there are reasons to be cautious. Dana points out one of them:
The policy implications of the Chetty, Friedman, and Rockoff paper are, however, far from clear. As the researchers note in their conclusion, their study was conducted in a low-stakes setting, one in which student test scores were used neither to evaluate nor pay teachers. In a little-noticed footnote (#64) on page 50, the economists write: “even in the low-stakes regime we study, some teachers in the upper tail of the VA [value-added] distribution have test score impacts consistent with test manipulation. If such behavior becomes more prevalent when VA is actually used to evaluate teachers, the predictive content of VA as a measure of true teacher quality could be compromised.” [Emphasis added.] The importance of this caveat cannot be overstated.
I’d offer a broader reason to be cautious, though. The usual policy response to this kind of study is to support pay-for-performance, where high-ranking teachers get paid more. But does that really improve the quality of teaching in public schools? That’s a little tricky. Here’s the best case for what happens:
- In the short term, good teachers get paid more and that’s it. The pool of teachers stays the same but overall costs go up.
- In the medium term, if we get comfortable with the validity and reliability of value-added scores, we might start using them to fire the very worst teachers. This would increase the average quality of the teaching force.
- In the long term, higher salaries for top teachers would (a) attract better students into the teaching profession, and (b) put the fear of God into the lousy teachers and motivate them to improve their performance.
How likely is this to happen? I’m not optimistic. #2 will be very difficult to implement for reasons both good (are value-added scores really reliable enough to wreck people’s careers over?) and bad (there’s always savage pushback against firing people). And #3 just seems dubious to me. Will a one-in-ten prospect of making, say, $80,000 instead of $60,000 really attract more bright kids into teaching? That doesn’t seem very likely. And on the opposite end of the spectrum, my experience, at least, is that truly terrible employees rarely improve much no matter what motivation they have. They might improve just barely enough to avoid the axe, but that’s about it. (Does that sound harsh? Sorry. But I doubt that anyone with very much management experience will seriously disagree. Feel free to school me in comments about this if you disagree.)
So in the end, I suspect that pay-for-performance will do little to attract better students into the teaching profession, will do little to motivate bad teachers to get better, and will therefore succeed only to the extent that it forces bad teachers out of the system entirely. Maybe it will do that, and maybe that’s worthwhile all by itself. But I’d need to be convinced.
If we really believe that good teachers make a substantial difference, then our policy response needs to be something that either (a) attracts better students into the teaching profession on a mass scale or (b) improves the performance of the existing teaching force on a mass scale. (Or both.) Unfortunately, I’m not sure there’s any way to accomplish (a) except to increase starting salaries significantly and then wait 40 years for the teaching force to turn over, and I’m not sure there’s any definitive way to accomplish (b) at all. It’s just a very gnarly problem.
