How Do Value-Added Indicators Compare to Other Measures of Teacher Effectiveness?



  • Value-added measures are positively related to almost all other commonly accepted measures of teacher performance such as principal evaluations and classroom observations.
  • While policymakers should consider the validity and reliability of all their measures, we know more about value-added than others.
  • The correlations appear fairly weak, but this is due primarily to lack of reliability in essentially all measures.
  • The measures should yield different performance results because they are trying to measure different aspects of teaching, but they differ also because all have problems with validity and reliability.
  • Using multiple measures can increase reliability; validity is also improved so long as the additional measures capture aspects of teaching we value.
  • Once we have two or three performance measures, the costs of more measures for accountability may not be justified. But additional formative assessments of teachers may still be worthwhile to help these teachers improve.


In the recent drive to revamp teacher evaluation and accountability, measures of a teacher’s value added have played the starring role. But the star of the show is not always the best actor, nor can the star succeed without a strong supporting cast. In assessing teacher performance, observations of classroom practice, portfolios of teachers’ work, student learning objectives, and surveys of students are all possible additions to the mix.

All these measures vary in what aspect of teacher performance they measure. While teaching is broadly intended to help students live fulfilling lives, we must be more specific about the elements of performance that contribute to that goal – differentiating contributions to academic skills, for instance, from those that develop social skills. Once we have established what aspect of teaching we intend to capture, the measures differ in how valid and reliable they are in capturing that aspect.

Although there are big holes in what we know about how evaluation measures stack up on these two criteria, we can draw some important conclusions from the evidence collected so far. In this brief, we will show how existing research can help district and state leaders who are thinking about using multiple measures of teacher performance to guide them in hiring, development, and retention.

[readon2 url=""]Continue reading...[/readon2]

Do Different Value-Added Models Tell Us the Same Things?



  • Statistical models that evaluate teachers based on growth in student achievement differ in how they account for student backgrounds, school, and classroom resources. They also differ by whether they compare teachers across a district (or state) or just within schools.
  • Statistical models that do not account for student background factors produce estimates of teacher quality that are highly correlated with estimates from value-added models that do control for student backgrounds, as long as each includes a measure of prior student achievement.
  • Even when correlations between models are high, different models will categorize many teachers differently.
  • Teachers of advantaged students benefit from models that do not control for student background factors, while teachers of disadvantaged students benefit from models that do.
  • The type of teacher comparisons, whether within or between schools, generally has a larger effect on teacher rankings than statistical adjustments for differences in student backgrounds across classrooms.


There are good reasons for re-thinking teacher evaluation. As we know, evaluation systems in most school districts appear to be far from rigorous. A recent study showed that more than 99 percent of teachers in a number of districts were rated “satisfactory,” which does not comport with empirical evidence that teachers differ substantially from each other in terms of their effectiveness. Likewise, the ratings do not reflect the assessment of the teacher workforce by administrators, other teachers, or students.

Evaluation systems that fail to recognize the true differences that we know exist among teachers greatly hamper the ability of school leaders and policymakers to make informed decisions about such matters as which teachers to hire, what teachers to help, which teachers to promote, and which teachers to dismiss. Thus it is encouraging that policymakers are developing more rigorous evaluation systems, many of which are partly based on student test scores.

Yet while the idea of using student test scores for teacher evaluations may be conceptually appealing, there is no universally accepted methodology for translating student growth into a measure of teacher performance. In this brief, we review what is known about how measures that use student growth align with one another, and what that agreement or disagreement might mean for policy.

[readon2 url=""]Continue reading...[/readon2]

2 new studies question value add measures

Evidence is overwhelming, as yet more studies show that using value add to measure teacher quality is fraught with error.

Academic tracking in secondary education appears to confound an increasingly common method for gauging differences in teacher quality, according to two recently released studies.

Failing to account for how students are sorted into more- or less-rigorous classes—as well as the effect different tracks have on student learning—can lead to biased "value added" estimates of middle and high school teachers' ability to boost their students' standardized-test scores, the papers conclude.

"I think it suggests that we're making even more errors than we need to—and probably pretty large errors—when we're applying value-added to the middle school level," said Douglas N. Harris, an associate professor of economics at Tulane University in New Orleans, whose study examines the application of a value-added approach to middle school math scores.

High-school-level findings from a separate second study, by C. Kirabo Jackson, an associate professor of human development and social policy at Northwestern University in Evanston, Ill., complement Mr. Harris' paper.

"At the elementary level, [value-added] is a pretty reliable measure, in terms of predicting how teachers will perform the following year," Mr. Jackson said. "At the high school level, it is quite a bit less reliable, so the scope for using this to improve student outcomes is much more limited."

The first study mentioned in this article concludes(emphasis ours)

We test the degree to which variation in measured performance is due to misalignment versus selection bias in a statewide sample of middle schools where students and teachers are assigned to explicit “tracks,” reflecting heterogeneous student ability and/or preferences. We find that failing to account for tracks leads to large biases in teacher value-added estimates.

A teacher of all lower track courses whose measured value-added is at the 50th percentile could increase her measured value-added to the 99th percentile simply by switching to all upper-track courses. We estimate that 75-95 percent of the bias is due to student sorting and the remainder due to test misalignment.

We also decompose the remaining bias into two parts, metric and multidimensionality misalignment, which work in opposite directions. Even after accounting for explicit tracking, the standard method for estimating teacher value-added may yield biased estimates.

The second study, replicates the findings and concludes

Unlike in elementary-school, high-school teacher effects may be confounded with both selection to tracks and unobserved track-level treatments. I document sizable confounding tracks effects, and show that traditional tests for the existence of teacher effects are likely biased. After accounting for these biases, algebra teachers have modest effects and there is little evidence of English teacher effects.

Unlike in elementary-school, value-added estimates are weak predictors of teachers’ future performance. Results indicate that either (a) teachers are less influential in high-school than in elementary-school, or (b) test-scores are a poor metric to measure teacher quality at the high-school level.

Corporate education reformers need to begin to address the science that is refuting their policies, the sooner this happens, the less damage is likely to be wrought.

Education News for 09-24-2012

State Education News

  • Extent of student-data scandal still unknown (Columbus Dispatch)
  • On a tight departure-and-arrival schedule, principals rotated in and out of the data czar’s office each year…Read more...

  • School districts find organized opposition is increasingly common (Columbus Dispatch)
  • There are some people in Upper Arlington who say that on Nov. 6, they will vote no on the school district’s tax request. That’s not new…Read more...

  • State will measure physical education (Columbus Dispatch)
  • Next year’s state report cards will feature a new measure for Ohio schools: how their students are doing in physical education…Read more...

  • Schools pile up millions in legal bills (Dayton Daily News)
  • Consulting with school attorneys is a necessary but unpredictable expense, local school officials say, that can cost tens of thousands of dollars for a single case…Read more...

  • Superintendents fight Ohio’s ‘Third Grade Guarantee’ (Washington Post)
  • The Chicago teachers strike was the biggest action that we’ve seen against aspects of modern school reform, but people in other places are fighting too. Here’s a report from Ohio…Read more...

Local Education News

  • Coleman to help, not lead, schools (Columbus Dispatch)
  • Columbus Mayor Michael B. Coleman plans to lend his leadership to the Columbus City Schools, including helping to select a replacement…Read more...

  • 'Brain drain' has attention of educators (Marion Star)
  • Data to support that Marion County residents who graduate from universities and colleges move out of Marion County after they graduate is hard to come by…Read more...


  • Better school lunches (Cleveland Plain Dealer)
  • While some kids may groan or even hold lunch boycotts, the United States Department of Agriculture's new focus on fresh fruits and vegetables in school lunches…Read more...

  • Outing evil (Columbus Dispatch)
  • Americans have been shaken in recent years when details have come to light of revered institutions covering up the sexual abuse of children…Read more...

  • Helping hands (Columbus Dispatch)
  • Mayor Michael B. Coleman and City Council President Andrew J. Ginther are stepping up to help the Columbus City School District through a difficult time…Read more...

  • Rejecting test scores as a core value (Los Angeles Times)
  • It wasn't about money. It was about respect. That's what Chicago teachers union president Karen Lewis kept reminding the public…Read more...

Education News for 07-09-2012

Statewide Stories of the Day

  • Principals will start seeing the same scrutiny as teachers when new evaluation system goes statewide next fall (Plain Dealer)
  • CLEVELAND - Teachers have been under increasing scrutiny the last few years, as Ohio and other states roll out new teacher evaluation plans. That spotlight also is falling on principals. Ohio will require school districts statewide to have an evaluation system for principals up and running by the 2013-14 school year, the same time that the new teacher evaluations take effect. Though districts already evaluate principals to varying degrees, Ohio has edged toward a more standard and rigorous measure of principal quality for a few years. Read more...

  • Ohio plans tougher high-school tests (Enquirer)
  • Ohio is ready to swap its statewide graduation test for a series of more rigorous high school exams beginning in 2014-15. The tests are designed to measure student readiness for college or a career, something the Ohio Graduation Test could never do, said James Herrholtz, associate superintendent of the Ohio Department of Education’s division of learning. “The OGT is a low hurdle,” he said. “It was never really designed to measure whether a student is college-ready.” Read more...

  • Reading guarantee for Ohio 3rd-graders starts with September screenings (Plain Dealer)
  • CLEVELAND — Much of the talk about Ohio's third-grade reading guarantee has centered on whether it's better to hold back a struggling student who can't read instead of promoting him to the next grade. But the new mandate involves a lot more than the retention issue. And some school officials are worried about how they're going to pay for it. Starting this year, school districts and charter schools will have to screen all students in kindergarten through third grade by Sept. 30. Read more...

  • Westerville phasing out popular magnet schools (Dispatch)
  • Longfellow Elementary School in Westerville never struggled in academics. Among central Ohio elementary schools in the 2010-11 school year, its state test scores were the highest, and they were seventh-best in the state. Students could land a coveted seat only by winning a lottery. But because of budget cuts, Longfellow closed for good this summer, as did Central College Elementary, another high-performing public school in Westerville. Both will consolidate into Hanby Elementary School. Read more...

Local Issues

  • Funny Math (The Other Paper)
  • Calculating data is boring, entering it into a computer system is mundane. Even the word data inspires narcolepsy. Maybe that’s why Columbus City Schools employees allegedly tried to liven up the process a bit by living on the data-entry edge; allegedly fudging attendance figures every June before submitting their State Report Card data to the Ohio Department of Education. After all, there’s surely nothing more thrilling than possibly misleading taxpayers and the state watchdogs in a ploy to enhance district graduation rates. Read more...

  • NB teachers reject contract (Courier)
  • NORTH BALTIMORE - A North Baltimore teachers' union recently rejected a contract offered by its school board. Terms were not released. According to the North Baltimore Education Association, teachers have been in negotiations with the North Baltimore school board since May. In June, school board declared an impasse in negotiations until a mediator could be contacted. On June 30, the current contract expired and the teachers' association has been working under the previous negotiated agreement. Read more...

  • Dayton-area young readers get a boost (Dayton Daily News)
  • DAYTON — Montgomery County-Dayton region has been selected to join a national effort to improve reading proficiency among third-grade students. The region will become a charter member of the Campaign for Grade-Level Reading Communities Network, a 124 member organization charged with implementing strategies to enhance the reading skills of low-income students. “Until third grade, children learn to read,” said Robyn Lightcap, director for ReadySetSoar, a local organization aimed at improving kindergarten readiness. Read more...

  • Shawnee High School grads create app for Android (Lima News)
  • LIMA — First the fun, then the function. That's the idea behind a series of new applications for Android that two 2008 Shawnee High School graduates helped create. The app, “TapDeck,” will be available this month, Adam Kriegel said. The name plays off “tape deck,” and also the company he and fellow Carnegie Mellon University graduates formed, TapAudio LLC. At the moment, a gimmicky tape recorder allows the user to record up to 10 seconds of audio that can be scrambled in a variety of ways, Kriegel said. Read more...