Academic scholars are often dismayed when policymakers pass laws that disregard or misinterpret their research findings. The use of value-added methods (VAMS) in education policy is a case in point.
About a decade ago, researchers reported that teachers are the most important school-level factor in students’ learning, and that that their effectiveness varies widely within schools (McCaffrey, Koretz, Lockwood, & Hamilton 2004; Rivkin, Hanushek, & Kain 2005; Rockoff 2004). Many policymakers interpreted these findings to mean that teacher quality rests with the individual rather than the school and that, because some teachers are more effective than others, schools should concentrate on increasing their number of effective teachers.
Based on these assumptions, proponents of VAMS began to argue that schools could be improved substantially if they would only dismiss teachers with low VAMS ratings and replace them with teachers who have average or higher ratings (Hanushek 2009). Although panels of scholars warned against using VAMS to make high-stakes decisions because of their statistical limitations (American Statistical Association, 2014; National Research Council & National Academy of Education, 2010), policymakers in many states and districts moved quickly to do just that, requiring that VAMS scores be used as a substantial component in teacher evaluation.
While researchers continue to analyze and improve VAMS models, it is important to step back and consider a prior set of questions:
- Does the wide variation in teachers’ effectiveness within schools simply mean that some teachers are inherently better than others, or is there a more complex and promising explanation of this finding?
- Is the strategy of augmenting human capital one teacher at a time likely to pay off for students? Or will relying on VAMS for teacher evaluations have unintended consequences that interfere with a school’s collective efforts to improve?
In this column, I bring an organizational perspective to the prospect of using VAMS to improve teacher quality. I suggest why, in addition to VAMS’ methodological limitations, reformers should be very cautious about relying on VAMS to make decisions that have important consequences for both teachers and their students.
Why Is There Variation In Teacher Effectiveness Within Schools?
In his classic analysis, “Social Capital in the Creation of Human Capital,” James Coleman (1988) argues that individuals’ human capital is transformed for the benefit of the organization by social capital, which “inheres in the structure of relations between actors and among actors” (p. S98). In education, this suggests that whatever level of human capital schools acquire through hiring can subsequently be developed through activities such as grade-level or subject-based teams of teachers, faculty committees, professional development, coaching, evaluation, and informal interactions. As teachers join together to solve problems and learn from one another, the school’s instructional capacity becomes greater than the sum of its parts.
Unfortunately, U.S. schools were never designed to benefit from social capital. In fact, over 40 years ago, historian David Tyack (1974) and sociologist Dan Lortie (1975) depicted the school as an organizational “egg crate,” where teachers work in the isolation of their classroom. In egg-crate schools, teachers focus on their own students largely to the exclusion of others, and they interact minimally and intermittently with their colleagues. As a result, their expertise remains locked within their classroom (Darling-Hammond 2001; Hargreaves & Fullan 2012; Johnson 1990; Kardos & Johnson 2007; Little 1990). This egg-crate model was efficient for managing the “factory school,” but did not serve students well; nor does it support the instructional needs of today's teachers.
Therefore, when teachers in the same school continue to work in isolation, they cannot benefit from the social capital that their school might provide. As a result, wide differences in teachers’ effectiveness persist over time.
The Evidence On School-Based Improvement Efforts
Studies have persuasively documented the benefits of systematic efforts to improve student learning through school-based improvement initiatives (Bryk, Sebring, Allensworth, Easton, & Luppescu 2010; McLaughlin & Talbert 2001; Rosenholtz 1989). Successful efforts increase norms of shared responsibility among teachers and create structures and opportunities for learning that promote interdependence—rather than independence—among them. That is social capital at work.
Many who dismiss the potential of social capital to improve schools doubt that teachers can improve significantly over time. However, a recent study by Kraft and Papay (2014)showed that teachers working in more favorable professional environments—as rated by a school’s staff—improved throughout the ten years they analyzed, while those who worked in environments judged to be less supportive stagnated. This and other studies challenge the conventional view that teachers reach a “plateau” in their development relatively early in their career (Rivkin et al. 2005). Creating a school context that supports teachers’ work can have important, lasting benefits for students and faculty throughout the school, whereas simply swapping out low-scoring for a high-scoring individuals without changing the context in which they work probably will not (Ladd & Sorenson 2014; Leana 2011; Lohr 2012).
Threats To School-Based Improvement Efforts
Not only are personnel polices based on VAMS scores likely to have, at best, modest effects on a school’s success, they may inadvertently undermine improvement efforts that are already underway. How so? Here, I suggest several possible unintended consequences of increasing reliance on VAMS (for a more detailed discussion see here).
(Continue reading at the Shanker Institute)