Via Hechinger Report.
School systems around the country are trying to use objective, quantifiable measures to identify which are the good teachers and which are the bad ones. One popular approach used in New York, Chicago and other cities, is to calculate a value-added performance measure (VAM). Essentially, you create a model that begins by calculating how much kids’ test scores, on average, increase each year. (Test score year 2 minus test score year 1). Then you give a high score to teachers who have students who post test-score gains above the average. And you give a low score to teachers whose students show smaller test-score gains. There are lots of mathematical tweaks, but the general idea is to build a model that answers this question: are the students of this particular teacher learning more or less than you expect them to? The teachers’ value-added scores are then used to figure out which teachers to train, fire or reward with bonuses.
Two academic researchers from the University of Southern California and the University of Pennsylvania looked at these value-added measures in six districts around the nation and found that there was weak to zero relationship between these new numbers and the content or quality of the teacher’s instruction.
“These results call into question the fixed and formulaic approach to teacher evaluation that’s being promoted in a lot of states right now,” said Morgan Polikoff, one of the study’s authors, in a video that explains his paper, “Instructional Alignment as a Measure of Teaching Quality,” published online in Education Evaluation and Policy Analysis on May 13, 2014. ”These measures are not yet up to the task of being put into, say, an index to make important summative decisions about teachers.”
Polikoff of the University of Southern California and Andrew Porter of the University of Pennsylvania looked at the value-added scores of 327 fourth- and eighth-grade mathematics and English language arts teachers across all six school districts included in the Measures of Effective Teaching (MET) study (New York City, Dallas, Denver, Charlotte-Mecklenburg, Memphis, and Hillsborough County, Florida). Specifically, they compared the teachers’ value added scores with how closely their instructional materials aligned with their state’s instructional standards and the content of the state tests. But teachers who were teaching the right things weren’t getting higher value-added scores.
They also looked at other measures of teacher quality, such as teacher observations and student evaluations. Similarly, teachers who won high marks from professional observers or students were also not getting higher value-added scores.
“What we’re left with is that state tests aren’t picking up what we think of as good teaching,” Polikoff said.
What’s interesting is that Polikoff’s and Porter’s research was funded by the Gates Foundation, which had been touting how teachers’ effectiveness could be estimated by their students’ progress on standardized tests. The foundation had come under fire from economists for flawed analysis. Now this new Gates Foundation’ commissioned research has proved the critics right. (The Gates Foundation is also among the funders of The Hechinger Report).
Polikoff said that the value-added measures do provide some information, but they’re meaningless if you want to use them to improve instruction. “If the things we think of as defining good instruction don’t seem to producing substantially better student achievement, then how is it that teachers will be able to use the value-added results to make instructional improvements?” he asked.
Polikoff concludes that the research community needs to develop new measures of teacher quality in order to “move the needle” on teacher performance.
You can read the entire report below
Educational Evaluation and Policy Analysis-2014-Polikoff
A good rundown of the changes the House made to SB229, from the Ennis, Roberts & Fischer Co law firm.
The Ohio House Education Committee has unveiled sweeping changes to Substitute Senate Bill 229 with regard to teacher and principal evaluations. The original version of SB 229, which passed the Senate unanimously on December 4th, 2013, modified frequency and composition of teacher evaluations and reduced some of the burden on school administrators. The new version of the Bill proposed by the House Education Committee, however, would modify both the OTES and OPES evaluation systems in ways that would undoubtedly place additional strain on the relatively untested evaluation systems. The proposed changes include the following:
Here's the Lesgislative Service commisions comparision document so oyu cna see what changes were made from the unanimously passed Senate version
- Bumps student growth measures back up to 50% from the 35% proposed by the Senate, unless a district elects to use an alternative “student survey” framework (available for grades 4-12), in which case the final rating would be comprised of 40% SGM, 40% teacher performance rating, and 20% student survey results;
- Requires that an evaluator use an average score if a teacher receives different scores on the observations and review components of the evaluations;
- Increases SGM from three to five total possible ratings: “Most Effective”, “Above Average”, “Average”, “Below Average”, and “Least Effective”;
- Adds new performance level rating of “Effective” that will exist in the realm between “Skilled” and “Developing”;
- Requires that at least one formal observation of a teacher be unannounced;
- Beginning in 2015, allows districts to evaluate “Accomplished” and “Skilled” teachers every other year, but only if the teacher’s SGM score is rated “Average” or higher (teachers must still receive one observation and a conference in the “off” year);
- District can elect not to evaluate 1) a teacher who is on leave for 70% or more of the year, and 2)a teacher who submitted notice of retirement before Dec. 1st;
- Teachers rated “Effective” “Developing” or “Ineffective” must be placed on an improvement plan;
- In 2015 and beyond, districts cannot assign students to a teacher who has been rated ineffective for two or more years (but does not specify what a district should do with these teachers!);
- A district is also prohibited from assigning a student teacher to a teacher who is “Developing” or “Ineffective” during the previous year;
- If a teacher with at least ten years of experience receives a designation of either “Least Effective” or “Below Average” on his/her SGM rating, that teacher may be rated “Developing” only once;
- Mandates that results of an evaluation must follow the teacher even if he/she is transferred to a new building or takes employment elsewhere;
- Requires ODE to develop a standardized framework for assessing SGM for all non-value added grade levels and subjects by 2016;
- By 2016, districts must administer assessments to students in each of grades K-12 for English Language Arts, Mathematics, Social Studies, and Science. Assessments must be selected by ODE and based on value-added progress dimension or vendor-developed student growth measures (may include assessments already required by law);
- Beginning next July, evaluators must verify completion of at least one evaluation training course outlined in the bill;
- After July 1, 2015, the State Board must ensure individuals seeking licensure as superintendent, assistant superintendent, principal, vocational director, administrative specialist, or supervisor have completed a teacher evaluator training;
- The revised bill mandates that the State Board of Education must develop a standards based system for principals and assistant principals, which districts must conform to;
- Third grade reading guarantee assessments must either be value-added or vendor-approved assessments;
- ODE must provide detailed report of school performance on evaluations to general assembly, and must accept comments for improvement from districts that it passes on to general assembly;
- Exempts from collective bargaining all amendments made by the bill to 3319.111, 3319.112, 3319.113, 3319.114, 3319.115, and 3319.117;
- Permits a district to enter into a MOU with union that stipulates value-added progress demission rating issued for 2014-2015 will not be used when making decisions regarding dismissal, retention, tenure or compensation.
SB229 Comparison Document