Researchers Give Failing Marks to Teacher Evaluation Systems

Via Hechinger Report.

School systems around the country are trying to use objective, quantifiable measures to identify which are the good teachers and which are the bad ones. One popular approach used in New York, Chicago and other cities, is to calculate a value-added performance measure (VAM). Essentially, you create a model that begins by calculating how much kids’ test scores, on average, increase each year. (Test score year 2 minus test score year 1). Then you give a high score to teachers who have students who post test-score gains above the average. And you give a low score to teachers whose students show smaller test-score gains. There are lots of mathematical tweaks, but the general idea is to build a model that answers this question: are the students of this particular teacher learning more or less than you expect them to? The teachers’ value-added scores are then used to figure out which teachers to train, fire or reward with bonuses.

Two academic researchers from the University of Southern California and the University of Pennsylvania looked at these value-added measures in six districts around the nation and found that there was weak to zero relationship between these new numbers and the content or quality of the teacher’s instruction.

“These results call into question the fixed and formulaic approach to teacher evaluation that’s being promoted in a lot of states right now,” said Morgan Polikoff, one of the study’s authors, in a video that explains his paper, “Instructional Alignment as a Measure of Teaching Quality,” published online in Education Evaluation and Policy Analysis on May 13, 2014. ”These measures are not yet up to the task of being put into, say, an index to make important summative decisions about teachers.”

Polikoff of the University of Southern California and Andrew Porter of the University of Pennsylvania looked at the value-added scores of 327 fourth- and eighth-grade mathematics and English language arts teachers across all six school districts included in the Measures of Effective Teaching (MET) study (New York City, Dallas, Denver, Charlotte-Mecklenburg, Memphis, and Hillsborough County, Florida). Specifically, they compared the teachers’ value added scores with how closely their instructional materials aligned with their state’s instructional standards and the content of the state tests. But teachers who were teaching the right things weren’t getting higher value-added scores.

They also looked at other measures of teacher quality, such as teacher observations and student evaluations. Similarly, teachers who won high marks from professional observers or students were also not getting higher value-added scores.

“What we’re left with is that state tests aren’t picking up what we think of as good teaching,” Polikoff said.

What’s interesting is that Polikoff’s and Porter’s research was funded by the Gates Foundation, which had been touting how teachers’ effectiveness could be estimated by their students’ progress on standardized tests. The foundation had come under fire from economists for flawed analysis. Now this new Gates Foundation’ commissioned research has proved the critics right. (The Gates Foundation is also among the funders of The Hechinger Report).

Polikoff said that the value-added measures do provide some information, but they’re meaningless if you want to use them to improve instruction. “If the things we think of as defining good instruction don’t seem to producing substantially better student achievement, then how is it that teachers will be able to use the value-added results to make instructional improvements?” he asked.

Polikoff concludes that the research community needs to develop new measures of teacher quality in order to “move the needle” on teacher performance.

You can read the entire report below

Educational Evaluation and Policy Analysis-2014-Polikoff

Share your comment:

The Hijacking of SB229

A good rundown of the changes the House made to SB229, from the Ennis, Roberts & Fischer Co law firm.

The Ohio House Education Committee has unveiled sweeping changes to Substitute Senate Bill 229 with regard to teacher and principal evaluations. The original version of SB 229, which passed the Senate unanimously on December 4th, 2013, modified frequency and composition of teacher evaluations and reduced some of the burden on school administrators. The new version of the Bill proposed by the House Education Committee, however, would modify both the OTES and OPES evaluation systems in ways that would undoubtedly place additional strain on the relatively untested evaluation systems. The proposed changes include the following:

  • Bumps student growth measures back up to 50% from the 35% proposed by the Senate, unless a district elects to use an alternative “student survey” framework (available for grades 4-12), in which case the final rating would be comprised of 40% SGM, 40% teacher performance rating, and 20% student survey results;
  • Requires that an evaluator use an average score if a teacher receives different scores on the observations and review components of the evaluations;
  • Increases SGM from three to five total possible ratings: “Most Effective”, “Above Average”, “Average”, “Below Average”, and “Least Effective”;
  • Adds new performance level rating of “Effective” that will exist in the realm between “Skilled” and “Developing”;
  • Requires that at least one formal observation of a teacher be unannounced;
  • Beginning in 2015, allows districts to evaluate “Accomplished” and “Skilled” teachers every other year, but only if the teacher’s SGM score is rated “Average” or higher (teachers must still receive one observation and a conference in the “off” year);
  • District can elect not to evaluate 1) a teacher who is on leave for 70% or more of the year, and 2)a teacher who submitted notice of retirement before Dec. 1st;
  • Teachers rated “Effective” “Developing” or “Ineffective” must be placed on an improvement plan;
  • In 2015 and beyond, districts cannot assign students to a teacher who has been rated ineffective for two or more years (but does not specify what a district should do with these teachers!);
  • A district is also prohibited from assigning a student teacher to a teacher who is “Developing” or “Ineffective” during the previous year;
  • If a teacher with at least ten years of experience receives a designation of either “Least Effective” or “Below Average” on his/her SGM rating, that teacher may be rated “Developing” only once;
  • Mandates that results of an evaluation must follow the teacher even if he/she is transferred to a new building or takes employment elsewhere;
  • Requires ODE to develop a standardized framework for assessing SGM for all non-value added grade levels and subjects by 2016;
  • By 2016, districts must administer assessments to students in each of grades K-12 for English Language Arts, Mathematics, Social Studies, and Science. Assessments must be selected by ODE and based on value-added progress dimension or vendor-developed student growth measures (may include assessments already required by law);
  • Beginning next July, evaluators must verify completion of at least one evaluation training course outlined in the bill;
  • After July 1, 2015, the State Board must ensure individuals seeking licensure as superintendent, assistant superintendent, principal, vocational director, administrative specialist, or supervisor have completed a teacher evaluator training;
  • The revised bill mandates that the State Board of Education must develop a standards based system for principals and assistant principals, which districts must conform to;
  • Third grade reading guarantee assessments must either be value-added or vendor-approved assessments;
  • ODE must provide detailed report of school performance on evaluations to general assembly, and must accept comments for improvement from districts that it passes on to general assembly;
  • Exempts from collective bargaining all amendments made by the bill to 3319.111, 3319.112, 3319.113, 3319.114, 3319.115, and 3319.117;
  • Permits a district to enter into a MOU with union that stipulates value-added progress demission rating issued for 2014-2015 will not be used when making decisions regarding dismissal, retention, tenure or compensation.

Here's the Lesgislative Service commisions comparision document so oyu cna see what changes were made from the unanimously passed Senate version

SB229 Comparison Document

Share your comment:

May 2014 Levy Results

School levies continued their normal pattern of results in the May 2014 primary, almost identically mirroring the May 2013 results.
N/R Failed Passed Pass %
New 38 27 41.5%
Renewal 8 74 90.2%
Over all 46 101 68.7%

Here's the full list of preliminary results:

District Local Result N/R Votes for Voters Against
Ada Hardin Passed Renewal 61.07 38.93
Allen East Allen Passed Renewal 67.07 32.93
Arcadia Hancock Passed Renewal 53.92 46.08
Athens Athens Passed Renewal 62.77 37.23
Aurora Portage Passed Renewal 64.78 35.22
Austintown Mahoning Failed New 34.18 65.82
Ayersville Defiance Failed New 21.71 78.29
Beaver Columbiana Passed Renewal 50.09 49.91
Bellaire Belmont Failed New 43.64 56.36
Belpre Washington Passed Renewal 71.35 28.65
Berne Union Fairfield Passed Renewal 53.64 46.36
Bethel Miami Passed New 53.10 46.90
Bettsville Seneca Failed New 47.89 52.11
Bluffton Allen Passed Renewal 70.95 29.05
Boardman Mahoning Passed Renewal 59.81 40.19
Boardman Mahoning Passed Renewal 59.80 40.20
Botkins Shelby Passed Renewal 60.95 39.05
Bristol Trumbull Failed Renewal 43.48 56.52
Brooklyn Cuyahoga Passed Renewal 53.07 46.93
Brookville Montgomery Passed New 54.62 45.38
Brunswick Medina Passed Renewal 62.91 37.09
Canal Winchester Franklin Passed Renewal 56.95 43.05
Carey Wyandot Passed Renewal 57.09 42.91
Clark-Shawnee Clark Passed New 51.27 48.73
Claymont Tuscarawas Passed Renewal 65.22 34.78
Cloverleaf Medina Passed New 56.38 43.62
Colonel Crawford Crawford Failed New 47.30 52.70
Coshocton County Coshocton Passed New 50.08 49.92
Danbury Ottawa Passed New 54.87 45.13
Danville Knox Passed Renewal 69.26 30.74
Defiance Defiance Passed New 54.71 45.29
Delphos Allen Passed Renewal 70.18 29.82
Delphos Allen Passed Renewal 69.73 30.27
Dover Tuscarawas Failed New 44.56 55.44
East Clinton Clinton Passed Renewal 62.09 37.91
Edison Jefferson Passed New 52.45 47.55
Elyria Lorain Passed Renewal 65.30 34.70
Fairfield Butler Passed New 57.87 42.13
Fairless Stark Failed New 41.14 58.86
Field Portage Failed New 46.39 53.61
Findlay Hancock Passed Renewal 66.45 33.55
Fort Loramie Shelby Passed New 74.30 25.70
Franklin Warren Passed New 52.66 47.34
Franklin Muskingum Failed New 42.04 57.96
Genoa Area Ottawa Failed New 33.74 66.26
Granville Licking Passed Renewal 63.19 36.81
Green Summit Passed Renewal 71.57 28.43
Greeneview Greene Failed New 35.90 64.10
Greenon Clark Failed New 49.15 50.85
Groveport Madison Franklin Passed New 53.37 46.63
Hopewell-Loudon Seneca Passed Renewal 61.89 38.11
Howland Trumbull Failed New 44.79 55.21
Indian Lake Logan Failed Renewal 46.89 53.11
Jackson Stark Passed Renewal 66.68 33.32
Jefferson Madison Passed New 50.17 49.83
Jefferson County JVS Jefferson Passed Renewal 61.12 38.88
Johnstown-Monroe Licking Passed New 53.76 46.24
Jonathan Alder Madison Passed New 51.97 48.03
Keystone Lorain Failed New 40.73 59.27
Kirtland Lake Passed Renewal 60.55 39.45
Lakeview Trumbull Passed Renewal 60.37 39.63
Lexington Richland Passed Renewal 71.25 28.75
Liberty Trumbull Passed Renewal 54.51 45.49
Liberty Trumbull Passed Renewal 51.01 48.99
Liberty Center Henry Passed New 56.58 43.42
Liberty Union-Thurston Fairfield Passed Renewal 50.33 49.67
Logan Elm Pickaway Passed New 55.03 44.97
Loveland Hamilton Passed New 55.45 44.55
Madison Lake Failed New 31.16 68.84
Madison-Plains Madison Passed Renewal 57.45 42.55
Manchester Summit Passed Renewal 61.78 38.22
Mathews Trumbull Failed New 41.67 58.33
McComb Hancock Passed Renewal 78.33 21.67
Mechanicsburg Champaign Failed Renewal 46.79 53.21
Middletown Butler Passed New 50.48 49.52
Millcreek-West Unity Williams Passed New 51.31 48.69
Mogadore Summit Failed New 49.10 50.90
Mohawk Wyandot Passed Renewal 56.46 43.54
Mount Gilead Morrow Failed New 46.60 53.40
National Trail Preble Passed Renewal 63.29 36.71
New Bremen Auglaize Passed Renewal 64.68 35.32
New Philadelphia Tuscarawas Failed New 49.88 50.12
Newbury Geauga Failed New 48.12 51.88
North Central Williams Passed New 63.16 36.84
North Royalton Cuyahoga Failed New 47.39 52.61
North Union Union Passed Renewal 61.88 38.12
Northridge Licking Passed Renewal 55.98 44.02
Northwest Stark Passed Renewal 53.48 46.52
Northwood Wood Passed New 50.64 49.36
Norwalk Huron Passed New 60.89 39.11
Olmsted Falls Cuyahoga Failed New 49.45 50.55
Osnaburg Stark Failed New 38.34 61.66
Parkway Mercer Passed Renewal 67.60 32.40
Parma Cuyahoga Passed Renewal 55.41 44.59
Patrick Henry Henry Passed Renewal 53.13 46.87
Perkins Erie Failed New 42.36 57.64
Perry Allen Passed Renewal 52.14 47.86
Pike-Delta-York Fulton Passed Renewal 52.36 47.64
Piqua Miami Passed Renewal 60.71 39.29
Pleasant Marion Failed New 48.05 51.95
Poland Mahoning Passed Renewal 60.77 39.23
Ravenna Portage Failed New 45.15 54.85
Ridgedale Marion Failed New 42.20 57.80
Ripley-Union-Lewis- Brown Failed New 48.53 51.47
Rittman Wayne Passed Renewal 78.12 21.88
River Valley Marion Failed New 46.57 53.43
Riverside Lake Passed Renewal 56.06 43.94
Rootstown Portage Passed Renewal 60.90 39.10
Russia Shelby Passed Renewal 72.33 27.67
Sandusky Erie Passed Renewal 69.86 30.14
Shaker Heights Cuyahoga Passed New 62.49 37.51
Sheffield-Sheffield Lake Lorain Passed Renewal 66.53 33.47
Sheffield-Sheffield Lake Lorain Passed Renewal 66.34 33.66
Sidney Shelby Passed Renewal 62.97 37.03
Southeast Portage Passed Renewal 69.80 30.20
Southern Perry Failed New 42.97 57.03
Southington Trumbull Failed New 39.27 60.73
Southwest Licking Licking Failed New 45.88 54.12
Springfield Mahoning Failed Renewal 48.38 51.62
St. Henry Consolidated Mercer Passed Renewal 71.09 28.91
St. Marys Auglaize Passed Renewal 69.55 30.45
Swanton Fulton Passed Renewal 56.31 43.69
Sylvania Lucas Failed New 47.57 52.43
Tecumseh Clark Passed Renewal 60.52 39.48
Tecumseh Clark Passed Renewal 58.85 41.15
Tiffin Seneca Passed Renewal 66.33 33.67
Tipp Miami Passed Renewal 60.33 39.67
Tri-County North Preble Passed New 59.27 40.73
Tri-Rivers JVS Marion Failed Renewal 49.69 50.31
Tri-Rivers JVS Marion Failed Renewal 37.05 62.95
Triad Champaign Failed New 36.46 63.54
Trimble Athens Passed New 51.56 48.44
Triway Wayne Passed Renewal 55.36 44.64
Tuscarawas Valley Tuscarawas Passed New 52.16 47.84
Union Belmont Failed New 41.25 58.75
Union-Scioto Ross Passed Renewal 59.01 40.99
United Columbiana Passed Renewal 65.85 34.15
Van Buren Hancock Failed New 38.19 61.81
Walnut Township Fairfield Failed New 21.98 78.02
Warren Washington Passed Renewal 63.89 36.11
Wellington Lorain Passed Renewal 60.21 39.79
Wellsville Columbiana Passed Renewal 73.09 26.91
West Branch Mahoning Failed New 29.31 70.69
West Liberty-Salem Champaign Failed Renewal 46.20 53.80
Willard Huron Passed Renewal 54.44 45.56
Woodridge Summit Passed Renewal 59.80 40.20
Zanesville Muskingum Failed Renewal 48.36 51.64

Share your comment:

Student Outcomes Harmed by Inequitable Funding

Why does the defunding of public education by Governor Kasich matter? Because it harms student outcomes more than any other controllable event according to new research
There exists an increasing body of evidence that substantive and sustained state school finance reforms matter for improving both the level and distribution of short term and long run student outcomes. A few studies have attempted to tackle school finance reforms broadly applying multi-state analyses over time. Card and Payne (2002) found “evidence that equalization of spending levels leads to a narrowing of test score outcomes across family background groups.” (p. 49) Jackson, Johnson and Persico (2014) use data from the Panel Study of Income Dynamics (PSID) to evaluate long term outcomes of children exposed to court-ordered school finance reforms, based on matching PSID records to childhood school districts for individuals born between 1955 and 1985 and followed up through 2011. They find that the “Effects of a 20% increase in school spending are large enough to reduce disparities in outcomes between children born to poor and non‐poor families by at least two‐thirds,” and further that “A 1% increase in per‐pupil spending increases adult wages by 1% for children from poor families.”
On balance, it is safe to say that a significant and growing body of rigorous empirical literature validates that state school finance reforms can have substantive, positive effects on student outcomes, including reductions in outcome disparities or increases in overall outcome levels. Further, it stands to reason that if positive changes to school funding have positive effects on short and long run outcomes both in terms of level and distribution, then negative changes to school funding likely have negative effects on student outcomes. Thus it is critically important to understand the impact of the recent recession on state school finance systems, the effects on long term student outcomes being several years down the line.

Share your comment:

Gates Foundation opposes release of teachers’ VAM scores

Join the Future was highly critical of the Plain Dealer and NPR when they published a small set of teachers value-add scores. Recently in Florida, a court case forced the Florida Department of Education to publish the value-add scores of their teachers too. In response to this, the Gates Foundation, via their K-12 Education Director, Vicki Phillips, issued this statement condemning this kind of public shaming

We at the Bill & Melinda Gates Foundation oppose the public release of individual teacher information because there is no evidence to suggest it will lead to improvement in teacher performance. It will not attract new intelligent and passionate individuals to the profession, and it will not make our schools more effective.

Teacher performance evaluations can only translate into better schools when they deliver concrete, personalized feedback that helps each individual teacher become more effective. These evaluations should include feedback from a variety of sources, and, perhaps most obviously, these individual evaluations must be kept private. As we’ve seen before, publicizing individual performance data may undermine the very work that could help empower teachers and improve our schools.

Share your comment:

Govermnent Data proves Governor Continues to Shrink Education Budget

Steve Dyer recently crunched the education budget numbers produced by LSC, and produced the table below. It shows the on going defunding of public education in Ohio by Governor Kasich. Since 2010 education funding has shrunk year-on-year from 21.7% of state funds to 19.4% this year - despite increased monies from the economic recovery being available.

Dyer notes
If this budget had the same commitment as the one prior to Kasich taking office, Ohio would have spent $1.3 billion more on education in FY15 than it actually is spending.

FY10 FY11 FY12 Fy13 FY14 FY15
Total ED Funding $12,064,962,135 $11,914,583,939 $11,280,775,954 $10,879,823,770 $11,520,765,322 $12,013,518,929
Total All Funds $55,676,487,088 $54,883,477,824 $52,854,463,737 $53,427,114,306 $59,484,815,814 $61,636,877,244
% of Funds to Education 21.7% 21.7% 21.3% 20.4% 19.4% 19.5%

Share your comment:

Get Involved

 First Name
 Last Name
 School District