data

OEA Response to PD and NPR Teacher shaming

Here's the statement from the Ohio Education Association, which represents over 121,000 educators

Responding to a series of newspaper, web and radio stories on value-added scroes of individual Ohio teachers, Patricia Frost-Brooks, President of the Ohio Education Association criticized the fairness of the stories and the wisdom of using value-added scores as such a prominent index of teacher success:

"The Ohio Education Association was not contacted for comment on the Plain Dealer/StateImpact Ohio stories, despite our expertise, which would have provided desperately needed context and perspective. Reporters and editors admitted this value-added data was 'flawed,' but they chose surprise and impact over fairness, balance and accuracy," Frost-Brooks said.

"We are all accountable for student success – teachers, support professionals, parents, students and elected officials. And the Ohio Education Association is committed to fair teacher evaluation systems that include student performance, among other multiple measures. But listing teachers as effective or ineffective based on narrow tests not designed to be used for this purpose is a disservice to everyone.

"Value-added ratings can never paint a complete or objective picture of an individual teacher’s work or performance. Trained educators can use a student’s value-added data, along with other student data, to improve student instruction. But the stories promote a simplistic and inaccurate view of value-added as a valid basis for high-stakes decisions on schools, teachers and students."

Very questionable that reporters would not contact the largest teachers assoication in crafting their story.

Battelle Blasts Papers decision

From our mailbag, Battelle for Kids condemns the Plain Dealer and NPR's decision to publish teacher's value-added scores, calling it "the poster child for name, blame, and shame and the antithesis of our approach to using value-added data"

To: All SOAR districts
From: Jim Mahoney and Bobby Moore
Date: June 17, 2013

Yesterday, a three-part series on value-added was launched by The Cleveland Plain Dealer and State Impact Ohio. It includes both articles and radio segments specific to value-added analysis as a measure of teacher effectiveness. Highlighted in the articles is a link to a database of teacher ratings, hosted by The Plain Dealer and the State Impact Ohio partnership.

Currently, Ohio laws governing the release of teacher records would apply to teacher value-added results. Thus, teacher level value-added information is subject to public records requests through ODE. Through The Plain Dealer and State Impact Ohio database, the general public can now access a teacher's overall composite rating derived from two years of his/her results in grades 4-8 math and reading. These data reflect information for less than 1/3 of the math and reading, grades 4-8 teachers in Ohio.

Battelle for Kids was not aware these ratings would be published in this way, at this time.

While Battelle for Kids does support the use of value-added information for school improvement and as one of several components of a multi-measures evaluation system, value-added should NOT be used in isolation to draw conclusions about a teacher's effectiveness.

Multiple data points over time from multiple perspectives are crucial because teaching and learning and the evaluation of teaching and learning are complex.

Therefore, we are NOT supportive of these ratings being publically available and discourage promoting the use of this public database.

Talking points and articles, to support your local conversations, are available on the Ohio Student Progress Portal.

http://cts.vresp.com/c/?BattelleForKids/f43a0e1b46/fb8aa9ca4e/313346eb88/sflang=en

Obviously, this is the poster child for name, blame, and shame and the antithesis of our approach to using value-added data.

Please call if you have any questions.

Thank you for all you do for Ohio's students!

-Jim and Bobby

Shame on the PD and NPR

When the Cleveland Plain Dealer and NPR decided to publish the names of 4,200 Ohio teachers and their value-added grades, their reasoning was specious and self-serving. Most of all, it is damaging to the teaching profession in Ohio.

Despite pointing out all the flaws, caveats, and controversies with the use of value-add as a means to evaluate teachers, both publications decided to go ahead and shame these 4,200 teacher anyway. The publication of teachers names and scores isn't new. It was first done by the LA Times, and was a factor in the suicide of one teacher. The LA Times findings and analysis was then discredited

The research on which the Los Angeles Times relied for its August 2010 teacher effectiveness reporting was demonstrably inadequate to support the published rankings. Using the same L.A. Unified School District data and the same methods as the Times, this study probes deeper and finds the earlier research to have serious weaknesses.

DUE DILIGENCE AND THE EVALUATION OF TEACHERS by National Education Policy Center

The Plain Dealer analysis is weaker than the LA Times, relying on just 2 years worth of data rather than 7. In fact, the Pleain Dealer and NPR stated they only published 4,200 teachers scores and not the 12,000 scores they had data for because most only had 1 years worth of data. A serious error as value-add is known to be highly unreliable and subject to massive variance.

Beyond the questionable statistical analysis, the publication of teachers names and value-added scores has been criticized by a great number of people, including corporate education reformer Bill Gates, in NYT op-ed titled "Shame Is Not the Solution"

LAST week, the New York State Court of Appeals ruled that teachers’ individual performance assessments could be made public. I have no opinion on the ruling as a matter of law, but as a harbinger of education policy in the United States, it is a big mistake.

I am a strong proponent of measuring teachers’ effectiveness, and my foundation works with many schools to help make sure that such evaluations improve the overall quality of teaching. But publicly ranking teachers by name will not help them get better at their jobs or improve student learning. On the contrary, it will make it a lot harder to implement teacher evaluation systems that work.

Gates isn't the only high profile corporate education reformer who is critical of such shaming, Wendy Knopp, CEO of Teach for America has also spoken out against the practice

Kopp is not shy about saying what she'd do differently as New York City schools chancellor. While the Bloomberg administration is fighting the United Federation of Teachers in court for the right to release to the news media individual teachers' "value added" ratings—an estimate of how effective a teacher is at improving his or her students' standardized test scores—Kopp says she finds the idea "baffling" and believes doing so would undermine trust among teachers and between teachers and administrators.

"The principals of very high performing schools would all say their No. 1 strategy is to build extraordinary teams," Kopp said. "I can't imagine it's a good organizational strategy to go publish the names of teachers and one data point about whether they are effective or not in the newspaper."

Indeed, if the editors of the Plain Dealer and NPR had read their own reporting, they would have realized the public release of this information was unsound, unfair and damaging. Let's look at the warning signs in their own reporting

...scores can vary from year to year.

Yet they relied upon only 1 years worth of data for much of their analysis, and just 2 for the teachers whose names they published.

...decided it was more important to provide information — even if flawed.

How can it be useful to the layperson to be provided with flawed information? Why would a newspaper knowingly publish flawed information?

...these scores are only a part of the criteria necessary for full and accurate evaluation of an individual teacher.

And yet they publish 4,200 teachers value-added scores based solely on value add, which at best makes up only 35% of a teachers evaluation. Lay people will not understand these scores are only a partial measurment of a teachers effectiveness, and a poor one at that.

...There are a lot of questions still about the particular formula Ohio.

Indeed, so many questions that one would best be advised to wait until those questions are answered before publically shaming teachers who were part of a pilot program being used to answer those questions.

...variables beyond a teacher’s control need to be considered in arriving at a fair and accurate formula.

Yet none of these reporters considered any of these factors in publishing teachers names, and readers will wholly miss that necassary context.

...The company that calculates value-added for Ohio says scores are most reliable with three years of data.

Again, the data is unreliable, especially with less than 3 years worth of data, yet the Plain Dealer and NRP decided they should shame teachers using just 2 years worth of data.

...Ohio’s value-added ratings do not account for the socioeconomic backgrounds of students, as they do in some other states.

How many "ineffective" teachers are really just working in depressed socioeconomic classrooms? The reporters seem not to care and publish the names anyway.

...Value-added scores are not a teacher’s full rating.

No where in the publication of these names are the teachers full ratings indicated. This again leaves lay-people and site visitors to think these flawed value-added scores are the final reflection of a teachers quality

...ratings are still something of an experiment.

How absurd is the decision to publish now seeming? Shaming people on the basis of the results of an experiement! By their very nature experiments can demonstrate something is wrong, not right.

...The details of how the scores are calculated aren’t public.

We don't even know if the value-added scores are correct and accurate, because the formula is secret. How can it be fair for the results of a secret forumla be public? Did that not rasie any alarm bells for the Plain Dealer and NPR?

...The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works.

But somehow NPR listeners and Cleveland Plain Dealer readers are supposed to understand the complexities, and read the necessary context into the publication of individual teacher scores?

...StateImpact/Plain Dealer analysis of initial state data suggests.

"Initial", "Suggests". They have decided to shame teachers without properly vetting the data and their own analysis - exactly the same problem the LA Times ran into that we highlighted at the top of this article.

It doesn't take a lot of "analysis" to understand that a failing newspaper needed controversy and eyeballs and that their decision to shame teachers was made in their own economic interests and not that of the public good. In the end then, the real shame falls not on teachers who are working hard everyday often in difficult situations made worse by draconian budget cuts, endless political meddling, and student poverty - but on the editors of these 2 publications for putting their own narrow self-interest above that of Ohio's children.

It's a disgrace that they ought to make 4,200 apologies for.

The Arbitrary Albatross: Standardized Testing and Teacher Evaluation

On Chicago's streets and Hollywood's silver screens, education reform has been cast as a false dilemma between students and teachers. Reputable actresses and liberal mayors have both fallen prey. At the center of this drama lie teacher evaluations. A linchpin of the debate, they weigh especially heavily around the necks of educators like me.

Think: Shaky Foundation

With the arrival of spring, testing season is now upon us: America's new national pastime. I believe student results from standardized tests should not be used to evaluate teachers because the data are imprecise and the effects are pernicious. Including such inaccurate measures is both unfair to teachers and detrimental to student learning.

As a large body of research suggests, standardized test data are imprecise for two main reasons. First, they do not account for individual and environmental factors affecting student performance, factors over which teachers have no control. (Think: commitment, social class, family.) Second, high-stakes, one-time tests increase the likelihood of random variation so that scores fluctuate in arbitrary ways not linked to teacher efficacy. (Think: sleep, allergies, the heartache of a recent breakup.)

High-stakes assessments are also ruinous to student learning. They encourage, at least, teaching to the test and, at most, outright cheating. This phenomenon is supported by Campbell's law, which states statistics are more likely to be corrupted when used in making decisions, which in turn corrupts the decision making process itself. (Think: presidential campaigns.)

As a teacher, if my livelihood is based on test results, then I will do everything possible to ensure high marks, including narrowing the curriculum and prepping fiercely for the test. The choice between an interesting project and a paycheck is no choice at all. These are amazing disincentives to student learning. Tying teachers' careers to standardized tests does not foster creative, passionate, skillful young adults. It does exactly the opposite.

[readon2 url="http://www.edutopia.org/blog/standardized-testing-and-teacher-evaluation-aaron-pribble"]Continue reading...[/readon2]

Education News for 05-28-2013

State Education News

  • Coventry schools lead state in financial stress, but may have righted the ship (Akron Beacon Journal)
  • For nearly as long as Ohio has been rating school districts based on their financial health, Coventry schools has been in fiscal watch — the second-lowest level possible…Read more...

  • Bulletproof backpacks used to protect some school children (Cleveland Plain Dealer)
  • Guns weren't the only thing people raced to buy after 20 students and six adults were killed at Sandy Hook Elementary School…Read more...

  • Columbus school board president clarifies stand on auditor (Columbus Dispatch)
  • Columbus Board of Education President Carol Perkins said this week that the district’s internal auditor’s post “is not going to be done away with,” but she issued a statement later saying that her words had been misinterpreted…Read more...

  • 3 referred to state in city schools data probe (Columbus Dispatch)
  • The Columbus school district has reported three former employees with ties to its data scandal to a state office that investigates educator wrongdoing…Read more...

  • Area schools to get $1.8M to meet new state requirements (Dayton Daily News)
  • Several districts and schools across the Miami Valley will get a $1.8 million boost to meet the state’s new early literacy and reading requirements, including two districts in Clark and all districts in Champaign counties…Read more...

  • State denies request for more information on attendance investigation (Dayton Daily News)
  • The Ohio Department of Education has denied the newspaper’s request to see Northridge Local Schools’ official written response to allegations that it scrubbed student attendance data…Read more...

  • Non-traditional high school graduates grows (Dayton Daily News)
  • The majority of Ohio public high school seniors still graduate from traditional schools but a growing number of them are graduating from non-traditional high schools…Read more...

  • Changes coming to GED; New standards, switch to digital format (Lorain Morning Journal)
  • It’s been three years since 21-year-old Larita White started working toward her GED, but her progress may be undone if she doesn’t receive the diploma before January…Read more...

  • State OKs plans to fix school data errors (New Philadelphia Times-Reporter)
  • Ohio schools that reportedly had errors in certain enrollment data have gotten state approval of plans to fix the issues…Read more...

  • It looks like there will be more money for more preschool in state (Ohio Public Radio)
  • Some conservative Ohio lawmakers and some faith leaders who generally support conservative causes want to put millions of additional dollars into more preschool…Read more...

Local Education News

  • Kids learning to COPE with stress (Columbus Dispatch)
  • Nothing gets 14-year-old Lyle Watters’ stomach tied up in knots as much as when his mom and stepdad argue…Read more...

  • Columbus schools may tap OSU provost as fill-in chief (Columbus Dispatch)
  • The Columbus school board president has asked Joseph Alutto, vice president and provost at Ohio State University, to lead Columbus City Schools until a permanent superintendent…Read more...

  • North Baltimore to evaluate digital device program (Findlay Courier)
  • Results of a pilot program that permitted North Baltimore students to use mobile electronic devices during school hours will be assessed by school officials this summer…Read more...

  • VB teachers, school begin contract talks (Findlay Courier)
  • Contract negotiations between Van Buren's teachers and administrators have begun and are proceeding well, Superintendent Tim Myers…Read more...

  • Board approves reading, math programs for 2013 (Toledo Blade)
  • At its regular meeting last week, the board of education approved the 2013 elementary intervention program, including its fees and instructors…Read more...

  • Board supports policy of student drug testing (Toledo Blade)
  • Fremont Ross High School students will undergo random drug testing next school year, following the school board’s support for a new drug testing policy…Read more...

  • District to provide iPads to students (Toledo Blade)
  • Seventh and eighth graders in Oregon schools will get a welcome amenity next year when classes start: a new iPad. The board of education last week approved a technology lease with Apple Inc. that will put an iPad…Read more...

Editorial

  • School board made good choice (Columbus Dispatch)
  • A public resolution by the Columbus Board of Education, expressing support for the recommendations of the Columbus Education Commission, is welcome…Read more...

  • Easy vote for lawmakers (Columbus Dispatch)
  • Ohio lawmakers have before them a simple piece of legislation that won’t cost a dime in state funds, has bipartisan sponsorship, enjoys the broadest possible community support and unites groups…Read more...

Why are Ohio's charter schools so poor?

In a previous post we took a look at the difference in performance between Ohio's traditional public schools and their charter school counterparts, and discovered a wide and growing gap. Why does this gap exist?

One of the clear reasons is in the quality of the workforce. Ohio's traditional public schools have invested a lot of time and money developing an experienced, highly qualified teacher workforce, an investment that Ohio's charter schools have resisted or failed to developing.

Steve Dyer at Innovation Ohio took a look at the latest teacher data made available by ODE and discovered that the typical (i.e. the mode) charter school teacher has 0 years of experience, while their traditional school counterpart had 14 years of experience. We took a look at this too, and came to the same conclusion.

Our analysis also showed that the average level of experience of a charter school teacher is only 4.9 years, while in traditional schools that figure is over 14 years.

Dyer also discovered, what he called "two equally stunning statistics"

1) The average Traditional Public School building has about 2/3 of its teachers with a masters degree. The average Charter School building has about 1/3 of its teachers with a masters degree.
[...]
2) About 1 out of 3 Charter Schools in Ohio have at least some core courses taught by someone with a temporary teaching certificate. Of the more than 3,200 Traditional Public School buildings in the report card data, not a single one has core courses taught by teachers with temporary certificates.

Furthermore, over 10% of Ohio's charter schools have class sizes greater than 25 students per teacher, 20 of them more than 30 students per teacher!

The major innovation attempted by Ohio's charter school community has not been to find ways to deliver higher quality education, but instead to find ways to minimize teacher costs in order to maximize profits. This is made very clear when once looks at this experience and qualification differences, education outcome quality differences and the subsequent difference in average salaries where the charter school average is $33,993 and traditional schools $57,303.