Science Fact

Corporate education reform science fiction, is having an unintended(?) science fact effect.

First the science

If VAM scores are at all accurate, there ought to be a significant correlation between a teacher's score one year compared to the next. In other words, good teachers should have somewhat consistently higher scores, and poor teachers ought to remain poor. He created a scatter plot that put the ratings from 2009 on one axis, and the ratings from 2010 on the other axis. What should we expect here? If there is a correlation, we should see some sort of upward sloping line.

There is one huge takeway from all this. VAM ratings are not an accurate reflection of a teacher's performance, even on the narrow indicators on which they focus. If an indicator is unreliable, it is a farce to call it "objective."

This travesty has the effect of discrediting the whole idea of using test score data to drive reform. What does it say about "reformers" when they are willing to base a large part of teacher and principal evaluations on such an indicator?

That travesty is now manifesting itself in real personal terms.

In 2009, 96 percent of their fifth graders were proficient in English, 89 percent in math. When the New York City Education Department released its numerical ratings recently, it seemed a sure bet that the P.S. 146 teachers would be at the very top.

Actually, they were near the very bottom.
[...]
Though 89 percent of P.S. 146 fifth graders were rated proficient in math in 2009, the year before, as fourth graders, 97 percent were rated as proficient. This resulted in the worst thing that can happen to a teacher in America today: negative value was added.

The difference between 89 percent and 97 percent proficiency at P.S. 146 is the result of three children scoring a 2 out of 4 instead of a 3 out of 4.

While Ms. Allanbrook does not believe in lots of test prep, her fourth-grade teachers do more of it than the rest of the school.

In New York City, fourth-grade test results can determine where a child will go to middle school. Fifth-grade scores have never mattered much, so teachers have been free to focus on project-based learning. While that may be good for a child’s intellectual development, it is hard on a teacher’s value-added score.

These teachers are not the only ones.

Bill Turque tells the story of teacher Sarah Wysocki, who was let go by D.C. public schools because her students got low standardized test scores, even though she received stellar personal evaluations as a teacher.

She was evaluated under the the D.C. teacher evaluation system, called IMPACT, a so-called “value-added” method of assessing teachers that uses complicated mathematical formulas that purport to tell how much “value” a teacher adds to how much a student learns.

As more data is demanded, more analysis can be done to demonstrate how unreliable it is for these purposes, and consequently we are guaranteed to read more stories of good teachers becoming victims of bad measurements. It's unfortunate we're going to have to go through all this to arrive at this understanding.

Simple thinking, bad reporting

Providing a broad based education for K-12 is a very complex endeavor. It's that complexity which makes it difficult to distill what policies work and what don't, when so many variables affect student outcomes.

However, a cottage industry is being developed to reduce the entire topic of public education to a letter grade or a single number. It's as though these architects of simplicity have read Hitchhikers Guide To The Galaxy and determined that maybe because the answer to "the earth, universe and everything" is 42, it's should be even easier to grade a school as a simple C, or a teacher to a 1.07.

The problem of course is that much like Douglas Adams' novel, this thinking is also science fiction. To distill complexities down to such simple terms, means making highly subjective decisions and ignoring, or worse, being oblivious to, multiple variables.

A few cases in point. Ohio is about to deploy a new school rating system, based upon subjective measures, and ignoring a host of other factors.

As an educator and parent I could rail for days about the lack of actual meaning behind any letter grade, whether an A or an F, and this is a decision that even Rick Santorum would call anachronistic. If your child brings home a ‘C’ on his report card, what does that mean? Does than mean he’s working his ass off and completing all of his homework but struggling with expressing his knowledge on written tests? Or does it mean that he’s uninterested in completing homework that isn’t challenging him while attaining perfect scores on assessments? Or does it just mean that he is earning consistent C’s on every single assignment whether in-class, homework, quizzes, or tests? Perhaps it’s some combination of the above, so what does that tell you or your son about what he needs to do to improve?

See how clear those letter grades are?

Separating the effects of economic factors from school performance doesn't appear to have been one of the major efforts undertaken, even though we have known for a long time that a students socioeconomic status, and that of the school district is the leading predictor of performance, by far.

Another recent example has been the use of teacher level value add scores by New York's print media

The article is a pretty remarkable display of both poor journalism and poor research. The reporters not only attempted to do something they couldn’t do, but they did it badly to boot. It’s unfortunate to have to waste one’s time addressing this kind of thing, but, no matter your opinion on charter schools, it’s a good example of how not to use the data that the Daily News and other newspapers released to the public.
[...]
However, we can’t take the performance categories – or the Daily News’ “analysis” of them – at face value. Their approach has one virtue – it’s easy to understand, and easy to do. But it has countless downsides, one of them being that it absolutely cannot prove – or even really suggest – what they’re saying it proves. I don’t know if the city’s charter teachers have higher value-added scores. It’s an interesting question (by my standards, at least), but the Daily News doesn’t address it meaningfully.

Though far from the only one, the reporters’ biggest problem was right in front of them. The article itself notes that only about half (32) of the city’s charter schools chose to participate in the rating program (it was voluntary for charters). This is actually the total number of participating schools in 2008, 2009 and 2010, most of which rotated in and out of the program each year. It’s apparently lost on these reporters that only a minority of charters participating means that the charter teachers in the TDR data do not necessarily reflect the population overall. This issue by itself renders their assertions invalid and irresponsible.

Simple thinking, and bad education reporting is a major impediment to real education reform that will improve the quality of our schools.

Why is it that a politician, such as Mayor Frank Jackson, can put forward plans to eliminate teacher seniority, and it not be pointed out that teaching experience matters, and that if his goal is to improve the quality of Cleveland's public schools, his chosen policy preference is antithetical to that?

Research suggests that students learn more from experienced teachers (those with at least five years of experience) than they do from less experienced teachers (NCES 2000d; Rivkin, Hanushek, and Kain 1998; Murnane and Phillips 1981.) These studies point primarily to the difference between teachers with fewer than five years of experience (new teachers) and teachers with five or more years of experience.

If that wasn't simple enough, of course there are even more complex answers to this question.

But these overall findings ignore the fact that the experience/achievement relationship differs a great deal by context. For instance, the returns to experience appear to vary by where teachers work. The relationship is more consistent among elementary school teachers (especially compared with those in high schools). The effect of experience on teacher productivity may also be mediated by the quality of their peers in the same school – i.e., that novice teachers with more effective peers in the same school do better.

Similarly, there is evidence that experience matters less – or less consistently – in poorer schools (also see here). There are several plausible explanations for this discrepancy, such as the possibility that teachers in poorer schools burn out more rapidly, or that there are difficulties in teaching lower-income children that are harder to adjust to.

The experience factor not only varies by where you teach, but also by what you teach. Math teachers seem to improve more quickly (and consistently) than reading teachers, while newer evidence suggests that the same is true for teachers who remain in the same grade for multiple years.

Finally, it bears mentioning the obvious point that the effect of teacher experience might be totally different if we were able to look at outcomes other than test scores. The idea that experience doesn’t matter after five or so years incorrectly implies that test scores are the only relevant outcome. Nobody believes that is the case. (And, for what it’s worth, teachers with whom I’ve spoken find the idea that they stop improving after four or five years laughable.)

Instead the debate over the Mayor's plan has not revolved around whether it has any basis in supportable fact, but instead around simplistic stories of the politics involved.

There are enough bad actors in the corporate school reform movement willing to put aside hard truths and solid facts in favor of their desire to profit from public education, but that should be no excuse for others to not challenge simplistic thinking and unsupported asertions which can be equally as damaging to the goal of delivering a quality education to all students.

Cleveland School Plan Needs Work

This is a very level headed and reasonable approach to modify Frank Jackson's corporate education reform plan into something that might work and would bring more people willingly into the process.

Columbus: Innovation Ohio, a progressive think tank headquartered in Columbus, today released an analysis of the education reform plan recently put forward by Cleveland Mayor Frank Jackson. Governor Kasich has indicated the plan might serve as a model for his own education reform effort, which presumably will include the new school funding formula he promised but so far has failed to deliver. The analysis is available at www.innovationohio.org.

IO said an analysis of the “Cleveland Plan” is important given Ohio’s history of expanding Cleveland education experiments, such as private school vouchers, state-wide. “If Governor Kasich is intent on using the Cleveland Plan as a model for other Ohio school districts, then it’s critical that we get it right,” said IO President Janetta King.

The analysis found a number of “things to like” about the Cleveland Plan, including:

• Innovations such as a Global Language Academy, an Environmental Science School, Early Childhood Education Academies in every neighborhood, and an English Immersion School for all children for whom English is a second language;
• A focus on high-quality preschool education, as well as on college and workforce readiness; and
• A series of proposed changes to state law that would, for example, give the Cleveland Metropolitan School District flexibility to manage its fiscal assets and close loopholes in existing law that allow poorly-performing Charter Schools to continue operating.

IO said other ideas, like adoption of a year-round school calendar, support for high-quality Charter Schools, and the aggressive pursuit of talented teachers, “have potential, but need more work and further fleshing-out.”

But Innovation Ohio said several Cleveland Plan ideas are fatally flawed as currently written and should either be modified substantially or jettisoned entirely. Among these are:

• A proposal to allow the transfer of local property tax revenue to Charter schools;
• The transfer of school oversight and other functions from the Cleveland School Board (accountable to the Mayor) to an unelected and less accountable “Cleveland Transformation Alliance”;
• A weighted per pupil funding formula with “money following the child” that, in IO’s view, would inevitably end up short-changing either students or schools;
• Several proposals relating to teacher compensation, collective bargaining and accountability, which IO says are exact replicas of provisions in last year’s Senate Bill 5, which Ohio voters overwhelmingly rejected with 61% of the vote in November.

Said IO President Janetta King:

“IO congratulates the authors of the Cleveland Plan for thinking outside the box and being willing to go big. Nothing is more important to Ohio’s future than our schools and our kids. That’s why education reform is so important, and it’s why all of us who truly care about our state, Republicans and Democrats, conservatives, liberals and moderates alike–must be willing to embrace change and challenge the status quo.

“But our goal cannot be change for the sake of change, or change that can’t work and will only make things worse. So Innovation Ohio has tried to be constructive in our analysis. Where we’ve been critical of the Cleveland Plan, we’ve offered alternative ideas and proposals that we believe are more likely to achieve the desired goals.

“But we recognize that we don’t have all the answers. Frankly, neither do the people who put the Cleveland Plan together. And that is why we believe any serious school reform discussion should and must include the voices of professional educators, parents, and other members of the community. We hope their exclusion will be rectified in the weeks and months ahead.

“So what is Innovation Ohio’s bottom-line take on the Cleveland Plan? We believe the Plan as written is a reasonable place to start, but would be a terrible place to end up. It needs work and IO stands ready to help any way we can.”

Survey: Teacher Job Satisfaction Drops to New Low

Via NEA

Teachers are less satisfied with their jobs than they have been in decades, according to the 2012 MetLife Survey of the American Teacher. Almost one-third of teachers are thinking of leaving the profession they love – due in part to the unconscionable cuts in education funding. NEA President Dennis Van Roekel described this finding as “shocking” and said it was clear evidence that ill-conceived economic policies are having devastating consequences on teachers and students across the country. More than three quarters of the teachers surveyed reported that their school’s budget had decreased.

“I have heard similar concerns from NEA members,” Van Roekel said. “They have told me that staff and important programs have been cut; early childhood education has been eliminated; computers and text-books were out of date; and classes such as history, art, PE and music—which provide a well-rounded education—are no longer offered.”

The MetLife Survey of the American Teacher: Teachers, Parents and the Economy, the 28th in an annual series commissioned by MetLife and conducted by Harris Interactive, examines the views of teachers, parents and students about the teaching profession, parent and community engagement, and effects of the economy on teaching and learning in schools.

Two-thirds of the teachers surveyed reported that layoffs of teachers, staff and parent/community liaisons occurred at their school in 2011, and three-quarters have experienced budget cuts in their schools in the last 12 months. The survey also found that teachers and parents of students in these cash-strapped schools are more likely to be pessimistic that student achievement will be better in five years than are teachers and parents of students in schools where budgets have remained the same or increased.

“This is not the way America should treat its students, the vast majority of whom attend public schools. And it is especially outrageous to students in schools of greatest need, “Van Roekel said.

The bright spot in the survey is that parent and community engagement with schools has increased. For example, fewer teachers and parents now believe that there is widespread parental disengagement with their children’s school and education in general.

Overall, the survey found that a majority of both teachers (77 percent) and parents (71 percent) agree that teachers are treated as professionals by the community. In addition, parents of students in schools with high parent engagement are more likely than those with low engagement to rate their child’s teachers as “excellent” or “good” on a range of measures.

Increasing parent and family involvement is an NEA priority and a top strategy of NEA’s Priority Schools Campaign, which focuses on schools in low-income areas.

“The survey’s findings underscore that education is a shared responsibility, particularly in the face of financial challenges,” said Dennis White, vice president of corporate contributions for MetLife. “Economic prosperity will depend on a new generation well-prepared to learn for a lifetime in order to compete and collaborate in a global economy.”

Read the Complete Survey

Value-Added Evaluation Hurts Teaching

Here’s the hype: New York City’s “worst teacher” was recently singled out and so labeled by the New York Post after the city’s education department released value-added test-score ratings to the media for thousands of city teachers, identifying each by name.

The tabloid treatment didn’t stop there. Reporters chased down teacher Pascale Mauclair, the subject of the “worst teacher” slam, bombarding her with questions about her lack of skill and commitment. They even went to her father’s home and told him his daughter was among the worst teachers in the city.

Now the facts: Mauclair is an experienced and much-admired English-as-a-second-language teacher. She works with new immigrant students who do not yet speak English at one of the city’s strongest elementary schools. Her school, PS 11, received an A from the city’s rating system and is led by one of the city’s most respected principals, Anna Efkarpides, who declares Mauclair an excellent teacher. She adds: “I would put my own children in her class.”

Most troubling is that the city released the scores while warning that huge margins of error surround the ratings: more than 30 percentile points in math and more than 50 percentile points in English language arts. Soon these scores will be used in a newly negotiated evaluation system that, as it is designed, will identify most teachers in New York state as less than effective.

Is this what we want to achieve with teacher-evaluation reform?

Everyone agrees that teacher evaluation in the United States needs an overhaul. Although successful systems exist, most districts are not using approaches that help teachers improve or remove those who cannot improve in a timely way. Clearly, we need a change.

[readon2 url="http://www.edweek.org/ew/articles/2012/03/05/24darlinghammond_ep.h31.html?tkn=XSLF8dtEku7dtKu1xSZfeIqk1QmVLXJ5Fp0i&cmp=clp-edweek"]Continue reading...[/readon2]

The Missing Link In Genuine School Reform

The big "reform" trucks have been rollin' down the education highway for nearly a decade now. Public school educators are used to faux reform's inconvenience and injustice by now--and some even accept endless testing, lockstep standards and curriculum, and systematic destruction of public schools as necessary for positive change. Parents and grandparents may like their children's schools and teachers, but have absorbed the incessant media drumbeat: public education has failed. Out with the old! Something Must Be Done!

If--like me--you still believe that public education is a civic good, an idea perfectly resonant with democratic equality, you're probably wondering if there's anything that can stop the big "reform" trucks. Those massive, exceptionally well-funded "reform" trucks with their professional media budgets, paid commentary and slick political arms.

I can tell you this: it won't be teachers alone who turn back the tide of "reform." Teachers have been backed into a corner, painted as unionists bent on their own security (whether they pay dues or not), unwilling to be "accountable." They have been replaced, willy-nilly, by untrained temps--without retaliatory strike-back from their national union leaders. They have been publicly humiliated by their own cities and media outlets, not to mention the Secretary of Education.

Besides, teaching--as an occupational cluster--tends to attract those who liked school and believe sincerely in the power of public education to do good in society. They're nurturers--four-fifths of them women, many with children and family responsibilities as well as professional careers. It's not surprising when teachers keep their heads down and follow orders, even when their hearts aren't in it.

[readon2 url="http://blogs.edweek.org/teachers/teacher_in_a_strange_land/2012/03/the_missing_link_in_genuine_school_reform.html"]Continue reading...[/readon2]