Test Scores Often Misused In Policy Decisions

Education policies that affect millions of students have long been tied to test scores, but a new paper suggests those scores are regularly misinterpreted.

According to the new research out of Mathematica, a statistical research group, the comparisons sometimes used to judge school performance are more indicative of demographic change than actual learning.

For example: Last week's release of National Assessment of Educational Progress scores led to much finger-pointing about what's working and what isn't in education reform. But according to Mathematica, policy assessments based on raw test data is extremely misleading -- especially because year-to-year comparisons measure different groups of students.

"Every time the NAEP results come out, you see a whole slew of headlines that make you slap your forehead," said Steven Glazerman, an author of the paper and a senior fellow at Mathematica. "You draw all the wrong conclusions over whether some school or district was effective or ineffective based on comparisons that can't be indicators of those changes."

[readon2 url="http://www.huffingtonpost.com/2011/12/14/test-scores-often-misused_n_1149501.html?ref=education&ncid=edlinkusaolp00000008"]Continue reading...[/readon2]

The Evidence on Charter Schools and Test Scores

The Shanker Institute has just released a policy brief titled "The Evidence on Charter Schools and Test Scores", which finds

The available research suggests that charter schools’ effects on test score gains vary by location, school/student characteristics and other factors. When there are differences, they tend to be modest. There is tentative evidence suggesting that high-performing charter schools share certain key features, especially private donations, large expansions of school time, tutoring programs and strong discipline policies. Finally, while there may be a role for state/local policies in ensuring quality as charters proliferate, scaling up proven approaches is constrained by the lack of adequate funding, and the few places where charter sectors as a whole have been shown to get very strong results seem to be those in which their presence is more limited. Overall, after more than 20 years of proliferation, charter schools face the same challenges as regular public schools in boosting student achievement, and future research should continue to focus on identifying the policies, practices and other characteristics that help explain the wide variation in their results.

Charter Review

A doomsday bill for public ed

Innovation Ohio follow up on the news that the public school privatization bill (HB136) is going to change, with some interesting new analysis on what some of the proposed changes might mean

Huffman admitted that in its current form, HB 136 could create a potential “doomsday scenario” for Ohio’s public schools. He said that estimates of the bill’s cost to public schools—$500 million just to pay for students who already attend private schools, and nearly $1 billion in lost revenue once the bill was fully phased in—were “valid.”

The good news is that Huffman announced that a scaled back version of HB 136 would either be introduced as a new bill or rolled into a new education plan Gov. Kasich is expected to submit next year.

The bad news is that even this “scaled back” version would cost districts up to $76.5 million, effectively doubling the cost of private school vouchers from last year.

And many of Huffman’s proposed changes would still present public schools with massive problems.

Take his proposal to cap the amount of state money any district could lose at $4,500 per student, for example.

Under this proposal, locally raised property taxes would still be sent to private schools. Why? Because the $4,500 per pupil in state money Huffman cites is a phantom figure. Its what a district might get before “deductions” for Charter Schools, Open Enrollment, and other voucher programs. Last year, for example, the state provided public school districts with $6.5 billion—which equated to nearly $3,800 for each of the 1.75 million children in those districts. But after “deductions” for charters, vouchers, and open enrollment, that $3,800 shrunk to roughly $3,200. So if districts get $3,200 per pupil from the state—but could lost up to $4,500 per pupil for private school vouchers—they would still lose money. And that would require the schools to either cut programs or seek more funding from taxpayers through local levies.

You can read the whole piece here.

Teach for America ‘research’ questioned

Recently I exchanged emails with a Teach for America employee in my city. On my last exchange, I tried to press her to answer at least one of my questions.

"Given the choice, would you see a doctor with 5 weeks of training or a certified doctor? A lawyer? An actuary?"

Answering with a ‘yes’ would be absurd. Answering with a ‘no’ would indicate a blatant disrespect for teachers.

Unfortunately this disrespect is exactly what we have going on in our country at this time: a blame-the-teacher mentality that ignores real world issues and concerns.

The TFA employee directed me to the organization's "research" page where TFA claims this: "A large and growing body of independent research shows that Teach For America corps members make as much of an impact on student achievement as veteran teachers."

This claim, based on the "studies" supplied by TFA, is misleading at best and demonstrably false at worst. I read all of the 12 "studies" available on TFA's website, and here is what I found.

[readon2 url="http://www.washingtonpost.com/blogs/answer-sheet/post/teach-for-america-research-questioned/2011/12/12/gIQANb40rO_blog.html?wprss=answer-sheet"]Continue reading...[/readon2]

What teachers didn't tell the governor

The Governor's education Czar, Robert Sommers, and his assistant Sarah Dove have finally published their report based upon feedback received via a web form regarding their corporate education reform proposals. The report can be read below.

We don't need to mention how this report lacks any scientific validity, because the reports authors do that for us

This summary is not meant to be a scientific compilation of the information. It is intended, rather, to present the general sentiment of the productive comments received. It is acknowledged that in any particular category, comments were received that would range across the entire spectrum of pros and cons.

It's one of the few honest things said in this highly charged and political document. Despite admitting that the methodology of this study is not sound, almost every single section of this document begins with the phrase "Teachers believe". In many cases what is asserted that teachers believe is not even supported by the actual feedback teachers provided. Earlier in the year, in a 5 part series, we published many of the actual comments teachers provided as input to this process. You can find that series here:

• Day 1
• Day 2
• Day 3
• Day 4
• Day 5

The report concludes with recommendations from a "steering committee". But we're never told who served on that committee, only that

"Robert Sommers, Director of the Governor’s Office of 21st Century Education, and Sarah Dove, Ohio’s Teacher Liaison, assembled a steering committee consisting of a cross-section of teachers representing schools and educators across the state.

When a document presents recommendations, do readers not deserve to know who exactly are making these recommendations, what the process for approving them was, and if there was any dissension?UPDATE: Commitee list is burried at the end of the document in the appendix, with no mention of who each person is, or who they represent.

Furthermore, for a process that had very little stakeholder input at all, this recommendation stood out for its audacity

The Ohio Department of Education must commit to providing increased communications with teachers about new evaluation and compensation models.

Little effort has been expended by the Department of Education in educating teachers on where the state is and where it is headed in the areas of evaluation and compensation. By providing teachers with a “big picture” version of the state’s evaluation framework, the state can lay the groundwork for educated and committed teachers. The Department of Education must reach out and collaborate with key stakeholders to assist with getting the needed communications to teachers and leaders across the state. ODE should develop and implement a strategic communications plan to identify key messages, important milestones and identify who is responsible for sharing information.

We agree, but are left wondering why this wasn't done during the preparation of this document?

At the end of the day however the biggest question we are left with is this, what is the point and purpose of this document from the governor's office? The Department of Education has already released its framework for evaluations. The ESB has worked for 2 years on the details of an evaluation system and local school districts and education associations have been working together on developing systems to meet RttT requirements. This flimsy, unscientific, political document, developed by an unnamed steering committee has added nothing to any of these efforts.

Ohio Evaluation Comp Reform

HB136 is dead, but will return

HB136 is dead. But a watered down version is likely to reappear in the form of another bill, or tucked quietly inside a bigger bill dealing with the new school funding formula. That was the message delivered by it's sponsor Rep Matt Huffman at a news conference yesterday.

Huffman said he'll change the funding mechanism so that no voucher will be worth more than the amount of per-pupil state aid that the home district receives. In a high-wealth district, that could be as little as a few hundred dollars.

"The practical effect is that not a lot of students from that district will use them," he acknowledged.

Huffman also said he'll limit the number of available vouchers to 1 percent of the home district's enrollment, which is roughly how many he expected would participate.

Statewide, that would be up to 17,000 students - far fewer than would have been possible in the original version of the bill.

He also will change the eligibility guidelines, tying them to the income levels that qualify children for state health coverage. And he will eliminate a provision that would have allowed families to spend unused voucher money on private high schools or college.

We'd speculate that it is unlikely that this new framework would appear in a bigger bill dealing with the funding formula. With over 300 school districts continuing to oppose this privatization plan in any form, a funding formula bill having its own issues, is not going to be a welcoming place for a contentious piece of legislation to be tucked inside.

Whatever the final form and function of this privatization proposal, it remains a terrible policy to transfer public monies to private schools at the expense of the majority of students. Even more so in light of recent revelations of ODE's inability to perform its basic oversight and accountability functions of private schools already receiving tax dollars.