New Research Uncovers Fresh Trouble for VAM Evaluations

As more and more schools implement various forms of Value-Added method (VAM) evaluation systems, we are learning some disturbing things about how reliable these methods are.

Education Week's Stephan Sawchuk, in "'Value-Added' Measures at Secondary Level Questioned," explains that value-added statistical modeling was once limited to analyzing large sets of data. These statistical models projected students' test score growth, based on their past performance, and thus estimated a growth target. But, now 30 states require teacher evaluations to use student performance, and that has expanded use of algorithms for high-stakes purposes. Value-added estimates are now being applied to secondary schools, even though the vast majority of research on their use has been limited to elementary schools.

Sawchuk reports on two major studies that should slow this rush to evaluate all teachers with experimental models. This month, Douglas Harris will be presenting "Bias of Public Sector Worker Performance Monitoring." It is based on a six years of Florida middle school data on 1.3 million math students.

Harris divides classes into three types, remedial, midlevel, and advanced. After controlling for tracking, he finds that between 30 to 70% of teachers would be placed in the wrong category by normative value-added models. Moreover, Harris discovers that teachers who taught more remedial classes tended to have lower value-added scores than teachers who taught mainly higher-level classes. "That phenomenon was not due to the best teachers' disproportionately teaching the more-rigorous classes, as is often asserted. Instead, the paper shows, even those teachers who taught courses at more than one level of rigor did better when their performance teaching the upper-level classes was compared against that from the lower-level classes."

[readon2 url="http://blogs.edweek.org/teachers/living-in-dialogue/2012/11/john_thompson_new_research_unc.html"]Continue reading...[/readon2]

3rd grade retention plan could cost $500 million

One of the signature policy initiatives in the Kasich MBR is the proposed change in the 3rd grade reading guarantee.

Specifically, according to LSC analysis, the bill (SB316) makes several changes to the third grade reading guarantee beginning with the 2012-2013 school year. Under current law, the third grade reading guarantee requires school districts and community schools to retain in third grade a student who scores in the "limited" range on the third grade English language arts assessment, unless the student's principal and reading teacher agree that the student is academically prepared for fourth grade or the student will receive intervention services in fourth grade. The bill changes the "cut" score and applies the guarantee to all students who do not receive at least a "proficient" (or passing) score on the assessment. The "limited" score, which currently triggers the guarantee, is the lowest of five scoring ranges and two levels below "proficient."

In short, more students will be held back, and less flexibility will be granted to educators in determining if a student who misses the proficient level can proceed to the fourth grade.

None of this expansion is funded, so let's take a look at what this policy might additionally cost cash strapped schools.

In October 2011, a total of 126,569 3rd grade Ohio public school students participated in the Reading Achievement Test. Here are the aggregated results, according to ODE statistics.

Level	Number	Percent
Advanced	22,987	18.2%
Accelerated	23,619	18.7%
Proficient	28,038	22.2%
Basic	23,574	18.6%
Limited	28,351	22.4%

In recent years the number of students scoring proficient or higher has varied from a high of 67.5% to a low of 53.3%. Remember, according to the new proposal, any student scoring below proficient is likely to be held back and made to repeat 3rd grade.

Again, according to ODE statistics, the median cost per pupil in Ohio per year in 2011 was $9,567.89, with an average of $9,961.57.

Under the new rules, the 51,925 students who failed to reach the minimum proficiency standard would have been at risk of being held back. At a median cost of $9,961 per student, districts could be on the hook for a total of $517,224,925 to fund that many students repeating 3rd grade.

To put that into some perspective, the crisis in Cleveland public schools is caused by a budget shortfall of $65 million. This unfunded manade could pay for that shortfall 8 times over.

The Evidence on Charter Schools and Test Scores

The Shanker Institute has just released a policy brief titled "The Evidence on Charter Schools and Test Scores", which finds

The available research suggests that charter schools’ effects on test score gains vary by location, school/student characteristics and other factors. When there are differences, they tend to be modest. There is tentative evidence suggesting that high-performing charter schools share certain key features, especially private donations, large expansions of school time, tutoring programs and strong discipline policies. Finally, while there may be a role for state/local policies in ensuring quality as charters proliferate, scaling up proven approaches is constrained by the lack of adequate funding, and the few places where charter sectors as a whole have been shown to get very strong results seem to be those in which their presence is more limited. Overall, after more than 20 years of proliferation, charter schools face the same challenges as regular public schools in boosting student achievement, and future research should continue to focus on identifying the policies, practices and other characteristics that help explain the wide variation in their results.

Charter Review

Next Season on Survivor

This came to us via our mail box and we thought we would share.

Have you heard about the next planned "Survivor” show?

Governor John Kasich (OH), Governor Walker (Wis) and Governor Christie (NJ) will be dropped in an elementary school classroom for 1 school year.

Each of them will be provided with a copy of his/her school district's curriculum, and a class of 20-25 students.

Each class will have a minimum of five learning-disabled children, three with A.D.H.D., one gifted child, and two who speak limited English. Three students will be labeled with severe behavior problems.

Each of them must complete lesson plans at least 3 days in advance, with annotations for curriculum objectives and modify, organize, or create their materials accordingly. They will be required to teach students, handle misconduct, implement technology, document attendance, write referrals, correct homework, make bulletin boards, compute grades, complete report cards, document benchmarks, communicate with parents, and arrange parent conferences. They must also stand in their doorway between class changes to monitor the hallways.

In addition, they will complete fire drills, tornado drills, and [Code Red] drills for shooting attacks each month.

They must attend workshops, faculty meetings, and attend curriculum development meetings. They must also tutor students who are behind and strive to get their 2 non-English speaking children proficient enough to take the SOLS tests. If they are sick or having a bad day they must not let it show.

Each day they must incorporate reading, writing, math, science, and social studies into the program. They must maintain discipline and provide an educationally stimulating environment to motivate students at all times. If all students do not wish to cooperate, work, or learn, the teacher will be held responsible.

These people will only have access to the public golf course on the weekends, but with their new salary, they will not be able to afford it. There will be no access to vendors who want to take them out to lunch, and lunch will be limited to thirty minutes, which is not counted as part of their work day.

They will be permitted to use a student restroom, as long as another survival candidate can supervise their class.

If the copier is operable, they may make copies of necessary materials before, or after, school. However, they cannot surpass their monthly limit of copies. They must also continually advance their education, at their expense, and on their own time.

The winner of this Season of Survivor will be allowed to return to their job.

Pass this to your friends who think teaching is easy, and to the ones that know it is hard.

News for March 21th, 2011

The week starts with a lot of coverage of S.B.5, and the reckless budget that takes a $3.1 billion axe to education in Ohio. Starting with the Dispatch

Dove, the Worthington teacher and Ohio's reigning teacher of the year, opposes Kasich's education agenda. He said the only benefit for teachers and schoolchildren is that the debate itself is making the public realize changes are needed.

"The downside is, when we get these short, quick answers and proposals on a subject that's incredibly complex," Dove said.

A Plain Dealer columnist writes

The manner in which Gov. John Kasich and his political allies are trying to ram this legislation through and their dismissive attitude toward public employees tell me that the voice of working people in this state doesn't matter to the politicians who support Senate Bill 5.

As public employees, we are the nurses, teachers, firefighters, librarians, cops and first responders who provide critical services every day for our fellow Ohioans. We are not to blame for the current financial crisis, but we are willing to do our part to fix this broken economy.

The Dispatch goes on to provide some limited coverage of the radical plan to privatize Ohio Schools

"Resources are extremely limited, and we think we should focus those resources on the schools that 90 percent of our children attend," said Matthew Dotson, of the Ohio Education Association, the state's largest teachers union.

Under Kasich's plan, tuition vouchers - 28,000 this year and 56,000 in 2013 - would be available to students in schools ranked in academic watch or academic emergency for two of the previous three years. That would be about 200 schools this year.

An article discusses where the current attar of play is with S.B.5. It's clear that the Republicans in the House are having some problems crafting a bill that can garner the votes needed

"The last thing we want to have happen to this bill is have it go to a conference committee," Blessing said. "It will really turn into a circus then."

House Speaker William G. Batchelder, R-Medina, said last week that there are a great number of potential amendments to the bill.

The pressure of rallies all across the state, like this one in Newark on Saturday is clearly having an effect.