cheating

The cheating will continue until morale improves

in

Atlanta wasn’t an isolated incident. Neither was El Paso, or Washington, DC, or Columbus. A new General Accounting Office report demonstrates that cheating by school officials on standardized tests has become commonplace despite the use of security measures the report recommends. The only solution is one that Education Secretary Arne Duncan has so far refused—removing the high stakes attached to standardized testing.

The latest embarrassment is in Columbus, where this month Ohio State Auditor Dave Yost seized records at 20 high schools. This is part of a two-year-old investigation into “scrubbing” 2.8 million attendance records of students who failed tests. Yost has recently widened his investigation to look into whether school administrators also changed grades to boost graduation rates.

A GOA reportreleased May 16 recommends adopting “leading practices to prevent test irregularities.” However, the report reveals that while all states and the District of Columbia use at least some of the recommended best practices, 33 states had confirmed instances of test cheating in the last two school years. And states where the worst offenses are occurring already have adopted most of the practices identified in the report, making it unlikely that greater security will improve test integrity.

Ohio employs five of the nine security plans recommended by the GOA report. Atlanta, where the superintendent and 34 other educators were recently indicted for changing test answers, has adopted eight of nine security practices, as has Texas, where the former El Paso superintendant is now in federal prison for a scheme to encourage low-performing students to drop out. And Washington, D.C., where 191 teachers at 70 schools were implicated in a rash of wrong-to-right erasure marks on tests, uses every single security measure.

The Department of Education responded to the GAO’s findings by holding a symposium on test integrity and issuing a follow-up report on best practices and policies. But the federal government convening a meeting and issuing yet another report might be even less effective at stopping cheating than increased security.

The report also noted that linking awards and recognition to improving test scores and threatening the jobs of principals for low test scores “could provide incentives to cheat.” But at a conference of education writers in April, Sec. Arne Duncan denied that linking test scores to career outcomes could drive educators to criminally manipulate the system.

“I reject the idea that the system forces people to cheat,” he said.

Maybe so, but cheating now seems inherent in the system, and our Education Secretary seems incurious as to why. It’s even hard to get him to admit there is an epidemic of test cheating. Asked about the Ohio investigation, Duncan said, “I almost don’t know of another situation like this.”

[readon2 url="http://jasonstanford.org/2013/05/the-cheating-will-continue-until-morale-improves/"]Continue reading...[/readon2]

What of test integrity?

in

The Atalanta Constitution Journal has a detailed report on the integrity of tests now being used to make high stakes decisions. Their findings are torubling.

The stain of cheating spread unchecked across 44 Atlanta schools before the state finally stepped in and cleaned it up. But across the country, oversight remains so haphazard that most states cannot guarantee the integrity of their standardized tests, The Atlanta Journal-Constitution has found.

Poor oversight means that cheating scandals in other states are inevitable. It also undermines a national education policy built on test scores, which the states and local districts use to fire teachers, close schools and direct millions of dollars in funding.

The AJC’s survey of the 50 state education departments found that many states do not use basic test security measures designed to stop cheating on tests. And most states make almost no attempt to screen test results for irregularities.

The whole article is well worth a read. We have long held that the increased stakes tied to test scores can only increase the incidence of cheating - it happens in every corporate system.

you can see the ACJ survey results here, which include Ohio.

Suspicious test scores

in

Here at JTF, we've been very quick to point out instances of cheating, either isolated, or systemic, as a quick search of our archives or twitter feeds will show. As public education is driven ever more into corporate types of management and measurement, coupled with high stakes tied to test scores, it should surprise no one that corporate types of behavior emerge - think Enron, Arthur Anderson, World Com, MF Global Holdings.

It is with that backdrop we turn to an investigative piece by the Dayton Daily News (DDN) in conjunction with the Atlanta Journal Constitution (AJC), titled "Suspect test scores found across Ohio schools".

Steep spikes and drops on standardized test scores, a pattern that has indicated cheating in Atlanta and other cities across the nation, have occurred in hundreds of school districts and charter schools across Ohio in the past seven years, a Dayton Daily News analysis found.

The analysis does not prove cheating has occurred in Ohio. But interviews and documents show that state officials do not employ vigorous statistical analyses to catch possible cheating, discipline only about a dozen teachers a year and direct Ohio’s test vendor to spend just $17,540 on analyzing suspicious scores out of its $39 million annual testing contract.

It's a weak piece that could be used and sensationalized by many, and the paper has come under almost instant withering criticism for it's approach.

One of the researchers involved in analyzing data for USA today's ground breaking cheating series took a look at the DDN analysis

Given my past role in reviewing data and methods used for detecting systematic cheating, I was delighted to have the opportunity a week ago to review Ohio assessment data that was being used as part of a national study released today by The Atlanta Constitution-Journal and affiliated Cox newspapers. My review, however, yielded serious concerns about the data used, the methods of analysis employed, and the conclusions drawn.
[...]
In short, here are some of my concerns about the methods:
  • As noted, the analysis is based on school-level data and not individual student-level data. Accordingly, it was not possible to ensure that the same students were in the group in both years.
  • The analysis of irregular jumps in test scores should have been coupled with irregularities in erasure data where this data was available.
  • The analysis by Cox generates predicted values for schools, but this does not incorporate demographic characteristics of the student population.
  • The limited details available on the study methods made it impossible to replicate and verify what the journalists were doing. Further, the rationale was unclear for some of the steps they took.

He wasn't the only expert to consider the DDN findings. Stephen Dyer, former newspaper reporter, architect of Ohio's prematurely abandoned evidence based model, and think tank fellow had this to say, after discussing similar analytical shortcomings as pointed out above

If you're going to write a story that suggests massive, statewide (and in AJC's case, national) cheating on standardized tests, you'd better be prepared to name the offenders and feel solid enough in your methodology to refute the state's education agency and largest teachers union, both of whom knocked the papers' methods. If you have to spend a large chunk of your story having competing experts defend and knock your statistical analysis, you need to re-do the analysis. Though it showed integrity for the paper to allow those critical comments in the story.

As a former reporter, I can say these issues would invariably pop up before big stories ran. Sometimes, it means delaying your story for a day or two, or in a few cases, never run them at all. As a journalist, you, as a general rule, cannot spend any time in your story defending your story. If you have to, it means you don't have it nailed down yet; it needs more time in the oven.

The DDN spend almost the entirety of their story defending their story.

Greg Mild, over at Plunderbund has an even harsher response, and points out some great absurdities of the DDN analysis

Furthermore, note that the “2,600 improbable changes” include spikes and drops in test results. These journalists are putting out this theory of irregularities and cheating by schools based on numbers that include falling scores! Right, because so many educators are interested in risking their careers by encouraging children to change their scores to incorrect answers to suffer a significant DROP in their test scores. Yet those numbers are touted by these “journalists” in their sweeping accusations of improbable scores and cheating.

We continue to believe that cheating is totally unacceptable and ought to be exposed when and where found, but the Dayton Daily News story, as they point out themselves, does not come close to demonstrating what they seem to want to sensationalize - widespread cheating, Atlanta style.

As we begin to rely more and more upon student test scores to measure schools and teachers, suspicions are going to grow, a few might be borne out, but many will be baseless - but each accusation serves to undermine public education and people's trust in it. It's another unintended failing of the corporate education reform schemes we're currently pursuing.

More cheating exposed

in

No sooner had we pegged cheating scandals as our number 4 story of the year, than a new scandal emerges, this time in Georgia.

A new investigative report details a second major standardized test cheating scandal in a Georgia school system, implicating 49 educators, including 11 principals. A key reason for the “disgraceful” cheating, investigators said, was pressure to meet No Child Left Behind requirements.

The probe (see here and here) by the Georgia governor’s Special Investigators team into cheating in the Dougherty County School System concluded that “hundreds of school children were harmed by extensive cheating.”

“While we did not find that Superintendent Sally Whatley or her senior staff knew that crimes or other misconduct were occurring, they should have known and were ultimately responsible for accurately testing and assessing students in this system. In that duty, they failed,” the 293-page report says.

Wherever we have corporate education reforms we find unsavory corporate behaviors.

Top 5 Ed stories of 2011

in

2011 has been a tumultuous year for education policy in Ohio. With a new administration and single party control of all the legislative levers, we have witnessed a lot of corporate education reform ideas rushed, with little discussion, into reality. We thought we would reflect on what has happened, and bring you our 5 top education stories of 2011.

5. Two Heads Are Worse Than One

The year started with Deborah Delisle as the State Superintendent, but pressure from the Governor and a board of education stacked with tea party activists, saw her quickly ousted.

"Last Friday, it was made known to me by two members of the governor's staff that my tenure was limited," Delisle said during the board's monthly meeting in Columbus. "They said they have the votes to replace me."

It was supposed to be a quick one-two step. Oust Delisle, install the Education Czar Bob Sommers. Somewhere along the line, for reasons still not wholly clear, there was a misstep and suddenly the administration was left scrambling to fill this critical roll. Candidates dropped out quickly and no new candidates from either far nor wide stepped forward. Almost by default, with one foot out the door, interim Superintendent Stan Heffner was appointed.

Heffner's first job was to implement the newly passed budget an axe staff to make up for a $6.3 million shortfall

“If we’re going to sponsor up to 20 schools and if we’re going to engage in the additional activities that House Bill 153 has charged us with, then there I already have an under-staffed office.”

He also still has Education Czar Sommers looking over his shoulder. It can't be easy working at ODE these days, not with greatly increased mandates, reduced budgets and two bosses.

4. Who me? Cheat?

With the rapid proliferation and implementation of corporate education policies came news of other corporate behaviors. Cheating.

The year started with serious questions being raised of the darling of corporate education reform, Michele Rhee, as evidence came to light that much of her success may have been a consequence of cheating. This was quickly surpassed by a massive cheating scandal unfolding in Atlanta

State investigators have uncovered a decade of systemic cheating in the Atlanta Public Schools and conclude that Superintendent Beverly Hall knew or should have known about it, The Atlanta Journal-Constitution has learned.

In a report that Gov. Nathan Deal planned to release today, the investigators name nearly 180 educators, including more than three dozen principals, as participants in cheating on state curriculum tests, officials said over the weekend. The investigators obtained scores of confessions.

New Jersey also fell under the shadow of suspicion

The Department of Education has ordered an investigation of 34 schools for possible cheating after an analysis of standardized test scores revealed irregularities.

It seems wherever one finds high stakes corporate education policies in effect, we find corporate types of behavior to bolster performance. With similar polices going into effect in Ohio, how long before these headlines hit home?

3. Not So Fast, Huffman

In one of the most audacious moves of the year, State Rep Matt Huffman threw up a legislative Hail Mary, in the hopes that the 1% could make a spectacular catch in the end zone. His bill, HB136 sought to privatize public education in Ohio, transferring hundreds of millions of dollars intended for public education to private schools. After blowing through committee on a party line vote, the radical nature of the bill caused the impossible to happen. Everyone in the education community in Ohio suddenly started to publicly oppose the effort. For a state where people can't agree on lunch, let alone education policy, this was unprecedented and caused Huffman to backtrack. HB136 looks dead for now, with the Hail Mary pass batted down, Huffman may still try for a field goal in the new year.

It's at this point we had to pause and consider. In which order should we place our top two stories? It was a very difficuly choice.

2. Senate Bill 5

If SB5 would have passed, it would have been the number one story. But having been resoundingly defeated it should put to bed the notion of dismantling collective bargaining rights in Ohio for at least another generation. The passage and subsequent repeal of SB5 was the most hotly contested political issue of 2011. In a campaign that went from protests and lock-outs at the Statehouse to signature collections in every neighborhood, to a $50 million campaign, each and every step of the way citizen efforts ate away the small portion of political capital governor Kasich had. The repercussions of SB5 will ripple through 2012, with the fight sure to continue for control of the legislature, but its defeat means it will not have lasting direct policy implications.

In any ordinary year, each of those stories would be huge news and carry great consequence for public education in Ohio, but there is one other story that will have a severe lasting impact on the state's education system.

1. To Be Continued...