Article

Testing Profits

Now that states and the federal government are attaching high stakes to standardized tests, these tests are coming under increasing scrutiny. They don't appear to be holding up well to this additional scrutiny

A top New York state education official acknowledged Wednesday that the mounting number of errors found on this year's math and English tests has eroded public trust in the statewide exams.

"The mistakes that have been revealed are really disturbing," New York State Board of Regents Chancellor Merryl Tisch said at a Midtown breakfast sponsored by Crain's New York Business.

"What happens here as a result of these mistakes is that it makes the public at large question the efficacy of the state testing system," said Ms. Tisch, whose board sets education policy for the state.

Still, Ms. Tisch said testing experts have told state officials that the exams are valid and can be used to evaluate students and, in some cases, teachers.

Over the past several weeks, a series of errors by test-maker Pearson PLC have come to light, ranging from typographical mistakes to a now-infamous nonsensical reading passage about a pineapple. This is the first year of a five-year, $32 million contract the state awarded to Pearson, which also publishes textbooks.

To date, 29 questions have been invalidated on various third- through eighth-grade math and English tests, which are used in New York City to determine whether students are promoted to the next grade.

Pearson didn't return a request for comment.

Mistake riddled tests are not the only problem being highlighted

Is it okay to ask a child to reveal a secret? Richard Goldberg doesn’t think so. Goldberg, the father of 8-year old twin boys, was dismayed to learn his third-grade sons were asked to write an essay about a secret they had and why it was hard to keep. The unusual question, which Goldberg called "entirely inappropriate" was on the standardized tests given to public school students in the third through eighth grade every spring.
[...]
The question will not, however, appear on any future versions of the test, Barra said. "We’ve looked at this question in light of concerns raised by parents, and it is clear that this is not an appropriate question for a state test," Barra said.

Increasingly, calls are being made to make these tests public, so they can be fully vetted.

I learned that the tests themselves are being kept secret because the state Department of Education and Pearson, their test development contractor, wrote strong confidentiality provisions into the contract. My understanding is that this was so that they both could reuse test questions in the future. In order for the questions to be reusable, they have to be kept secret, otherwise students could prep too easily for the tests, and Pearson’s other customers would be able to get the tests from the public domain.

We only know about the gaffes because students exposed them. Educators have been sworn to secrecy. The Education Department has emphasized their concerns about test prep, but to me the secrecy seems rooted in economics: Secrecy saves New York on future test development costs and makes it easier for Pearson to re-sell the questions it created for New York (at New York taxpayers’ expense) in other states.

Two things strike me as odd about this. First, it’s uncommon to keep tests completely secret after the fact of their administration. Letting people see the test is a basic part of education.

The purpose of testing is to measure how well a student knows subject matter and to identify what areas need work. If the only thing one knows about a child’s performance on a test is his grade, and one can’t review the actual test, the test is pedagogically useless and can only serve a punitive purpose.

If the broader community of parents, educators and researchers can’t see tests, then we have no way of judging the connection between them and curricula or how to help our children.

A paper by the National Board on Educational Testing and Public Policy titled "Errors in Standardized Tests: A Systemic Problem" found

This paper contains a sizable collection of testing errors made in the last twenty-five years. It thus offers testimony to counter the implausible demands of educational policy makers for a single, error-free, accurate, and valid test used with large groups of children for purposes of sorting, selection, and trend-tracking.

No company can offer flawless products. Even highly reputable testing contractors that offer customers high-quality products and services produce tests that are susceptible to error. But while a patient dissatisfied with a diagnosis or treatment may seek a second or third opinion, for a child in a New York City school (and in dozens of other states and hundreds of other cities and towns), there is only one opinion that counts – a single test score. If that is in error, a long time may elapse before the mistake is brought to light – if it ever is.

This paper has shown that human error can be, and often is, present in all phases of the testing process. Error can creep into the development of items. It can be made in the setting of a passing score. It can occur in the establishment of norming groups, and it is sometimes found in the scoring of questions.
[…]
Measuring trends in achievement is an area of assessment that is laden with complications. The documented struggles experienced by the National Center for Education Statistics (NCES) and Harcourt Educational Measurement testify to the complexity inherent in measuring changes in achievement. Perhaps such measurement requires an assessment program that does only that. The National Center of Educational Statistics carefully tries to avoid even small changes in the NAEP tests, and examines the impact of each change on the test’s accuracy. Many state DOEs, however, unlike NCES, are measuring both individual student achievement and aggregate changes in achievement scores with the same test – a test that oftentimes contains very different questions from administration to administration. This practice counters the hard-learned lesson offered by Beaton,“If you want to measure change, do not change the measure”(Beaton et al., 1990, p. 165).

Furthermore, while it is a generally held opinion that consumers should adhere to the advice of the product developers (as is done when installing an infant car seat or when taking medication), the advice of test developers and contractors often goes unheeded in the realm of high-stakes decision-making. The presidents of two major test developers – Harcourt Brace and CTB McGraw Hill – were on record that their tests should not be used as the sole criterion for making high-stakes educational decisions (Myers, 2001; Mathews, 2000a). Yet more than half of the state DOEs are using test results as the basis for important decisions that, perhaps, these tests were not designed to support.

Finally, all of these concerns should be viewed in the context of the testing industry today. Lines (2000) observed that errors are more likely in testing programs with greater degrees of centralization and commercialization, where increased profits can only be realized by increasing market share,“The few producers cannot compete on price, because any price fall will be instantly matched by others .... What competition there is comes through marketing”(p. 1). In Minnesota, Judge Oleisky (Kurvers et al. v. NCS, Inc., 2002) observed that Basic Skills Test errors were caused by NCS’ drive to cut costs and raise profits by delivering substandard service – demonstrating that profits may be increased through methods other than marketing.

It clearly appears that profit is winning the day over quality, when it comes to standardized tests.

Here's the full paper.

Errors in Standardized Tests: A Systemic Problem

Tea partiers threaten public education

Not content with the Governor's $3 billion dollar state budget assault on public education, tea partiers, supported by the far right "1851 Center for Constitutional Law" - an offshoot of the right wing Buckeye Institute, are seeking to assault public education funding at the local level too.

Taxpayers for Westerville Schools, a group that opposed a 6.9-mill levy that voters approved in March, has begun collecting signatures to repeal an equal portion of an 11.4-mill levy approved in 2009.

The group is reaching back to that levy because state law bars the repeal of temporary tax issues, such as the five-year levy passed this year. The 2009 tax issue is permanent.

This is a move so radical and extreme that it has only ever been proposed once in the history of the state. If the "Center for Constitutional law" really cared about the Ohio constitution and public education it would be lobbying for a constitutional funding formula for our schools instead of trying to defund them. But rather than do that, they have published a document that contains the broad tactics groups can use to defund public education, a document that contains such information as

Warning: if you follow the advice in this guide, proponents of higher spending and taxation will assert, as always, that children will suffer unless new levies are enacted, while current revenue sources are maintained. However, if you’ve read this far, you and your neighbors (1) have likely already heard and dispelled this argument; (2) are aware that your local school district has a spending problem, not a revenue problem; and (3) simply want to keep more of what you have rightfully earned, and want to this seemingly endless cycle of tax hikes to stop.

Clearly they think every district has a spending problem, and every citizen is over taxed - regardless of whether voters in places like Westerville disagreed by passing a levy just months ago. Their roadmap even includes this nugget:

(6) Keep a low profile. Remember, only once every five years can an attempt be made to reduce any given levy. If your school district’s teachers union gets wind of your plans too early in the process, they may quickly gather signatures and place a .000001 mil reduction of the levy tax on the ballot before you are able to gather and submit signatures for your more significant reduction.

Wanting to operate in the shadows was evident yesterday when confronted over twitter

@jointhefutureOH @DispatchEteam @dougcaruso @cbinkley We are all WCSD residents concerned about our schools' future-NOT a tea party group.

We responded

@TFWS1 Really? All just a coincidence you're involved with the 1851 center? Same agenda as the tea party, same support. Same, same.

As did others

@TFWS1 @jointhefutureOH Sounds like the tenets of the Tea Party? Why fight the association to Tea Party? What's the difference?

At this point, this tea party group tried to make ridiculous claims about the 1851 Constitutional Law Center

@ascheurer @jointhefutureOH They're a non-proft, non-partisan legal ctr dedicatd to protctng the constitut rights of Ohioans from govt abuse

A quick survey of their agenda and their board of directors quickly dispels any notion this is a non-partisan group.

What is striking about this recent move by the tea party to attack public education is their unwillingness to embrace their agenda. Instead, as the 1851 center urges, they want to "keep a low profile". We're going to see to it that that doesn't happen.

Voucher expansion pressure

There's some good news being reported today. It appears Columbus lawmakers have listened to the out-pouring of dissent at a number of the Governor's education policy proposals, and are considering changes and delays

Republican leaders in the Senate plan to slow down Gov. John Kasich’s initiatives for holding back third-graders who aren’t proficient in reading and for a tougher report-card rating system for schools and districts.

Under the Senate plan, new report cards would be issued by Sept. 1, 2013, for the 2012-13 school year, not this summer for the current school year. And the so-called reading guarantee would start in the 2013-14 school year, instead of this fall.
[...]
School district officials, teachers unions and state education groups have urged lawmakers to hold off on the plan so they can better inform parents and teachers of the coming changes.

Under the amendment, a new report-card rating system planned for this school year would be put off, and a task force would be established to provide recommendations to lawmakers by Oct. 1 about the new letter-grade rating system.

The new school rating proposal had come under specific attack, from many diverse groups, as it would have lowered the ratings on the majority of Ohio's schools. One of the unintended consequences of this would have been to expand the geographic eligibility of the state's private school voucher program

The EdChoice program could also see a significant change not only in the number of schools and students eligible for a voucher, but also where these schools are located under the newly proposed A-F system. Under the proposed A-F system more schools would be rated D and F, resulting in an increase in the number of eligible schools. Using performance data from 2010-11 the Ohio Department of Education ran a simulation to demonstrate how schools might fare under the new system (you can read more about the proposed A-F system here). Using that data 273 schools and approximately 105,000 students would now be eligible for the EdChoice program. A majority of eligible schools still remain in Big 8 districts but a couple of new districts such as Hamilton City and South Western City would now have eligible schools on the list under the new A-F system.

With the state's voucher program massively undersubscribed, expanding the geographic availability would be a boon to the profiteers and their advocates. Not something to be considered while there is a push for greater accountability for private schools that take tax payer funded vouchers.

Given the recent news of the Dragonfly Autism school suddenly shuttering its doors, there's never been a more urgent need for oversight and accountability of these types of schools

Dragonfly Academy, a local private school for autistic children, unexpectedly closed its doors Thursday morning amid allegations from parents that promised services were not being provided.

Parents were notified via text message from the school’s executive director, Brianne Bixby-Nightingale, at 10:30 p.m. Wednesday that the school would be closed Thursday and today for “restructuring,” several parents confirmed.

Dragonfly’s six-member board of directors apparently resigned last month.
[...]
Among the parents’ claims are that the school failed to provide required therapies and that it did not have enough qualified staff.

Gallaway confirmed Thursday that both of the school’s intervention specialists had quit.

Former Governor Ted Strickland recently blasted the expansion of Ohio's voucher program Strickland said Ohio's voucher program, which allows students in struggling districts to receive funding to attend private schools, is damaging the quality of Ohio public education.

"Vouchers simply is a way to enter into a private situation where the majority of our students are left behind and a few students may be able to benefit using public tax dollars and I think that's wrong and it's harmful to society," Strickland said.

It's good that lawmakers are now slowing down these corporate reforms and listening to stakeholders. We can only hope this proposed task force takes a long hard look at some of the unintended consequences of the Governor's ideas that might harm public education by increasing further the amount of unaccountable privatization.

The Best Laid Plans of Mice and Men

“As a policy wonk, I push for high academic expectations for all students,” writes Scott Joftus in Education Next. “As a father, however, I find that what matters most to me is that my daughters are happy in school.”

“Over more than 20 years in the field of education—including two with Teach For America—I have helped promote state standards, the Common Core, the hiring of teachers with strong content knowledge, longer class periods for math and reading, and extra support for struggling students, to name a few. I have recently discovered, however, that what I believe as an education policy wonk is not always what I believe as a father.”

Joftus’s wonk side believes “student learning flourishes in classrooms that include students with a wide range of abilities and backgrounds.” However, as a Dad, he admits to getting angry when a troubled kindergartener disrupts his daughter’s class and forces the “talented, but inexperienced” teacher to spend more than half of her time trying to keep this boy on task.

“I feel for children like him; my company works with schools and districts to improve outcomes for these kids. But I was angry. The other children were clearly uncomfortable. His disruptions reduced learning time for my daughter, and seemed to steal some of her innocence and excitement about school.”

Commenters on the Ed Next blog offer both praise and criticism for Joftus. “Teachers have been fighting policy wonks who have been destroying the happy learning environment for decades,” writes one. “But you don’t listen, it is only when it becomes personal that you reconsider your opinions and admit the possibility that teachers have been right all along.” “Had you guys listened twenty years ago, and respected our wisdom on safe and orderly schools, this educational civil war would not have had to happen,” observes veteran teacher and ed blogger John Thompson.

Rocketship schools CEO John Danner admits to similar cognitive dissonance when sending his kids to school. “However, I would challenge you as your kids grow to think more about how those skills jibe with rigor,” he writes. “Rigor is actually a form of compassion. A teacher who expects a lot of their students prevents them from feeling the frustration your children feel now, but much later in their school career. The real problem you are seeing is that your child’s teacher has high expectations but doesn’t understand how to differentiate.

[readon2 url="http://blog.coreknowledge.org/2012/05/07/the-best-laid-plans-of-mice-and-men/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+TheCoreKnowledgeBlog+%28The+Core+Knowledge+Blog%29"]Continue reading...[/readon2]

Opportunity Knocks

Here's a new one for the ol' Reformy Thesaurus: the "Opportunity Culture" in education.

Sure sounds good, doesn't it? Who doesn't want our American kids to have more opportunities in life? Except--oops--this campaign, rolled out by Public Impact, is actually about opportunities for "teacherpreneurs" to make more money by teaching oversized classes--and of course, for school districts to seize that same opportunity to save money through "innovative" staffing models.

How did this exciting window of opportunity emerge? Public Impact explains:

Only 25 percent of classes are taught by excellent teachers. With an excellent teacher versus an average teacher, students make about an extra half-year of progress every year--closing achievement gaps fast, leaping ahead to become honors students, and surging forward like top international peers.

That's a whole lot of leaping and surging. Unfortunately, it's based on a faux statistic, sitting triumphantly on a pyramid of dubious research, prettied up with some post-modern infographics. Like other overhyped blah-blah of "reform"--the "three great teachers in a row" myth, for example, or nearly every "fact" in Waiting for Superman--it's a triumph of slick media slogans over substance. A quick look at the Opportunity Culture Advisory Team tells you what the real purpose of the OC is: cutting teachers, privatizing services, plugging charters and cultivating a little astroturf to cover the scars.

The Opportunity Culture's bold plan begins with a policy recommendation: Schools should be required by law to identify the top 25% of their teachers. Then, once that simple task is completed, OC suggests ten exciting new models for staffing schools, beginning with giving these excellent teachers a lot more students (plus a merit pay carrot) and ending with enlisting "accountable remote teachers down the street or across the nation" who would "provide live, but not in-person instruction while on-site teammates manage administrative duties and develop the whole child."

[readon2 url="http://blogs.edweek.org/teachers/teacher_in_a_strange_land/2012/05/heres_a_new_one_for.html"]Continue Reading...[/readon2]

Radical legislation being planned

We have wrote briefly about ALEC, a far right organization dedicated to pushing radical legislation (such as SB5 and HB194) through state legislatures. Their next salvo is being prepped for education, according to leaked documents.

Mark your calendars, people, because ALEC, the American Legislative Exchange Council is having their spring pow-wow in Charlotte, NC on May 11th, and their agenda is simply brimming with ideas meant to undermine American democracy. The agenda was revealed by Common Cause, who obtained packets ahead of the conference taking place at the Westin Charlotte (in case any of you in Charlotte would like to show up to voice your opposition), and proving that ALEC is a squid-like demon with their tentacles in every aspect of American life, they have covered all their bases.

That does sound bad! But what exactly are they planning? (Here's their leaked agenda (pdf)), and a sample:

Online Course Choice for Students
Summary
This bill opens up the world of high-quality online course instruction to students. Each year, students in public school grades 7-12 would have the option to enroll in up to two online courses that award college credit or meet standards for core academic courses. The state would create standards and accountability measures to ensure that they are providing students with a course catalog containing only high-quality online course offerings. Funding for each online course is driven by the free-market in an open and competitive process, rather than simply allocating a portion of student funding unrelated to the actual cost to deliver the course. Finally, after completion of each online course, parents and students provide feedback via the web in an open forum to rate the effectiveness of the course. This feedback, combined with test scores, provides a quality indicator ranking that is available to all.

More privatization of instruction, market driven of course. Worse proposals are still to come, including

District and School Freedom Act
Summary
This legislation creates a mechanism for public school districts and schools to request exemption from state education standards and regulations. Under this act, any district or school can create a list of state regulations or standards that, if exempted from, the district or school could operate more efficiently and better serve students.
Model Legislation
Be it enacted by the Legislature of the State of XXXX:
A. Notwithstanding any other law, a school district, tradition, or charter school may receive exemptions from statutes and rules as prescribed in this section.
B. The school district, traditional, or charter school may identify and submit exemptions to statutes and rules relating to schools, governing boards and school districts to the state board of education for approval. The state board of education shall review and may approve the exemptions submitted, except for those statutes and rules that directly apply to the following:
1. Health and safety.
2. Requirements for the graduation of pupils from high school.
3. Special education.
4. Financial compliance and procurement requirements.
C. The state board of education may adjust the list of exemptions to comply with federal and state law.

A free for all! It's a mighty strange world we live in where groups can opt to of laws they don't like, but that is what ALEC is proposing here.

So if you see or hear of bills being introduced in Ohio similar to these, you know where the idea came from. We'll be keeping an eye out.