Choosing blindly

As we continue to explore areas of education reform currently under discussed, we wanted to bring this recently released study from the Brookings Institute's Brown Center on Education Policy, titled "Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core", to your attention.

Students learn principally through interactions with people (teachers and peers) and instructional materials (textbooks, workbooks, instructional software, web-based content, homework, projects, quizzes, and tests). But education policymakers focus primarily on factors removed from those interactions, such as academic standards, teacher evaluation systems, and school accountability policies. It’s as if the medical profession worried about the administration of hospitals and patient insurance but paid no attention to the treatments that doctors give their patients.

There is strong evidence that the choice of instructional materials has large effects on student learning—effects that rival in size those that are associated with differences in teacher effectiveness. But whereas improving teacher quality through changes in the preparation and professional development of teachers and the human resources policies surrounding their employment is challenging, expensive, and time-consuming, making better choices among available instructional materials should be relatively easy, inexpensive, and quick.

Administrators are prevented from making better choices of instructional materials by the lack of evidence on the effectiveness of the materials currently in use. For example, the vast majority of elementary school mathematics curricula examined by the Institute of Education Sciences What Works Clearinghouse either have no studies of their effectiveness or have no studies that meet reasonable standards of evidence.

Not only is little information available on the effectiveness of most instructional materials, there is also very little systematic information on which materials are being used in which schools. In every state except one, it is impossible to find out what materials districts are currently using without contacting the districts one at a time to ask them. And the districts may not even know what materials they use if adoption decisions are made by individual schools. The National Center for Education Statistics (NCES), which has the mission of collecting and disseminating information related to education in the U.S., collects no information on the usage of particular instructional materials.

This scandalous lack of information will only become more troubling as two major policy initiatives—the Common Core standards and efforts to improve teacher effectiveness—are implemented. Publishers of instructional materials are lining up to declare the alignment of their materials with the Common Core standards using the most superficial of definitions. The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials, but there is currently no basis to measure the quality of materials. Efforts to improve teacher effectiveness will also fall short if they focus solely on the selection and retention of teachers and ignore the instructional tools that teachers are given to practice their craft.

The full report can be read here.

Blind eyes and cruel intentions

Research has long shown the achievement gap to be dominated by poverty and the differences in income. It is remarkable then, that the Dispatch would publish an 800 plus word article on the achievement gap, and not once in the entire article mention poverty. This is important, because just like the Dispatch, the state plan to grade schools based upon closing this achievement gap also does not address the issue of poverty either. Somehow, simply wishing the gap to be closed is good enough policy, when coupled with punishment for the schools when the miracles fail to happen.

A recent paper, titled "Education and Poverty: Confronting the Evidence" from the Duke Sanford School of Public Policy had this to say

Current U.S. policy initiatives to improve the U.S. education system, including No Child Left Behind, test-based evaluation of teachers and the promotion of competition, are misguided because they either deny or set to the side a basic body of evidence documenting that students from disadvantaged households on average perform less well in school than those from more advantaged families. Because these policy initiatives do not directly address the educational challenges experienced by disadvantaged students, they have contributed little -- and are not likely to contribute much in the future -- to raising overall student achievement or to reducing achievement and educational attainment gaps between advantaged and disadvantaged students. Moreover, such policies have the potential to do serious harm. Addressing the educational challenges faced by children from disadvantaged families will require a broader and bolder approach to education policy than the recent efforts to reform schools.

These bolder, broader efforts would require sacrifices beyond the school walls, and prove politically difficult for law makers to confront. This in part explains why Ohio is the only state in the country that does not have a school funding formula, let alone a constitutional one.

What it doesn't explain however, is Republican policy to make the poverty situation worse by trying to enact budget measures like slashing food stamps

From food stamps to child tax credits and Social Service block grants, House Republicans began rolling out a new wave of domestic budget cuts Monday but less for debt reduction — and more to sustain future Pentagon spending without relying on new taxes.

A policy that would ensure more students go to school hungry is particularly cruel.

Data released by the Census Department recently showed the percentage of Americans living in poverty is the highest in 15 years, with children feeling the rise most acutely. The news has direct implications for reformers intent on narrowing the academic achievement gap. As the NYT reported recently

Education was historically considered a great equalizer in American society, capable of lifting less advantaged children and improving their chances for success as adults. But a body of recently published scholarship suggests that the achievement gap between rich and poor children is widening, a development that threatens to dilute education’s leveling effects.

It is a well-known fact that children from affluent families tend to do better in school. Yet the income divide has received far less attention from policy makers and government officials than gaps in student accomplishment by race.

Now, in analyses of long-term data published in recent months, researchers are finding that while the achievement gap between white and black students has narrowed significantly over the past few decades, the gap between rich and poor students has grown substantially during the same period.

“We have moved from a society in the 1950s and 1960s, in which race was more consequential than family income, to one today in which family income appears more determinative of educational success than race,” said Sean F. Reardon, a Stanford University sociologist. Professor Reardon is the author of a study that found that the gap in standardized test scores between affluent and low-income students had grown by about 40 percent since the 1960s, and is now double the testing gap between blacks and whites.

It is with some releif then that Ohio Democrats are proposing at least a modest restoration in funding for schools, after the savage cuts made by the previous budget

Ohio - Ohio House Democrats want to funnel tax dollars back to schools and local governments handed a whopping cut in Republican Gov. John Kasich's state budget passed last year.

House Minority Leader Armond Budish, a Beachood Democrat, and five Democratic members of the House Finance Committee appeared at a Statehouse news conference Monday morning calling for $400 million in additional funding for schools and local governments hit by cuts in the state's operating budget.

"The Kids and Communities First Fund will keep teachers in the classroom and police and firefighters on the streets in communities across Ohio," Budish said.

As long as policy makers and newspaper reporting ignore the very real problem of poverty and its connection to student achievement, we are deluding ourselves into thinking all we need to do is come up with a new test, or a new way to grade schools and everything will be A-ok.

Education News for 04-19-2012

Local Issues

• Town hall session on teen drinking sparks interest (Newark Advocate)

The students of Granville High School's CHAMPS group kicked off this past week's town hall meeting on underage alcohol use by shocking the audience. They read anonymous stories from Granville teens who said they used drugs and alcohol on a regular basis. Read more..

• Parents, Teachers Angry Over CMSD Staff Layoffs (WJW – Cleveland)

The decision by the Cleveland Municipal School District to respond to a $66-million budget deficit, by laying off 508 teachers, is being criticized by teachers and parents. Justin Hons, 32, is being laid-off from his job as a social studies teacher at John Hay High School, and it’s his fourth lay-off notice in 11 years. Read more..

• Raising Tusc summit: Give grads skills that employers require (New Philadelphia Times Reporter)

Schools of the 21st century must focus on graduating students who have the skills employers are requiring, a Georgia educator told area business and school leaders Wednesday. Frank Pinson, chief executive officer of the Floyd County College and Career Academy, was keynote speaker at the Raising Tuscarawas summit at the Performing Arts Center at Kent State University Tuscarawas. Read more..

• Bellaire School District asking community for funding ideas (WTOV-Steubenville)

Bellaire School District officials will hold a community meeting on Thursday focusing on the future of the financially strapped district. In 35 years, the district has not had any additional money for operations, and Superintendent Tony Scott said he hopes community members can generate ideas for funding. Read more..

• CPS cuts 10 percent of teaching staff (Cincinnati Enquirer)

Cincinnati Public Schools will have 237 fewer teachers next school year. The seven-member board of education voted unanimously Wednesday to eliminate about 10 percent of its teaching staff to help fill a $43 million budget gap. The cuts, which the Enquirer reported earlier this month, come as the district works to maintain its place as Ohio’s highest-rated urban school district. Read more..

• Reynoldsburg City School Students Will Pay More To Play (10-TV-Columbus)

Reynoldsburg City School District officials said that many were questioning the school booster club following a rise in prices for students wishing to participate in sports. The school board voted on Tuesday night to raise the fee to participate in sports to more than $100 to make up for the money it expected to receive from a booster club, 10TV's Tanisha Mallett reported. Read more..

• Student's mother seeks reprimand against police chief, principal (Springfield News Sun)

A mother is asking for the Enon police chief and principal of Indian Valley School to be reprimanded after she said her son was unjustly questioned about a missing cell phone against school policies. The incident happened March 28 after the mother of an Indian Valley School parent reported her daughter’s cell phone missing. Read more..

• Taking gun to school gets student 3 weekends in jail (Columbus Dispatch)

A Columbus high-school student who took a gun to school last year was placed on probation yesterday and ordered to spend three weekends in jail. Razoar B.D. Harding, 18, of Briar Ridge Road on the East Side, pleaded guilty in March to two counts of carrying a concealed weapon. Read more..

Value-Added Versus Observations

Value-Added Versus Observations, Part One: Reliability

Although most new teacher evaluations are still in various phases of pre-implementation, it’s safe to say that classroom observations and/or value-added (VA) scores will be the most heavily-weighted components toward teachers’ final scores, depending on whether teachers are in tested grades and subjects. One gets the general sense that many – perhaps most – teachers strongly prefer the former (observations, especially peer observations) over the latter (VA).

One of the most common arguments against VA is that the scores are error-prone and unstable over time – i.e., that they are unreliable. And it’s true that the scores fluctuate between years (also see here), with much of this instability due to measurement error, rather than “real” performance changes. On a related note, different model specifications and different tests can yield very different results for the same teacher/class.

These findings are very important, and often too casually dismissed by VA supporters, but the issue of reliability is, to varying degrees, endemic to all performance measurement. Actually, many of the standard reliability-based criticisms of value-added could also be leveled against observations. Since we cannot observe “true” teacher performance, it’s tough to say which is “better” or “worse,” despite the certainty with which both “sides” often present their respective cases. And, the fact that both entail some level of measurement error doesn’t by itself speak to whether they should be part of evaluations.*

Nevertheless, many states and districts have already made the choice to use both measures, and in these places, the existence of imprecision is less important than how to deal with it. Viewed from this perspective, VA and observations are in many respects more alike than different.

[readon2 url="http://shankerblog.org/?p=5621"]Continue reading part I[/readon2]

Value-Added Versus Observations, Part Two: Validity

In a previous post, I compared value-added (VA) and classroom observations in terms of reliability – the degree to which they are free of error and stable over repeated measurements. But even the most reliable measures aren’t useful unless they are valid – that is, unless they’re measuring what we want them to measure.

Arguments over the validity of teacher performance measures, especially value-added, dominate our discourse on evaluations. There are, in my view, three interrelated issues to keep in mind when discussing the validity of VA and observations. The first is definitional – in a research context, validity is less about a measure itself than the inferences one draws from it. The second point might follow from the first: The validity of VA and observations should be assessed in the context of how they’re being used.

Third and finally, given the difficulties in determining whether either measure is valid in and of itself, as well as the fact that so many states and districts are already moving ahead with new systems, the best approach at this point may be to judge validity in terms of whether the evaluations are improving outcomes. And, unfortunately, there is little indication that this is happening in most places.

Let’s start by quickly defining what is usually meant by validity. Put simply, whereas reliability is about the precision of the answers, validity addresses whether we’re using them to answer the correct questions. For example, a person’s weight is a reliable measure, but this doesn’t necessarily mean it’s valid for gauging the risk of heart disease. Similarly, in the context of VA and observations, the question is: Are these indicators, even if they can be precisely estimated (i.e., they are reliable), measuring teacher performance in a manner that is meaningful for student learning?

[readon2 url="http://shankerblog.org/?p=5670"]Continue reading part II[/readon2]

Studies Give Nuanced Look at Teacher Effectiveness

The massive Measures of Effective Teaching Project is finding that teacher effectiveness assessments similar to those used in some district value-added systems aren't good at showing which differences are important between the most and least effective educators, and often totally misunderstand the "messy middle" that most teachers occupy. Yet the project's latest findings suggest more nuanced teacher tests, multiple classroom observations and even student feedback can all create a better picture of what effective teaching looks like.

Researchers dug into the latest wave of findings from the study of more than 3,000 classes for a standing-room-only ballroom at the American Educational Research Association's annual conference here on Saturday.

"The beauty of multiple measures isn't that there are more of them—more can be more confusing—these need to be alligned to the outcomes we care about," said Steve Cantrell, who oversees the MET project for the Bill & Melinda Gates Foundation. Existing teacher evaluation systems often use indicators that are not effective at guaging student achievement, and moreover that lump teachers into too-simplistic categories.

"The middle is a lot messier than a lot of state policies would lead us to believe," Cantrell said. "Teachers don't fall neatly into quartiles. Based on the practice data, if I look at the quartiles, all that separates the 25th and 75th on a class (observation) instrument is .68—less than 10 percent of the scale distribution. In a lot of systems, the 75th percentile teacher is considered a leader and the 25th percentile considered a laggard. ...This would suggest they're a lot closer than being off by two categories."

[readon2 url="http://blogs.edweek.org/edweek/inside-school-research/2012/04/the_most_and_least_effective.html"]Continue reading...[/readon2]

Parents *STILL* choose public schools

HB153, the budget bill, increased the availability of school vouchers for private schools from 14,000 to 30,000 last year and to 60,000 this year. Clearly the legislators was expecting a private school voucher gold rush.

When that didn't happen last year, excuses were made. This year, according to reports in the Dispatch, application levels are almost as miserable.

An additional 1,544 requests for a new special-needs voucher program were made by Sunday’s deadline.

Among the 17,438 applications for an Educational Choice Scholarship were 3,814 new applicants. That deadline was Friday. About 17,000 applications were filed last year to use vouchers this school year.

There are 60,000 vouchers available next school year for students in low-performing schools. They’re worth $4,250 for younger students and $5,000 for high-school students.

The Dispatch headline catagorizes this as "School-voucher programs prove popular". the reality and the truth is obviously quite different. Approximately 3,438 additional students have appliedfor vouchers above the original cap of 14,000. That amounts to just 5.7% of the new 60,000 cap.

The voucher expansion in Ohio is clearly a massive failure. Lawmakers obvously expected demand for these vouchers to be over 30,000, hence the increase to 60,000 this year. For those that continue to believe that parents in Ohio want "choice", for the second year in a row, they have been proven to be wrong. Parents in Ohio continue to overwhelmingly support public schools - it's time politicians listened and began to support it fully too.