Article

Teach the Books, Touch the Heart

FRANZ KAFKA wrote that “a book must be the ax for the frozen sea inside us.” I once shared this quotation with a class of seventh graders, and it didn’t seem to require any explanation. Related in Opinion

We’d just finished John Steinbeck’s “Of Mice and Men.” When we read the end together out loud in class, my toughest boy, a star basketball player, wept a little, and so did I. “Are you crying?” one girl asked, as she crept out of her chair to get a closer look. “I am,” I told her, “and the funny thing is I’ve read it many times.”

But they understood. When George shoots Lennie, the tragedy is that we realize it was always going to happen. In my 14 years of teaching in a New York City public middle school, I’ve taught kids with incarcerated parents, abusive parents, neglectful parents; kids who are parents themselves; kids who are homeless or who live in crowded apartments in violent neighborhoods; kids who grew up in developing countries. They understand, more than I ever will, the novel’s terrible logic — the giving way of dreams to fate.

For the last seven years, I have worked as a reading enrichment teacher, reading classic works of literature with small groups of students from grades six to eight. I originally proposed this idea to my principal after learning that a former stellar student of mine had transferred out of a selective high school — one that often attracts the literary-minded offspring of Manhattan’s elite — into a less competitive setting. The daughter of immigrants, with a father in jail, she perhaps felt uncomfortable with her new classmates. I thought additional “cultural capital” could help students like her fare better in high school, where they would inevitably encounter, perhaps for the first time, peers who came from homes lined with bookshelves, whose parents had earned not G.E.D.’s but Ph.D.’s.

Along with “Of Mice and Men,” my groups read: “Sounder,” “The Red Pony,” “A Raisin in the Sun,” “Lord of the Flies,” “The Catcher in the Rye,” “Romeo and Juliet” and “Macbeth.” The students didn’t always read from the expected perspective. Holden Caulfield was a punk, unfairly dismissive of parents who had given him every advantage. About “The Red Pony,” one student said, “it’s about being a dude, it’s about dudeness.” I had never before seen the parallels between Scarface and Macbeth, nor had I heard Lady Macbeth’s soliloquies read as raps, but both made sense; the interpretations were playful, but serious. Once introduced to Steinbeck’s writing, one boy went on to read “The Grapes of Wrath” and told me repeatedly how amazing it was that “all these people hate each other, and they’re all white.” His historical perspective was broadening, his sense of his own country deepening. Year after year, ex-students visited and told me how prepared they had felt in their freshman year as a result of the classes.

And yet I do not know how to measure those results. As student test scores have become the dominant means of evaluating schools, I have been asked to calculate my reading enrichment program’s impact on those scores. I found that some students made gains of over 100 points on the statewide English Language Arts test, while other students in the same group had flat or negative results. In other words, my students’ test scores did not reliably indicate that reading classic literature added value.

[readon2 url="http://www.nytimes.com/2012/04/22/opinion/sunday/taking-emotions-out-of-our-schools.html?_r=1&pagewanted=all"]Contimue reading...[/readon2]

Experience Counts

Among the more bizarre trends in education reform debate has been the emergence of an argument that experience doesn’t really matter. The problem appears to be that some researchers have not found ways to measure the importance of experience very effectively, and so, cheered on by cost-cutting and union-bashing allies, they tell us that after the first few years, teacher experience doesn’t matter. They have the test scores to prove it, they say.

I’m not here to argue the opposite. I’ve seen new teachers who have a skill set that rivals some of their veteran colleagues. However, I’ve never met a teacher who didn’t believe they could still improve. After all, we’re in the learning business. With experience comes not only time to learn more content and more pedagogy, but also to learn more about children, psychology and brain neurology, about working effectively with peers, administrators, and the community.

Think of other professions, and let me know if you know of any where experience isn’t valued. If education research isn’t showing the value of experience, then I think we should be asking questions like, “What’s wrong with their research methods? What’s wrong with the measures they’ve chosen? What’s wrong with schools and education systems that they can’t put experience to better use?”

This morning, I heard an interesting story about the oil industry, and the experience gap that is emerging among its engineers and other workers. To my untrained eye, this seems like an industry where experience wouldn’t matter. You’re dealing with physics, chemistry, machinery, manual labor – does the oil rig know or care how old or how experienced the workers are? Is there any chance that the properties of oil are unpredictable? If you can build, repair, or operate machinery in another industry, is the oil industry machinery so different?

[readon2 url="http://accomplishedcaliforniateachers.wordpress.com/2012/04/20/experience-counts/"]Continue reading...[/readon2]

Accountability for vouchers

On the idea of vouchers in the new world of accountability

The rise of testing-based accountability measures immeasurably complicates this argument for vouchers. Under No Child Left Behind and Race to the Top, public schools have to demonstrate their performance on certain standardized measures in order to receive funding. Race to the Top further centralizes educational affairs by encouraging states to adopt a nationwide core curriculum and by emphasizing a testing-driven component for teacher evaluations. The argument on behalf of such measures is that public dollars demand proof that they will be spent in a valuable way, and standardized testing is, apparently, the best way to establish this value. (Yes, this argument may be flawed in many, many ways, but let us leave that to the side for the moment as well.) If one wants to establish a centralized, federally-run public school system, No Child Left Behind and Race to the Top provide a sturdy foundation for that enterprise.

If, however, one wants to support a pluralist, voucher-driven kind education reform, this movement toward standards-based accountability could prove much more problematic. If the premise of this accountability is that public dollars require proof of effectiveness, what is the reason for demanding that a school run as a public institution (that is, a public school) should be held to any different standard than a school run as a private institution? Both would receive tax dollars from state and potentially local and federal governments: why should they be held to two different standards?
[...]
Once you've bought into the idea that standardized testing establishes a school's quality, it becomes harder to resist the idea that public money ought to go only to those private entities that have demonstrated their effectiveness in teaching. Moreover, these standards for accountability will, as both Louisiana and federal reforms demonstrate, tend to come from bureaucrats working in central government offices.

Under a universalized voucher program and homogenous standards system, the federal government, which would be the engine that de facto drives education policy under this "reformist" vision, would have increasing control over private schools. Why? Over a period of years, private schools would become increasingly dependent upon government tax dollars, and he who pays the piper picks the tune. Maybe not now, maybe not a few years from now, but eventually legislators and regulators could start placing further demands upon these newly dependent private schools. After all, if education is truly in a state of crisis, shouldn't government be demanding the best from schools in exchange for the taxpayer's hard-earned dollars?

In Ohio, much of this discussion has already begun to happen.

  • A 2009 newspaper report titled "Ohio voucher students must do better on tests "
  • But that doesn't mean these schools [voucher eecipients] — and their students — shouldn't be held accountable.

    That's needed more than ever after the Cincinnati Enquirer reported that voucher students performed poorly on state achievement tests.

    An analysis of about 2,900 students revealed that six in 10 did not pass math, science or social studies, four in 10 failed reading and three in 10 failed writing.

  • A 2010 article titled "Voucher students' test scores lag"
  • Several thousand Ohio students who used Education Choice scholarships to attend private schools are doing no better than students at the public schools they left behind, state test data show.

  • A 2011 report titled "Ohio Vouchers Fail to Raise Student Achievement"
  • The latest data from the Ohio Department of Education showed that students in the state’s voucher programs did not perform better academically than their peers in public schools. In fact, public school students outscored those in voucher schools in many cases.

  • And when voucher students were compared to the much maligned Cleveland Schools, this headline was produced "Cleveland students hold their own with voucher students on state tests"
  • The push is on to expand school voucher programs in Ohio, but new state data suggests that students who attend private schools with the help of taxpayer-funded vouchers don't necessarily fare better academically than the children they leave behind.

    Cleveland public school students often outperformed voucher students on 2009-10 state proficiency tests, according to data from the Ohio Department of Education.

As the article we opened with discusses, tax payers are unlikely to want to see their precious dollars fund private schools, in light of the overwhelming evidence that these private schools are producing results no better, and in many cases, a lot worse than their public school alternatives.

It would be reasonable to suggest that we ought to have accountability for private schools that receive tax dollars. If they cannot produce results as good as, or better, than their public school counterparts, they ought not to be able to continue to receive tax payer assistance.

If we are to continue to pursue a policy of choice, we have a duty to ensure those choices are of the highest possible quality available, and that goes for private schools too.

Choosing blindly

As we continue to explore areas of education reform currently under discussed, we wanted to bring this recently released study from the Brookings Institute's Brown Center on Education Policy, titled "Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core", to your attention.

Students learn principally through interactions with people (teachers and peers) and instructional materials (textbooks, workbooks, instructional software, web-based content, homework, projects, quizzes, and tests). But education policymakers focus primarily on factors removed from those interactions, such as academic standards, teacher evaluation systems, and school accountability policies. It’s as if the medical profession worried about the administration of hospitals and patient insurance but paid no attention to the treatments that doctors give their patients.

There is strong evidence that the choice of instructional materials has large effects on student learning—effects that rival in size those that are associated with differences in teacher effectiveness. But whereas improving teacher quality through changes in the preparation and professional development of teachers and the human resources policies surrounding their employment is challenging, expensive, and time-consuming, making better choices among available instructional materials should be relatively easy, inexpensive, and quick.

Administrators are prevented from making better choices of instructional materials by the lack of evidence on the effectiveness of the materials currently in use. For example, the vast majority of elementary school mathematics curricula examined by the Institute of Education Sciences What Works Clearinghouse either have no studies of their effectiveness or have no studies that meet reasonable standards of evidence.

Not only is little information available on the effectiveness of most instructional materials, there is also very little systematic information on which materials are being used in which schools. In every state except one, it is impossible to find out what materials districts are currently using without contacting the districts one at a time to ask them. And the districts may not even know what materials they use if adoption decisions are made by individual schools. The National Center for Education Statistics (NCES), which has the mission of collecting and disseminating information related to education in the U.S., collects no information on the usage of particular instructional materials.

This scandalous lack of information will only become more troubling as two major policy initiatives—the Common Core standards and efforts to improve teacher effectiveness—are implemented. Publishers of instructional materials are lining up to declare the alignment of their materials with the Common Core standards using the most superficial of definitions. The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials, but there is currently no basis to measure the quality of materials. Efforts to improve teacher effectiveness will also fall short if they focus solely on the selection and retention of teachers and ignore the instructional tools that teachers are given to practice their craft.

The full report can be read here.

Blind eyes and cruel intentions

Research has long shown the achievement gap to be dominated by poverty and the differences in income. It is remarkable then, that the Dispatch would publish an 800 plus word article on the achievement gap, and not once in the entire article mention poverty. This is important, because just like the Dispatch, the state plan to grade schools based upon closing this achievement gap also does not address the issue of poverty either. Somehow, simply wishing the gap to be closed is good enough policy, when coupled with punishment for the schools when the miracles fail to happen.

A recent paper, titled "Education and Poverty: Confronting the Evidence" from the Duke Sanford School of Public Policy had this to say

Current U.S. policy initiatives to improve the U.S. education system, including No Child Left Behind, test-based evaluation of teachers and the promotion of competition, are misguided because they either deny or set to the side a basic body of evidence documenting that students from disadvantaged households on average perform less well in school than those from more advantaged families. Because these policy initiatives do not directly address the educational challenges experienced by disadvantaged students, they have contributed little -- and are not likely to contribute much in the future -- to raising overall student achievement or to reducing achievement and educational attainment gaps between advantaged and disadvantaged students. Moreover, such policies have the potential to do serious harm. Addressing the educational challenges faced by children from disadvantaged families will require a broader and bolder approach to education policy than the recent efforts to reform schools.

These bolder, broader efforts would require sacrifices beyond the school walls, and prove politically difficult for law makers to confront. This in part explains why Ohio is the only state in the country that does not have a school funding formula, let alone a constitutional one.

What it doesn't explain however, is Republican policy to make the poverty situation worse by trying to enact budget measures like slashing food stamps

From food stamps to child tax credits and Social Service block grants, House Republicans began rolling out a new wave of domestic budget cuts Monday but less for debt reduction — and more to sustain future Pentagon spending without relying on new taxes.

A policy that would ensure more students go to school hungry is particularly cruel.

Data released by the Census Department recently showed the percentage of Americans living in poverty is the highest in 15 years, with children feeling the rise most acutely. The news has direct implications for reformers intent on narrowing the academic achievement gap. As the NYT reported recently

Education was historically considered a great equalizer in American society, capable of lifting less advantaged children and improving their chances for success as adults. But a body of recently published scholarship suggests that the achievement gap between rich and poor children is widening, a development that threatens to dilute education’s leveling effects.

It is a well-known fact that children from affluent families tend to do better in school. Yet the income divide has received far less attention from policy makers and government officials than gaps in student accomplishment by race.

Now, in analyses of long-term data published in recent months, researchers are finding that while the achievement gap between white and black students has narrowed significantly over the past few decades, the gap between rich and poor students has grown substantially during the same period.

“We have moved from a society in the 1950s and 1960s, in which race was more consequential than family income, to one today in which family income appears more determinative of educational success than race,” said Sean F. Reardon, a Stanford University sociologist. Professor Reardon is the author of a study that found that the gap in standardized test scores between affluent and low-income students had grown by about 40 percent since the 1960s, and is now double the testing gap between blacks and whites.

It is with some releif then that Ohio Democrats are proposing at least a modest restoration in funding for schools, after the savage cuts made by the previous budget

Ohio - Ohio House Democrats want to funnel tax dollars back to schools and local governments handed a whopping cut in Republican Gov. John Kasich's state budget passed last year.

House Minority Leader Armond Budish, a Beachood Democrat, and five Democratic members of the House Finance Committee appeared at a Statehouse news conference Monday morning calling for $400 million in additional funding for schools and local governments hit by cuts in the state's operating budget.

"The Kids and Communities First Fund will keep teachers in the classroom and police and firefighters on the streets in communities across Ohio," Budish said.

As long as policy makers and newspaper reporting ignore the very real problem of poverty and its connection to student achievement, we are deluding ourselves into thinking all we need to do is come up with a new test, or a new way to grade schools and everything will be A-ok.

Value-Added Versus Observations

Value-Added Versus Observations, Part One: Reliability

Although most new teacher evaluations are still in various phases of pre-implementation, it’s safe to say that classroom observations and/or value-added (VA) scores will be the most heavily-weighted components toward teachers’ final scores, depending on whether teachers are in tested grades and subjects. One gets the general sense that many – perhaps most – teachers strongly prefer the former (observations, especially peer observations) over the latter (VA).

One of the most common arguments against VA is that the scores are error-prone and unstable over time – i.e., that they are unreliable. And it’s true that the scores fluctuate between years (also see here), with much of this instability due to measurement error, rather than “real” performance changes. On a related note, different model specifications and different tests can yield very different results for the same teacher/class.

These findings are very important, and often too casually dismissed by VA supporters, but the issue of reliability is, to varying degrees, endemic to all performance measurement. Actually, many of the standard reliability-based criticisms of value-added could also be leveled against observations. Since we cannot observe “true” teacher performance, it’s tough to say which is “better” or “worse,” despite the certainty with which both “sides” often present their respective cases. And, the fact that both entail some level of measurement error doesn’t by itself speak to whether they should be part of evaluations.*

Nevertheless, many states and districts have already made the choice to use both measures, and in these places, the existence of imprecision is less important than how to deal with it. Viewed from this perspective, VA and observations are in many respects more alike than different.

[readon2 url="http://shankerblog.org/?p=5621"]Continue reading part I[/readon2]

Value-Added Versus Observations, Part Two: Validity

In a previous post, I compared value-added (VA) and classroom observations in terms of reliability – the degree to which they are free of error and stable over repeated measurements. But even the most reliable measures aren’t useful unless they are valid – that is, unless they’re measuring what we want them to measure.

Arguments over the validity of teacher performance measures, especially value-added, dominate our discourse on evaluations. There are, in my view, three interrelated issues to keep in mind when discussing the validity of VA and observations. The first is definitional – in a research context, validity is less about a measure itself than the inferences one draws from it. The second point might follow from the first: The validity of VA and observations should be assessed in the context of how they’re being used.

Third and finally, given the difficulties in determining whether either measure is valid in and of itself, as well as the fact that so many states and districts are already moving ahead with new systems, the best approach at this point may be to judge validity in terms of whether the evaluations are improving outcomes. And, unfortunately, there is little indication that this is happening in most places.

Let’s start by quickly defining what is usually meant by validity. Put simply, whereas reliability is about the precision of the answers, validity addresses whether we’re using them to answer the correct questions. For example, a person’s weight is a reliable measure, but this doesn’t necessarily mean it’s valid for gauging the risk of heart disease. Similarly, in the context of VA and observations, the question is: Are these indicators, even if they can be precisely estimated (i.e., they are reliable), measuring teacher performance in a manner that is meaningful for student learning?

[readon2 url="http://shankerblog.org/?p=5670"]Continue reading part II[/readon2]