MAKING SCHOOLS (SEEM TO) WORK! Hedrick Smith makes “enormous” claims. But do his claims actually work? // link // print // previous // next //
WINERIP DOES IT AGAIN: Michael Winerip does it again in today’s New York Times. He helps us lay out two basic points about state-run testing programs:
State-run tests are sometimes “easy:” Many states run testing programs which report the number of kids who are “proficient” on a given grade level. But “proficiency” is in the eye of the beholder; a state can set the bar for “proficiency” as high or as low as it wants. How “easy” are some state-run programs? Winerip notes the contrast between results on some state-run programs and results on the federally-administered NAEP (the National Assessment of Educational Progress, known as “The Nation’s Report Card”):
WINERIP (11/2/05): Take Florida, where 30 percent of fourth graders were proficient in reading on this federal test in 2005. Yet on the Florida state test, 71 percent of fourth graders were proficient in reading in 2005. It's a big difference: Are nearly three-quarters of your fourth graders proficient? Or less than a third? And it's typical.As Winerip notes, the gap is similar in New York—and even wider in Tennessee. Where do you set the bar for “proficiency?” That is always a matter of judgment—and states can set it as low as they want. Citizens need to keep this in mind when they get pleasing scores from a state-run program. Why, other journalists might even consider this point—if it isn’t too taxing, too boring, too depressing, too awkward or just too much trouble, of course.
Uh-oh! Trends may differ: Uh-oh! Sometimes, kids get smarter on a state test—and dumber on the federal test simultaneously. Winerip offers examples:
WINERIP: [B]asic trends on the two sets of tests are often contradictory. In Florida, the federal fourth-grade reading proficiency scores were down two percentage points between 2003 and 2005 (bad news); on the state test they were up 11 points (good news). In New York, on the federal test, fourth-grade reading proficiency was down one point; on the state test, up six points. In short, it's hard to answer the age-old question: Are fourth graders getting smarter or dumber?Minor differences may be statistical blips. But sometimes, kids “improve” on state tests because the states keep making their state tests easier. This produces a jump in the scores. But the score gains reflect a change in the tests, not in real student achievement.
As we’ve often noted, Winerip does superlative work at the Times. Here at THE HOWLER, both these points will be in play in the endless but seminal series we brazenly kick off today.
Special report: Making schools (seem to) work!
PART 1—AN IMPORTANT TOPIC: Few topics could be more important. When he introduced his recent two-hour special, Hedrick Smith said he’d tell a surprising story—a story of educational success:
SMITH (10/5/05): Good evening, I’m Hedrick Smith.The program was called Making Schools Work with Hedrick Smith. It aired on many major PBS stations in the first week in October.This is an often surprising story of educational success. Because these are public schools—some of them charter schools—serving our most disadvantaged kids, who many had given up on.
Who the heck is Hedrick Smith? As the bio on his program’s web site explains, Smith is a Pulitzer Prize-winning former New York Times reporter and an Emmy Award-winning producer/correspondent. Indeed, Making Schools Work is “the sixth prime time miniseries that [Smith] has created for the PBS national program service,” the site notes. In short, Smith is a man with a major resumé—and PBS is often considered our smartest news service. As he continued, he said a bit more about what his new program would show:
SMITH (continuing directly): Our story begins where reform began in the 1980s, with models designed to make change school by school. Then we move, as reformers moved in the 1990s, to wholesale reform—to re-engineering entire school districts.But has this “wholesale reform” really worked? Did it produce results in those districts? As he continued, Smith clearly seemed to say that it had. We won’t “find one magic formula,” he said. But we would see a “common denominator—results, lifting scores and closing achievement gaps not just for a few hundred kids but for nearly two million children from our inner cities to rural America.” Indeed, on the front page of the program’s web site, the claims for this PBS show are profound:
“MAKING SCHOOLS WORK” WEB SITE: No topic worries American families more than the quality of our schools. Making Schools Work with Hedrick Smith offers a rare and often surprising look at success in unexpected places, with enormous implications for public schools nationwide.How enormous are the implications? “This web site follows our production team into classrooms from coast to coast to see how some American communities are creating a small revolution in our schools,” the text continues. And again: “The common denominator is results—lifting scores and closing achievement gaps, not just for a few hundred children but for nearly two million, from our inner cities to rural America.”
Smith and his web site were making large claims. His program would have “enormous implications for public schools nationwide”—indeed, it would show us “a small revolution in our schools,” a revolution which has already affected nearly two million low-income kids. And make no mistake—claims like this are always pleasing. Over at the Bergen Record, one scribe—Mary Ellen Schoonmaker—swallowed the story line whole:
SCHOONMAKER (10/6/05): Last night on public television, the two-hour documentary "Making Schools Work, with Hedrick Smith" showcased schools in urban and rural settings around the country where low-income students are succeeding. The theme of the program was there are no excuses. Even troubled kids and failing students can turn their lives around. Test scores can go up. And learning can take place.After describing a few of the highlighted schools, Schoonmaker spoon-fed her readers the message. “The common thread in all these approaches is confidence, which is contagious,” she wrote. “Teachers and principals have a plan. They believe in what they are doing because they know it works, and their energy and enthusiasm rub off on the kids.”The strategies for success were all different.
But do the plans at these schools really work? Few questions could be more important. And because the question is so important, we ought to be especially careful in formulating an answer. At the New York Times, Anita Gates had previewed the program—and had offered one cautioning note:
GATES: (10/5/05): ''Making Schools Work,'' narrated by the journalist Hedrick Smith, visits other schools that have had impressive successes, mostly with drastic, all-encompassing programs. The program ends with examples of districtwide reform, including Anthony J. Alvarado's success in District 2 in New York (focusing on Public School 126 in Chinatown) and a disaster of sorts in San Diego.It won’t be easy,” Gates said as she finished, after citing that oddly upbeat remark about poverty—poverty, which has been deeply and tragically linked, for decades, with low achievement in our schools. But Gates herself never seemed to doubt that the schools on this program had actually “had impressive successes” due to their “all-encompassing programs.” Like Schoonmaker, she seemed to believe what she had been told about these featured schools and school districts.One educator sums it up: ''There is no link between poverty and the ability to learn.'' It is all about making quality teaching available. It requires cooperation and high expectations. It won't be easy.
But what if these featured schools and districts haven’t had “impressive successes?” What of their “impressive successes” have been, to some extent, gimmicked up? What if Smith and his team have somehow misjudged their degree of success? What then are we to make of their all-encompassing programs—the programs Smith would extend to the nation? Because Smith’s topic is so important, we ought to make sure that his judgments are real. Before we start to promote the programs which have produced these impressive successes, we really ought to try to make sure that the impressive successes are real.
Smith examines a series of schools and districts—schools and districts which serve large numbers of deeply deserving low-income kids. He says they’ve achieved “a small revolution,” with implications that are “enormous.” But time and again, when we looked at these schools, we seemed to find something different. Un-oh! For those who want to truly believe, there are ways to make schools seem to work. And we often thought we saw wishful thinking—and slippery analyses—driving Smith’s claims of successes.
TOMORROW—PART 2: That grade school in Washington has a great staff. But how sure can we be that it’s “working?”