A fascinating, maddening report: The good news: The New York Times is doing a lot of work these days about public school issues.
The bad news: It can be maddening to read their reports, given the frequent technical blundering. A few weeks ago, Sam Dillon made a cosmic blunder in a lengthy front-page report (see THE DAILY HOWLER, 10/8/10). Today, we occasionally gnashed our teeth as we read this report by Sharon Otterman.
Otterman reports on Geoffrey Canadas ballyhooed Promise Academy schools, part of his Harlem Childrens Zone. Canada is one of the stars of Davis Guggenheims Waiting for Superman; in NBCs recent teacher-bashing special broadcasts, Canada bashed teachers with the best of them. But how well have his schools been doing? Otterman attempted to say, but in various ways, she failed.
In some ways, Otterman seems to suggest that Canadas schools havent done all that well. Here you see her first attempt to quantify performance. She is discussing Promise Academy I, one of Canadas two Harlem schools:
OTTERMAN (10/13/10): On a recent Thursday, the current high school students, neatly attired in blue and white uniforms, got special help in college note-taking skills, and chatted animatedly about velocity in an advanced physics class .
You really have to put money into personnel, said Marquitta Speller, who has been the high school principal since January. I dont think you can experience the same level of success without the same level of resources.
But most of the seventh graders, now starting their third year in the school, are still struggling. Just 15 percent passed the 2010 state English test, a number that Mr. Canada said was unacceptably low but not out of line with the schools experience in lifting student performance over time. Several teachers have been fired as a result of the low scores, and others were reassigned, he said.
Fifteen percent passed the state reading test? That sounds very low. (Presumably, Otterman is discussing the performance by those seventh graders on their sixth grade tests, back in April.) But 2010 was the year when New York State suddenly made its statewide tests harder; Otterman never says what the passing rate was in New York City or New York State as a whole. How did that group of students do as compared to other sixth graders? Theres no way to know from this report. Absent that info, the passing rate doesnt really tell you much about the relative success of this school.
As she continues, Otterman does somewhat better. In this passage, she provides the basis for making some comparisons, withholds the basis for others:
OTTERMAN (continuing directly): Giving administrators the ability to fire teachers for poor performance is one of the central suggestions of Waiting for Superman. Overall, 38 percent of Promise Academy Is students in third through sixth grade passed the 2010 English test under the states new guidelines, placing it in the lower half of charter schools citywide, and below the citys overall passing rate of 42 percent. In Harlem as a whole, just 29 percent of children passed.
Promise Academy II, an elementary school that occupies part of a public school building, did better, with 62 percent passing in English, among the top 10 percent of charters Both schools continued to outperform the city in math, with 60 percent passing in one school and 81 percent in the other.
In the highlighted passage, Otterman provides the types of data which let readers make relevant comparisons. In grades 3-6, on the statewide reading tests, Promise Academy I did somewhat worse than New York City as a whole, somewhat better than the rest of Harlem. (She didnt provide the statewide passing rate.) Promise Academy II outperformed the city, by a substantial margin. But by the second paragraph, we are once again deprived of a basis for making real comparisons. Both schools outperformed the city in math, were told. But by how much? By two points? By a lot? We arent allowed to know.
Otterman covers important ground, something the Times has been doing lately. But the technical bungling is never far off in this papers educational reporting. By the way, how does Otterman know that the highlighted claim is accurate?
OTTERMAN: [Promise Academy I], which opened in 2004 in a gleaming new building on 125th Street, should have had a senior class by now, but the batch of students that started then, as sixth graders, was dismissed by the board en masse before reaching the ninth grade after it judged the students performance too weak to found a high school on. Mr. Canada called the dismissal a tragedy.
On a recent Thursday, the current high school students, neatly attired in blue and white uniforms, got special help in college note-taking skills, and chatted animatedly about velocity in an advanced physics class. Most were well below grade level when they first got to the school and took three or four years to catch up; many are now ahead.
Did this school really dump a whole years worth of eighth graders because they werent doing well enough? If true, thats a remarkable story. But how does Otterman know that the highlighted claim in that passage is accurate? Thats the kind of claim schools like this love to make. But how do we know that its accurate?
Did Otterman trust but verifycheck this claim against actual data? Or is this just something she was told? Even at the top of the heap, education reporters still dont feel the need to let readers know.
Special report: Matt Millers strange tale!
PART 2REFORM FIRST, EVIDENCE LATER (permalink): Matt Miller probably isnt from Mars, though he did a fairly good impression in Sundays Washington Post. His piece topped the papers Outlook section, positioned next to a manifesto about public schools from a familiar, shopworn gang of educational reformers.
Those fiery authors included Joel Klein, chancellor of New York Citys schools. In a rational world, Klein would be announcing his resignation in the wake of Jennifer Medinas long, embarrassing news report in Mondays New York Times (see THE DAILY HOWLER, 10/12/10).
Compared to Klein and his sovereign (Mayor Bloomberg), Miller comes off as almost human, though we did have reservations as we read his piece. Our reservations leaped to life in his closing paragraph.
In his Outlook piece, Miller pimped an upscale version of the latest hot educational fad. He very much thinks the public schools would benefit from better teachers. Wellnot from better teachers, exactly. Miller thinks the public schools would benefit from smarter teachersteachers drawn from the upper third of their college class.
On the surface, this seems to make perfect sense. Why wouldnt it be a good idea to attract smarter teachers?
On Sunday, Miller and his co-author, Paul Kihn, oohed and aahed about this proposal, driving the latest very-hot fad as our intellectual leaders pretend to address the challenges facing our schools.
Miller and Kihn went on and on, discussing how great it would be if we could recruit smarter teachers. They dreamed of various strategies which might lure top students into the classroom; their piece sat next to Kleins manifesto, which trashed current teachers well. (And their infernal unions. More on Kleins piece tomorrow.) But because weve actually worked in low-income public schools, a question kept popping into our heads as we read Millers piece.
Yes, it surely seems to make sense; it seems to make sense to lure smarter people into the teaching profession. But what makes Miller so darn sure that this strategy would actually work in some significant fashion? That it would work in low-income elementary schools, where we face our greatest challenges? What makes him want to promote the idea that this approach is the silver bullet? What draws him to this teacher-based approach, in lieu of everything else?
Miller cajoled and implored throughout, sounding very human. But because weve actually worked in low-income schools, we had our reservations. We also had some reservation because weve looked at some actual studies. And sure enough! Finally, at the start of his very last paragraph, Miller coughed this up:
MILLER (10/10/10): Some U.S. researchers say there's little evidence that teachers with stronger academic backgrounds produce higher student achievement
That came right after Miller said that the social and economic returns could be enormous if we recruit smarter teachers (our emphasis).
Miller is certainly right on one score. The social and economic returns could be enormousbut then again, the returns could be minuscule! And good lord! If some researchers say theres little evidence, shouldnt somebody check their claims before we go stampeding off? In her recent column on Waiting for Superman, Gail Collins did make one excellent point. She offered this account of the way these things have frequently worked:
COLLINS (9/30/10): Older teachers tend to respond to calls for education reform with cynicism because they've been down this road so many times before. In 1955, a best seller, ''Why Johnny Can't Read,'' stunned the country with its description of a 12-year-old who suffered from being ''exposed to an ordinary American school.'' Since then, the calls for reform have come as regularly as the locusts. Social promotion has been eliminated repeatedly, schools have been made bigger, then smaller.
Its true. Quite routinely, someone comes up with some new silver bullet; we stampede off in that direction, with our experts running hard to get to the front of the queue. Today, the solutions all seem to revolve around the idea that our teachers, and their infernal unions, have somehow caused the massive problems found in our low-income schools. (Unmistakably, this simplistic message has been folded into a decades-old war directed at unions in general. In making this complaint, the reformers never mention the actual rise in test scores these infernal teachers have helped produce over the past dozen years.) Miller offers a courteous, upper-end form of this current silver bullet. But like the other faddish reformers, he discusses teacher improvement and nothing else, giving the unmistakable sense that this will solve our problems.
Will it? In some situations, smarter teachers would probably help; being smart is an advantage in all sorts of circumstances. But is academic ranking really the answer to the challenges we face in our low-income elementary schools? As we read Millers screed, we kept thinking of those studies of Teach for Americastudies which seemed to show that the bright young Princeton kids didnt do better than regular teachers, except perhaps at the high school level, pretty much as you might expect.
Those studies seemed to suggest that academic smarts may not be a silver bullet. But so what? The Bloomberg/Klein/Charlie Rose crowd is in love with the idea of the bright young Ivy Leaguers condescending to save the black kids. (For certain elites, this has been a pleasing novel for at least forty years.) They have pimped TFA to the ends of the earth, even when studies dont support the wildly embellished claims the org makes on its own behalf.
TFA tends to embellish a lot. This seems to be fine with the swells.
Well admit itwe recalled those studies as we read Millers true-believing piece. And then, hot damn! Right there in his closing paragraph, he finally gave us the semi-bad newsand of course, like all space invaders, he promptly flew to Finland! Here is Millers full closing paragraph, in which he returns to a theme he pushed all through his piece. Finland and Singapore do it, he says. That makes it the answer for us:
MILLER: Some U.S. researchers say there's little evidence that teachers with stronger academic backgrounds produce higher student achievement, but this conclusion is starkly at odds with the experiences of Singapore, Finland and South Korea. Our McKinsey colleagues have studied more than 50 school systems around the world and have never seen a nation achieve or sustain world-class educational performance without drawing its teachers from the top third of their class. Should we really bet our children's future on the possibility that our country might be the exception?
Correlation isnt causation, of course. Except when experts and swellsand space invaders?get on their latest roll.
All through his piece, as it has been scripted, Miller describes the wonders of Finland. If Miller is a space invader, could this be the tip-off? Could this be the flaw in the programming of these human-seeming visitors? By some minor technical error, have they perhaps been programmed to discuss things in Finland too much?
Back to Millers closing paragraph, which is hugely important:
McKinsey is a think tank; it has prepared a report on this topic. Finland is a middle-class, high-literacy nation which has done good things with its schools. For ourselves, we wouldnt oppose an attempt to lure smarter students into the classroombut will that strategy produce good results in our nations low-income schools? We dont have the slightest idea, although our experience suggests that there are other, very major factors that teachers cant address by themselves, no matter how good their college grades were. And were sorry, but you cant find out by flying to Finland. Or even to Singapore.
Here at THE HOWLER, we would support Millers basic idea. Presumably, its a good idea to lure smarter college students into the teaching profession. But a camp-meeting fervor surrounds the current, rolling attack on our teachersand, as noted, its neatly packaged with a decades-long war against unions in general.
It might be better to get better teachersbut would that approach be enough? In the case of Millers specific idea, would it work in our low-income schools? In our low-income, first-grade classrooms? And by the way: Are there any other approaches which might also be pursued? In the current teacher-trashing environment, gangs of roving superintendents, reformers and experts blame our problems on bad teachers. As youll note, they rarely suggest that we look for ways to attract smarter people to the jobs they hold.
Miller wants to attract smarter teachers. But when it comes to our struggling low-income schools, does he actually know what hes talking about? Or is he just chasing the current fad? With single-minded devotion, he pursues a version of the reform which all the experts are talking about. But as even Collins was able to note, the track record of these experts is embarrassing, awful.
No ones track record looks worse than Kleins in the wake of Mondays news report. But so what! On Sunday, the Outlook section put his teacher-trashing manifesto right at the top of its page one, next to Millers piece. But:
Does Joel Klein know whats he talking about? Could it be that he has been sent here from Mars?
Well consider those questions tomorrow. Quick guesses: No, and perhaps.
Tomorrow: Chancellor Kleins manifesto
Friday: What ETS doesnt know