13 April 2001
Our current howler (part II): See NAEP test
Synopsis: The Times reviewed NAEPs state-by-state scores. But they worked at a Dick-and-Jane level.
Test Results From States Reveal Gaps In Learning
Kate Zernike, The New York Times, 4/9/01
"F" for School Reform
Michael Kelly, The Washington Post, 4/11/01
The NAEP released its state-by-state scores. Again, the headline cried out in pain. "Test Results From States Reveal Gaps In Learning," it said. But what kinds of gaps did the test scores "reveal?" Kate Zernike explained one of the gaps:
ZERNIKE (paragraph 4): The gaps between the top-scoring students and the lowest-scoring students remained wide, suggesting that in some fourth-grade classrooms, some students are doing fourth-grade work while others are working at the first-grade level
According to Zernike, the state-by-state test scores "suggested" this: "In some fourth grade classrooms, some students are doing fourth-grade work while others are working at the first-grade level." The statement is simply remarkable. The NAEP tested 8000 students in forty states, we are told, and this is what the results "suggested" (or "revealed"). But who on earth didnt already know that some fourth graders are so afflicted? Surely, anyone who has spent ten minutes in urban classrooms would know that many delightful, deserving kids are years below traditional grade level, even by fourth or fifth grade. Did the NAEP tests really "suggest" this fact? Was this "revealed" by the results? This is like saying a study "suggests" that some football players weigh more than 200 pounds.
Later, Zernike continues describing what these new NAEP scores "reveal." She quotes John Barth, a senior education associate on the panel which oversees the NAEP:
ZERNIKE (7-8): Mr. Barth said a rule of thumb was that 10 points equaled what a student learned in one grade
The best fourth-grade math students, Mr. Barth said, were doing 36 points better than the worst eighth-grade math students. "At best, those eighth graders are doing work on a third-grade level," he said. "Its rudimentary arithmetic. I dont think anyone can believe those kids are going to exit their education with the skills and knowledge they need."
Again, for whom can this possibly come as newswhen tests reveal that some eighth-graders are working on third-grade level (or below)? We dont blame Barth for stating the obviousthat kids like this are in a fine mess. We do express substantial surprise when the Times seems to think this is news.
Who doesnt know that many kids are far below traditional norms? On Monday, the Times seemed not to. But then, the Times seemed not to know other things too, as we read this second-day article. As such, this article helped display the Dick-and-Jane level on which the ed press often works.
ITS BEEN THIRTY-FIVE YEARS SINCE OUR CITY SCHOOLS came center stage, delivered by a set of first-person books written by young urban teachers (Death at an Early Age, Jonathan Kozol; 36 Children, Herbert Kohl). The books helped awaken the nations conscience, and, after 400 years of doing everything possible to eliminate literacy in the black community, the nation began to put urban schooling high on its public agenda. Urban ed became an official priority; teaching at a city elementary school was a draft deferment during Vietnam, for example. The fact that massive numbers of urban kids were reading far below traditional grade level was described again and again at the timerightly decried as a national crisis.
But then a few things happened. It soon became clear that it wouldnt be easy to fix this newly deplored situation. The "urban ed" books had sometimes suggested that a new, improved attitude was all it would take; city kids would achieve like anyone else if their teachers would stop being racists. But low achievement in urban schools continued into the 1970sand major papers around the country began to tire of the topic. And now, an alternative "solution" began to appearschools began cheating on standardized tests, creating a pleasing new set of impressions, letting school districts swear to fleeing taxpayers that they were "headed in the right direction." All over the country, remarkable cheating scandals occurred, publicly described and thoroughly documented. But newspapers preferred to churn "happy talk" talespleasing stories about "schools that work," urban schools which produced super test scores. Despite the rash of cheating scandals, newspapers almost never made the slightest effort to find out how these scores were producedand major journals often stopped describing the overall plight of our troubled city schools. By the year 2001, a writer at the New York Times could say a remarkable thing. She could say that the newest NAEP testing "suggested" that some kids were doing quite poorlycould express surprise when told that fourth graders often work on a first grade level. How could it be that a major ed writer could possibly treat that fact as news? It reflects the way our city schools slid off the radar screen long agoand it reflects the disinterest our great papers bring to the plight of our struggling city schools.
IT WOULD BE GROSSLY UNFAIR TO BLAME the current reporter for this nationwide cult of indifference. But one part of Zernikes piece helped show how indifference about the public schools has been furthered just in the last year. On Monday, Zernike reported one clear piece of news; according to the newest NAEP results, she says, the achievement gap between black and white students hasnt narrowed in the past eight years. Near the end of her piece, she ties this fact to certain themes from Candidate Bushs campaign:
ZERNIKE (10): Picking up this theme, George W. Bush ran for president on a platform of leaving "no child behind" and praised test results that showed the minority achievement gap closing in Texas. But as in many states, that narrowing was based on state tests. And as president, Mr. Bush has said that states should have to verify the results of their tests by using the [NAEP] tests, making it a sort of educational second opinion.
(11) The test results to be released today suggest that states will have a hard time verifying their data. The Texas scores, for example, show that both the minority achievement gap and the gap between top and bottom scores remained the same, despite state tests showing that the gaps had narrowed.
Alas! On Texas state-run TAAS tests, the black-white gap had been narrowing. On the NAEP tests, the gap stayed the same.
Why, oh why, might that have happened? One obvious possibilityTexas schools may have cheated their keisters off on the state tests, over which they had administrative control. The NAEP tests are harder to gimmick. Is that what happened? We have no way of knowing. But we do know thislast April, several papers published stories suggesting that something was wrong with the TAAS (see links below). There had been TAAS cheating scandals all over Texas, tied perhaps to top-down pressure to improve those minority test scores. And there were indications that the narrowing black-white achievement gap on the TAAS could not be reproduced in other settings.
This issue was raised in the Washington Post, the Baltimore Sun, the Sacramento Bee, the Dallas Morning News and the New Republic. And what did our crackerjack press corps do? What else? They completely ignored it! In the fall, a RAND report further suggested the possibility that the TAAS gains were bunk; the press corps avoided that, too. What has actually gone on in the Texas schools? You dont have the slightest ideain large part because the press corps doesnt try to find out. Decades ago, the mainstream press corps walked away from the plight of our city schools. And even in the middle of a White House campaign, signs that were being badly spun still cant excite their attention.
Its been thirty-five years since Death at an Early Ageand the Times seems surprised when tests "suggest" that fourth-graders work on first-grade level. Why is the Times so surprised by this fact? Because we make no effort to sustain a discussion about our urban schools. Long after city schools came center stage, our press corps keeps rediscovering their problems. So forget about Johnny not readinghes ten. Whats the press corps current excuse for its sorry, Dick-and-Jane-like performance?
Visit our incomparable archives: Last spring, we urged the press to review the stories about apparent problems with the TAAS. See THE DAILY HOWLER, 5/19/00.
Last fall, we discussed the new RAND study casting doubt on the TAAS. See THE DAILY HOWLER, 10/31/00.
As usual, you heard it here first. We first discussed cheating on standardized tests in the fall of 1999. See THE DAILY HOWLER, 9/24/99.
The occasional update (4/13/01)
See Mike declaim: One of the problems with the ed discourse is that everyone knows hes an expert. On Wednesday, Michael Kelly proved the point in his blustery piece in the Post. Example: Kelly noted that the average NAEP score in fourth grade reading was unchanged from 1992 to 2000. The furious scribe then said this:
KELLY (8): Meanwhile, "high-performing" students did better [in 2000]32 percent scored at or above proficient and 8 percent at advanced, up from, respectively, 29 percent and 5 percent in 1992. Ambitious parents who send their children to the increasingly demanding schools that serve the meritocratic overclass will not be surprised to learn that this improvement directly corresponded to an increase in reading and homework assignments since 1992.
Kelly reports a modest change in the numbers over an eight-year span. These numbers, of course, represent the performance of two national samples of fourth-grade students. Kelly assumes that this minor change in performance by the sample groups represents an actual change in the overall fourth-grade population. And he even goes on to let us know why this alleged change has occurred.
Indeed, Kelly not only knows why the high are getting higher. He also knows why the low are (allegedly) getting lowerand he knows whose fault that is. The fault, of course, is President Clintons. And the pundit seizes the latest chance to drag Chelsea in for no reason on earth:
KELLY (9): So this is the 21st century to which Bill Clinton built a bridge, a nation with a growing and entrenched lifelong chasm between those (like Chelsea Clinton) who grow up in ambitious families and attend demanding schools and those who dont have such luck; a nation where a stunning 60 percent of poor children and minority children are shoveled through the schools and out the other end, largely illiterate and innumerate.
Kelly never fails us. Were the upper ranks of the nations fourth graders really better equipped in the year 2000? To be honest, you cant tell from NAEPs numbers. Were the lower ranks really doing less well? Trust usKelly dont know that one either. The change in performance by the sample group seems to be slight, and there are a hundred ways in which the NAEP testing can involve minor technical error or chance variation. (Were the 1992 and 2000 tests really equivalent? Were the two sample groups perfectly chosen? Does the NAEP really measure overall reading skill? Were the tests administered the same way both years?) Throughout this piece, Kelly swears that Dumbbell Clintons plans made absolutely no sense at allbut he puts an unskeptical, crackpot faith in the work of the quite-human NAEP.
Kelly dont seem to know what hes talking about. At least theres been no change in that.
[For the record: In other settings, we have aggressively criticized the education plans of both the Clinton and Bush I administrations. Each administration stressed "raising standards"a sounds-good solution that has nothing to do with what is actually wrong in city schools. When fourth graders are reading on first grade level, for example, they do not need "higher standards." And by the way: if these kids are denied social promotionanother favorite sounds-good solutionthey will still be sitting in fourth-grade classrooms in which "some students are doing fourth-grade work while others are working at the first-grade level." The actual state of the urban classroomrarely described in the major mediacompletely defies the sounds-good solutions which press experts routinely applaud.]