HOW SWEET IS CHARLOTTE? Have Charlotte’s schools really made big gains? The data suggest they have not: // link // print // previous // next //
HARD WORK VS. THE JOYS OF THE TRIBE: As readers know, we’re inclined to think the Plame story is overrated. But how amazing can your “press corps” be? In this morning’s Post, Carol Leonnig discusses the thrilling new Bob Woodward matter. Here is the start of her paragraph 3, in which she makes an inaccurate statement:
LEONNIG (11/17/05): Woodward testified Monday that contrary to Special Counsel Patrick J. Fitzgerald's public statements, a senior government official—not Libby— was the first Bush administration official to tell a reporter about Plame and her role at the CIA.According to this presentation, Fitzgerald said that Libby “was the first Bush administration official to tell a reporter about Plame and her role at the CIA.” But as everyone and his uncle has noted by now, Fitzgerald actually said that Libby was the first official known to do this. And guess what? Leonnig knew this fact all along! Incredibly, here’s her paragraph 11:
LEONNIG: Libby's lawyers have asked whether Fitzgerald will correct his statement that Libby was the first administration official to leak information about Plame to a reporter. Fitzgerald's spokesman, Randall Samborn, declined to comment. But a source close to the probe said there is no reason for the prosecutor to correct the record, because he specifically said at his news conference Oct. 28 that Libby was the "first official known" at that time to have provided such information to a reporter.Incredible, isn’t it? Leonnig knew what Fitzgerald had actually said—but she typed up the wrong account anyway! We always marvel at work like this. Sadly, such work is fairly common.
By the way, a word on Woodward: Succumbing to the joys of the tribe, liberals are now scoring the scribe as a store-bought, stenographer, Bush Admin lackey. It feels very good to say such things, but we have a somewhat different view—partly because we actually read Woodward’s last book, Plan of Attack. Yes, there are some silly, Bush-friendly anecdotes in it, several of which we discussed when we did extensive critiques of the book. (The George Tenet “slam dunk” anecdote is the most significant. We even suggested that Woodward must have included it as some sort of quid pro quo for access.) But uh-oh! The book is also full of material which shows the Admin is a very bad light. In substantial detail, Woodward shows Cheney and Bush exceeding the state of the intelligence on Iraq starting in August 2002—and his portrait of Colin Powell preparing his UN report is deeply, deeply embarrassing to Powell. This book is full of material that incriminates the Admin. But few liberals have bothered to say this.
But then, you know how we liberals are! As George Bush has said, reading books can be “hard work”—and it seems that few of us bothered with Plan of Attack. Now we enjoy the pleasures of the tribe, crying, boo-hooing, and blubbering vastly about Vile Woodward’s deep perfidy. But what happened when Plan of Attack was released? Simple! Industrious conservatives grabbed the handful of Bush-friendly anecdotes (“slam dunk” in particular) and trumpeted these unlikely tales to the skies. Everyone on earth heard about them. And liberals, playing the role of the lazy grasshopper, did and said nothing about this. The most famous journalist in DC wrote a book full of Bush-bashing material. But reading books can be hard work. Today, we see a corollary: The joys of the tribe are quite easy.
Big picture? In the past twenty years, conservatives have developed a powerful message machine. They play the role of industrious ants. We sing and play, like the grasshopper.
VISIT OUR INCOMPARABLE ARCHIVES: We re-discussed Plan of Attack from June 21 through June 27 of this year, with links to previous, real-time reports. See THE DAILY HOWLER, 6/21/05, then click ahead to each part of the series.
Special Report: Gush, Gush, Sweet Charlotte!
PART 2—HOW SWEET IS CHARLOTTE: How well have the Charlotte-Mecklenburg schools done in the past dozen years? In Making Schools Work with Hedrick Smith, this large urban/suburban school district is the second (and final) school district feted for its allegedly potent reforms. The system’s self-praise is quite abundant; during the show, and on the program’s web site, former superintendent Eric Smith makes it sound like he invented a “science of teaching”—a “science” that has more or less eliminated achievement gaps in Charlotte schools. (“Again, I think the success of a school isn't dependent on the children we serve. We can compensate for any deficiencies or additional needs children bring to us.” See THE DAILY HOWLER, 11/16/05 for this and other striking statements.) But then, his journalistic namesake, Hedrick Smith, heartily vouches for Charlotte’s gains too. If we hope to improve the nation’s low-income schools, we must learn how to identify districts that are achieving unusual success, so that we can try to learn from their methods. So: How much success has Charlotte had? The question isn’t easy to answer, but we must refine the skills with which we hunt after “districts that work.”
How well has Charlotte-Mecklenburg done—in particular, for its low-income and minority students? On its web site, Making Schools Work makes a string of claims for the district, one of which we highlight:
“MAKING SCHOOLS WORK” WEB SITE: Charlotte-Mecklenburg under Smith took a centralized approach to reform... Smith and his staff set high expectations for all students and then promoted Equity-plus, a concept that insured low-performing inner city schools were given sufficient resources to lift their students to district-wide levels of achievement.Let’s repeat that highlighted claim: “[F]rom 1995 to 2001, the number of African-American students in Charlotte schools reading on grade level more than doubled, rising from 35 percent to 70 percent.” (Yes, we know: 35 to 70 is not “more than double.”) If true, that would be a fantastic achievement. But did these gains really occur?After eight years of system-wide reform, Charlotte-Mecklenburg shot to the top on NAEP tests in early 2004, with each ethnic group outperforming its peers in nine other major urban school districts in reading and math. Moreover, from 1995 to 2001, the number of African-American students in Charlotte schools reading on grade level more than doubled, rising from 35 percent to 70 percent. What is especially significant is that Charlotte maintained educational momentum and continuity after Eric Smith left the district in 2002. The new team, headed by his former deputy superintendent, James Pughsley, pursued the original goals, thereby sustaining and building on the current reforms.
How about it? From 1995-2201, did Charlotte really double the number of black kids reading on grade level? As we have noted in the past, North Carolina maintains an excellent web site providing the state’s testing results. And no—according to the most relevant data, Charlotte’s black kids didn’t “more than double” their passing rate on the state’s mandated reading tests during the period in question. The state’s end-of-grade reading tests are administered to all students in grades 3-8. For grades 3-8 combined, here are the percentages of Charlotte black students “at or above grade level:”
1995: 39.7 percentIn fact, the district’s passing rate among black kids went from 40 percent to 57. That may seem like a pretty good hike—until you see what was happening elsewhere. But the basic claim on the web site doesn’t seem to be accurate. (For the record, we sent two e-mails to Hedrick Smith asking about the source of his data. Details below.)
2001: 57.2 percent
But so what if the web site overstated these gains? Isn’t the actual gain in Charlotte’s black passing rate really quite dramatic? Our answer: Yes, it would be fairly dramatic—if we knew that the data really mean what they seem. But remember: A gain in “passing rates” doesn’t mean anything, unless the tests in question have maintained their level of difficulty over the years. And uh-oh! In this case, reading scores on several measures seem to suggest that North Carolina’s statewide tests have gotten easier. Hang on, because the following chain of reasoning is extremely important—if we want to refine our skill at identifying “districts that work.”
From 1995 to 2001, the passing rate for black kids in Charlotte went from 40 percent to 57. But uh-oh! During this same period, the passing rate among black kids rose all over the state. Indeed, from 1995 to 2001, the passing rate for black kids statewide jumped from 47.4 percent to 60.7. The Charlotte rate did move up more than that of the state as a whole (17 percent to 13 percent). But the rate was rising among all school systems—suggesting the possibility that the state reading tests were, in some way, getting easier.
This point becomes a bit more clear when we look at the rise in passing rates over a longer period. According to the North Carolina state data, passing rates in reading among black kids have soared in the past eleven years (testing began in 1994):
Passing rates in reading, North Carolina end-of-grade tests, African-American students, grades 3-8:Wow! Forget about Charlotte—if you put your faith in appearances, North Carolina as a state has worked a revolution in the past eleven years. In 1994, 43 percent of black kids statewide passed the end-of-grade reading tests. By this year, the rate had jumped to 76 percent! But then, all groups of kids in North Carolina were improving their passing rate during this period. Here are the passing rates for Tarheel kids as a whole:
1994: Charlotte, 36.0 percent. Statewide, 43.2 percent.
2005: Charlotte, 75.6 percent. Statewide, 75.8 percent.
Passing rates in reading, North Carolina end-of-grade tests, all students, grades 3-8:Wow! Forget the black kids—statewide, among all kids, the passing rate in reading jumped from 58 percent to 86 percent. That would suggest a giant gain in achievement. But are these surface appearances real? Or did the state tests get easier?
1994: Statewide, 57.5 percent.
2005: Statewide, 85.7 percent.
Alas, the state’s performance on the National Assessment of Educational Progress (NAEP) suggests the latter explanation. The NAEP is generally regarded as a good “control” against which to measure exploding scores on various state-run testing programs—programs which are subject to political pressure to make passing rates go up. And on the NAEP, North Carolina kids have made almost no progress in reading over this eleven-year period (click here, then click on North Carolina). Here are the numbers of state fourth-graders passing the test at the “basic” and “proficient” levels over this time span:
Achievement rates in reading, North Carolina, all students, grade 4, NAEP:On the NAEP, the achievement rates ticked just up a hair. And at grade 8—the only other grade tested on the NAEP—achievement rates have declined since 1998, the first year North Carolina eighth-graders were tested:
1994: 59 percent passed at the “basic” level; 41 percent scored “below basic.” (30 percent were judged “proficient.”)
2005: 62 percent passed at the “basic” level; 38 percent scored “below basic.” (29 percent were judged “proficient.”)
Achievement rates in reading, North Carolina, all students, grade 8, NAEP:On the NAEP, North Carolina achievement rates were stagnant—ticking up slightly in fourth grade, declining (by a greater margin) in eighth. In short, North Carolina’s NAEP scores betray no sign of the vast improvement suggested by the state’s own tests. This suggests that the statewide tests have gotten easier in some way over the years. If we actually care about what is true, we’ll be slow to accept the impression given by the large score gains on the North Carolina end-of-grade tests.
1998: 74 percent passed at the “basic” level; 26 percent scored “below basic.” (30 percent were judged “proficient.”)
2005: 69 percent passed at the “basic” level; 31 percent scored “below basic.” (27 percent were judged “proficient.”)
What does this mean about Charlotte’s schools? Has Charlotte really achieved massive gains in grade-level reading among its black kids?
We’d have to judge that it has not. In the past eleven years, black kids in the Charlotte system have greatly improved their passing rate on the statewide reading tests. Indeed, they have improved their passing rate slightly more than blacks have done on a statewide basis. But over the course of these eleven years, the Charlotte system is doing slightly better than a state population which has shown no gains on the NAEP. Safest guess: Charlotte-Mecklenburg has improved a tad. The statewide population has been stagnant.
If we actually care about what is true, we won’t rush to accept those glittering scores on those North Carolina state tests. Next question: Has Charlotte done slightly better than the state because of the Eric Smith reforms? In the next few days, we’ll address that question. But note the lessons that we learn from this fairly simple analysis. Making Schools Work let viewers think that the Charlotte schools had made massive gains among its black kids in recent years. There’s a word for this sort of thing: Cheerleading. But then, such uninquisitive boosterism has long dominated education reporting. Press organs luvv to find “schools and districts that work”—even where the evidence is very shaky, as it most assuredly is when we gush, gush over sweet Charlotte.
TOMORROW: One possible source of improvement.
FOR THE RECORD: Where did Making Schools Work get the following data: “Moreover, from 1995 to 2001, the number of African-American students in Charlotte schools reading on grade level more than doubled, rising from 35 percent to 70 percent.” We don’t know. We e-mailed the program twice, asking for the source of these numbers. (It was the only question we sent.) Hedrick Smith responded to our first e-mail, but provided no answer. We sent our question again, but got no reply. The claim may simply be a mistake, or it may come from some other set of data. But those NAEP scores suggest that North Carolina has made few gains in reading achievement since 1994. Charlotte’s score gains on the statewide tests slightly exceed those of the state as a whole. But Charlotte is exceeding a state population which seems to be holding place—and is only exceeding it by a slight margin. More on this matter tomorrow—and then, a look at the system’s “reforms.”
GROWTH/NO GROWTH: For the record, here are the passing rates for all students in Charlotte and North Carolina as a whole over the past eleven years. Again, these are the rates achieved on the state-run, state-devised reading tests:
Passing rates in reading, North Carolina end-of-grade tests, all students, grades 3-8:Again, Charlotte’s score gains do seem huge—but the state’s gains are almost as large. This same state has shown no growth on the NAEP tests over this period. If we care about what’s true, we’ll look long and hard at this pattern.
1994: Charlotte, 52.8 percent. Statewide, 57.5 percent.
2005: Charlotte, 84.0 percent. Statewide, 85.7 percent.