Caveat lector

Our current howler (part III): How they know

Synopsis: How does Pew know if a statement is "negative?" That’s easy—they just ask the boss.

The First 100 Days: How Bush Versus Clinton Fared in the Press
Project for Excellence in Journalism, 5/01

It’s almost impossible to overstate the ineptitude of Pew’s latest study. Pew studied seven news orgs—and seven only—and claimed to have studied "the media." And how did Pew select those orgs? The logic is hard to make out.

There are four broadcast news networks—Pew studied them all. Meanwhile, there are three weekly newsmags—Pew studied just one. And when it came to newspapers, bad judgment ran riot. Pew selected only two—the Washington Post and the New York Times—while noting that both are "reputedly liberal," with editorial boards that have "liberal attitudes." The lapse in judgment is simply surreal.

Imagine if Pew had studied the Wall Street Journal and the Washington Times, and then had used the resulting data to make sweeping statements about "the media." That would have shown bizarre judgment. And so does the mess which Pew did hatch, in which Pew managed to obscure the key point which its research showed—that even in the "reputedly liberal" papers, President Bush received more favorable news reporting than President Clinton received eight years earlier. As part of the overall mess which Pew conjured, its study opens with a nugget statement which seems to say precisely the opposite (see THE DAILY HOWLER, 5/22/01). For sheer, complete, yowling ineptitude, it rarely gets better than that.

Meanwhile, how does Pew know when a story is "negative" (or "positive")? Nothing in its report is worth a fig unless Pew can make that judgment. After all, in assessing the tone of the Bush/Clinton coverage, Pew rated every "story" it studied as "positive," "negative" or "neutral/balanced." Does Pew have a good way to make these assessments? When major journalists review Pew’s studies, they simply never ask (note Getler and Kelly last week).

But on face, these judgments would seem to be highly subjective, and Pew’s descriptions of its method are far from reassuring. Here is Pew’s first explanation of how the "tone" of each story was assessed:

But how does Pew know if a comment is "clearly negative?" That can be a tricky judgment, requiring a good ear for the political discourse. And is a news report "negative" just because it quotes people making "negative" statements? In its report, Pew offers fleeting overviews of some negative and positive stories. Consider one "negative" case:

According to Russert’s report, even Clinton’s strongest supporters were saying that he was stumbling. But if that’s true, it surely is news, and Pew doesn’t claim that Russert was wrong. But to Pew, reporting what Clinton’s supporters were saying seems to count as "negative" reporting. Presumably, even that part of Russert’s report would go down in the "negative" column.

Surely this is not what people mean when they ask if Clinton got a raw deal. But clearly, that is part of what Pew has in mind in judging news reports:

This is offered as an example of Bush’s "positive" coverage. But was the reporting true or false? If true—if "even many Democrats agree[d] that Bush ha[d] presided over" a superior transition—then surely the Times was right to say so. Again, this is surely not what most people mean when they wonder if Bush got an "easy ride"—but this is clearly one of the ways that Pew assesses its "stories." And consider one last example, of "positive" reporting on Clinton:

But were the drug companies resisting? If they were, and Hager reported it, is that an example of "negative reporting" as people understand the term? Surely, when people claim that Clinton got "negative coverage," they don’t refer to things like this—to a perfectly accurate statement that the drug companies opposed Clinton’s plan. We don’t know why Hager only quoted experts who said good things about the plan. (Was that the bulk of expert opinion?) But when he says the companies oppose the plan, should that be scored as a "negative" statement? Pew seems to score it as such.

It’s hard to make an objective assessment of whether reporting is "positive" or "negative." Routinely, this point is completely obscured by these Pew reports. Typically, Pew shows no sign of having any idea that they are offering, at best, a crude measure. And mainstream pundits quote Pew’s results as if they were gifts from the gods.

In fact, Pew is so much a part of the Official Press that it is virtually the PR arm of the press corps, offering timid critiques from the margin. And the entire press corps treats Pew’s work as if it were set in stone. The present, hopelessly flawed report shows the problems that result. Pew has grown exceptionally lazy. It claims that sixty days is the same as a hundred; thinks seven news orgs make up "the media;" and doesn’t seem to have the slightest idea of how crude a measure it employs. Indeed, part of the comedy in Pew report came in the one place where it addresses this last problem. How does Pew know that a statement is negative? Read what Pew itself said:

Sic, sic, sic! (Maybe Pew could hire some proofreaders.) But how does Pew know if a comment is "negative?" Simple, folks—they just ask the manager! We’re always intrigued by Pew’s apparent lack of awareness of the basic problems involved in its methods.

Next: There were far fewer stories on Bush. Politely, Pew ignored one key reason.

The press corps’ latest disgrace: We congratulate Salon for its new report on the "trashing of the White House" hoax. We strongly suggest that you read the report. You know what to do. Just click here.

Drilling for crude: The crudeness of Pew’s measure can be sketched in a thought experiment. Imagine a president coming to power in two parallel universes. He offers the same budget plan in each universe. But in one universe, he offers bribes to every member of Congress, and robs a bank to fund the effort. All his deeds become fully known. In that universe, the news reports would be full of people making "negative" statements. Would that mean that the president was a victim of "negative coverage?" Not as that phrase is normally understood. But to Pew, that president would be getting highly "negative" coverage. Conclusion? Whatever Pew is measuring with its described method, it bears a fairly crude relation to what people normally think of as "negative coverage."

One final caveat. Our bank-robbing president would get lots of "negative" stories—unless President Clinton also lived in that universe, and had recently gotten a speeding ticket. In that case, of course, our giver of bribes would get no press coverage at all.