28 September 1999
Our current howler (part I): Playing the margin
Synopsis: Gore led Bradley in all three polls. At the Post, that meant they were even.
Bradley Pulls Even in N.H.
Dan Balz, The Washington Post, 9/19/99
Gore slips in New York (Inside Politics)
Greg Pierce, The Washington Times, 9/16/99
More proof (Inside Politics)
Greg Pierce, The Washington Times, 9/15/99
Republican road to White House suddenly bumpy
Jill Lawrence, USA Today, 9/16/99
Moynihans defection to Bradley is latest gore
Susan Page, USA Today, 9/24/99
Bradley Closes Gap With Gore
John Harwood, The Wall Street Journal, 9/16/99
Here at THE HOWLER, we have no idea who's going to win the
Democratic nomination. We don't know who's going to win in New
Hampshire. We don't maintain a polling unit. All we know is what
we read in the newspapers.
But our analysts really jumped to attention at a page one headline
in the September 19 Post. "Bradley pulls even in N.H.,"
said the headline, sitting at the top of the big fat Sunday paper.
Our analysts were surprised because, at the time, they knew
of no poll that had shown the New Hampshire race even. And they
couldn't help chuckling when they read the data contained in the
Dan Balz report.
Balz reported on three separate polls of likely New Hampshire
Democratic voters. All three polls showed Al Gore ahead of rival
Bill Bradley, by margins of four, five and seven points.
The polls were closer than earlier Granite State polls, in
which Gore had enjoyed a much healthier lead. But how had the
Post said the race was even when Gore was ahead in all
The answer? It's that persistent ol' debbil, "margin of
error," the device routinely used to make races seem tighterwhen
the press corps wants a race nice and tight. The truth is, when
three separate polls showed Gore ahead by four to seven points,
the odds were extremely slimvery slim indeedthat the
race was actually tied. But don't tell that to the Washington
press corps, which never met a technical measure that it can't
misapply. No, nothing much will ever turn on the error lodged
in the Post headline that morning. But we thought it might be
worth taking a day to enjoy a good chuckle at "margin of
error," before going on to examine more significant aspects
of recent poll/survey reporting.
The Post headline reflected an increasing trend among political
reporters. When the difference between two hopefuls is less than
a poll's "margin of error" (perhaps doubled), writers
declare the race a "statistical tie." Before the Post
headline on the 19th, the trend had been widely exhibited in reporting
on New Hampshire/New York polling. Here's Greg Pierce on September
PIERCE (9/16): In a match-up with Republican front-runner George
W. Bush [in New York], Mr. Bradley led 47 percent to 37 percent.
Mr. Gore, meanwhile, was in a statistical tie with the Texas
governor; 46 percent for the vice president and 43 percent
for Mr. Bush.
This formulation has been seen again and again in recent polling
stories. It helps explain why others, besides the Post headline
writer, had declared Bradley to be even in New Hampshire. Jill
Lawrence, on September 16:
LAWRENCE: Bradley, the former New Jersey senator and former
professional basketball player, is showing increasing strength
in key primary states. Polls show him in statistical ties with
Gore in New Hampshire and New York.
The New Hampshire polls to which Lawrence referred were the
same polls described in the Post's later story. Susan Page seemed
to be a bit more careful with her language, but seemed to imply
the same thing:
PAGE: In three recent statewide polls, Bradley has pulled
neck-and-neck with Gore in New Hampshire...A Marist Institute
Poll conducted last week out the two men in a perfect 42%-42%
tie in New York.
Apparently, Page discerns two kinds of "ties;" "perfect"
ties, where the numbers are even, and other "ties,"
where they're not.
Again, without getting into the actual stats, when three separate
polls showed Gore four to seven points ahead, it was very unlikely
that the race was tied. (It's harder to say that from one single
poll.) But let's focus on one particular polla WMUR-CNN poll
released on September 13. Here's how Greg Pierce described it:
PIERCE (9/15): A WMUR-CNN poll released Monday showed Mr. Gore
is favored by 46 percent of likely voters in the 2000 Democratic
presidential primary, compared with 41 percent who support Mr.
Bradley. That was within the poll's margin of error of 5 percentage
Was that poll a "statistical tie?" It's not
clear what the term even means. It is certainly possible
that, had all Dems been polled, the result would have been a tie.
A single poll can't prove that a survey of all Dems would have
broken down 46-41 for Gore. But, while it is possible that Gore
and Bradley were actually tied, it is equally possiblerepeat,
equally possiblethat Gore was actually in a ten-point
lead. ("Margin of error" doesn't work just one way.)
But we defy you to find any writer who reported these results
as a "statistical ten-point lead" for Gore.
Nopethe rule is, make the story more thrilling. So the press
corps routinely cites one possibility while failing to mention
the other. In doing so, it misapplies the concept of "margin
of error," and gins up a term"statistical tie"that
would be hard to explain or defend in these contexts.
By the way, some writers have avoided these undesirable impulses,
reporting these polls in simple, plain language that doesn't pretend
to be technical. Here, for example, was Pierce's lead paragraph
on that WMUR poll:
PIERCE (9/15): Bill Bradley, boosted by his strength among
independent voters, is in a close race with Vice President Al
Gore in the Democratic presidential race in New Hampshire, says
a new poll that echoes the results of two recent surveys.
Is 46-41 a "close race?" That, of course, is a judgment
call, but Pierce didn't pretend that the hopefuls were now somehow
known to be "tied." And John Harwood, in the Journal,
made the whole thing look easy. Here's how he described the New
Hampshire polls, in a story concerning a new nationwide survey:
HARWOOD: The [nationwide] survey, on the heels of recent
polls showing Mr. Bradley drawing close to Mr. Gore in the critical
primary state of New Hampshire, suggests the ex-basketball
star is moving into a strong position to capitalize on an early
breakthrough in the coast-to-coast crush of primaries next month.
Indeed, that is exactly what the three polls most likely reflectedBradley
now was close to Gore (and improving his position). They
did not show that Bradley was "even" or "statistically
even," although excitable scribes wanted to say so.
One last point: despite the headline in the September 19 Post,
Balz had not said that Bradley was even. Here is what he
had actually said:
BALZ: Five months before the nation's first primary, former
New Jersey senator Bill Bradley has virtually pulled even
with Al Gore in New Hampshire, turning a lopsided advantage for
the vice president into a fiercely competitive contest.
Later in his article, he referred to the "narrowing"
of the New Hampshire race. The headline misstated his writing.
Yepas Balz's writing showed (overlooking the headline), it's
possible to describe close poll results without goosing them up
with references to "statistical ties." The WMUR-CNN
poll had shown a "statistical" five-point lead.
Scribes would be better off simply saying it.
Tomorrow: A recent Pew survey revealed a Big Fact that
the press corps dare never utter.
Has anyone here seen Fred: We remain concerned about
Fred Wertheimer, still missing from Ceci Connolly's finance stories.
On Saturday, Connolly penned a story in the Post about Gov. Bush's
successful fund-raising. Connolly reported that Bush "is
on track to bring in more than $75 million for the primary campaign,"
and may even skip federal funds for the general election. Realistically,
that would mean that Bush would have to raise around $150 million
In our view, there's nothing wrong with thatmore power to
him, if the governor can swing it. But remember how upset Connolly
was, back in April, at the thought that Al Gore might raise around
$35 million? The good-guv scribe was worried sick that a hopeful
would dream of raising such sums. So she called in Fred
Wertheimer to fire off quotes about the problems with amassing
In her Saturday piece, Connolly imagines Bush raising five
times as much, and not a word is said about any troubling implications.
Has Connolly lost Fred Wertheimer's number? THE HOWLER is sick