Join us
campaign desk

Measuring a Speech’s Success

In speech reax stories, competing theories of politics
September 14, 2009

Sign up for The Media Today, CJR’s daily newsletter.

Barack Obama’s big health care speech last week has, in general, been deemed a success by the media and the political class. But beneath that consensus lie some interesting differences in the criteria used by observers to evaluate the president’s address—differences that, in turn, reveal competing theories about how political power is obtained and wielded. These are probably legion, but for the purposes of the present roundup, we can see three categories. Call them the straightforward polling theory, the sophisticated polling theory, and the it’s-not-about-the-polls theory.

First, the straightforward polling theory. The idea here is that a president speaks about a controversial topic, uses his silver-tongued charisma to persuade undecided but attentive listeners, and swings public opinion in his favor. Then Congress, responsive to the public will, passes a bill that represents what the public wants. Sen. Chuck Schumer, the New York Democrat, spelled out this logic in partisan terms: “I don’t think anyone thought after hearing the speech 12 Republican senators would get up and say, ‘I’m for you, Mr. President.’ But when they go back to their constituencies who did hear the speech, they may feel differently in terms of how to approach this.”

If you’re an Obama booster who thinks this way, you’re probably pretty happy at the moment, seeing as various post-speech polls and focus groups found that viewers were happy with what they heard. “Double-digit post-speech jump for Obama plan,” CNN declared. If these polls are sound, Obama’s achievement would be remarkable: research tells us that presidents generally have little success moving public opinion on domestic policy matters (though it also tells us that big-event speeches are probably the best tool they’ve got).

But, for reasons Mark Blumenthal describes here, the insta-polls are probably not very meaningful. In fact, as Blumenthal notes here, “instant reaction” to this speech was fairly typical—and, typically, “instant reaction” doesn’t amount to much in the long run.

This brings us to the sophisticated polling theory, which rests on essentially the same logic but addresses some of the critiques lobbed at the straightforward approach. Nate Silver advances this theory here. The gist of the argument is that criticisms of the “instant polls” that note their oversampling of Democrats aren’t really relevant here; on health care reform, Democrats—who don’t like Obama’s policy nearly as much as they like him—are a logical target (or, as Silver puts it, “the lowest-hanging fruit”). Silver frames the issue nicely:

Obama still has a lot of credibility with these people, and health care has become so central to his mission that it would seem that one or another of these things has to give: either their questions about health care will lead them to become disillusioned with Obama in general, or the health care numbers are due to bounce back.

Sign up for CJR’s daily email

Silver makes a persuasive case for “bounce back,” though not everyone would agree: in an interview two weeks ago, the political scientist Brandice Canes-Wrone described a similar hypothetical scenario, and said the most likely result would be a decline in the president’s approval ratings, not support for his policies.

That note of skepticism brings us to the final category, the it’s-not-about-(or least not all about)-the-polls theory. In this approach, we don’t need to puzzle over sampling bias and the other arcana of polling methodology, because Obama’s audience was not primarily the public—it was Congress. This theory, if never really explicit, is actually the subtext for some of the major media coverage about the event, including stories in The New York Times and Politico that noted the speech’s salutary reception by Congressional Democrats.

Both stories are chock-full of quotes from newly emboldened and energized Democrats. “The president’s speech breathed new life into what we’re doing,” says Max Baucus. “The president’s speech was really a game-changer,” says Harry Reid. “He clearly had an impact on our side of the aisle,” says Gerry Connolly. And Charles Rangel told the Times that the speech had had “a great effect, taking away that mean-spiritedness, that depth of tension, that negativism” that had overtaken the health care debate.

But politics is about more than good vibes. So under this theory, the speech succeeded because all those good feelings will translate into political capital—the president will get his way in Congress, while cutting out the public as the middleman. Isn’t that right, Rep. Rangel?

Or maybe not. From the Times:

“I don’t think he solved a thing in terms of votes,” Representative Charles B. Rangel, Democrat of New York, said of Mr. Obama’s speech. “I think he did a lot in changing the atmosphere.”

Hmm. Sometimes, it seems, success is a funny thing.

Has America ever needed a media defender more than now? Help us by joining CJR today.

Greg Marx is an associate editor at CJR. Follow him on Twitter @gregamarx.