Sign up for The Media Today, CJR’s daily newsletter.
Last week, a Washington Post headline provoked a collective groan of embarrassment: apparently “seven percent of all American adults” think that chocolate milk comes from brown cows.
Unsurprisingly, The Internet took notice. Jokes about American intelligence were plentiful. Many palms hit many faces.
ICYMI: Ouch! These are headlines editors probably wish they could take back
I first saw the story when a journalist I respect posted the link to Twitter. Like most people, I recoiled in shame and horror, but also found the remainder of the article thought provoking and worthwhile. The chocolate milk question was part of a survey, commissioned by a dairy advocacy group, ostensibly to gauge what Americans know about agriculture and food production. It turns out: not very much. It was a reminder of the importance of knowing where our food comes from, a topic that touches on other relevant issues, like the much-buzzed-about rural-urban divide and the modern economy.
By the following day, NPR had picked up the story, running with a similarly lighthearted take. Those tuning into All Things Considered heard hosts Ari Shapiro and Audie Cornish mix in audio clips from Jean Ragalie-Carr, president of the National Dairy Council:
SHAPIRO: A recent survey looked into Americans’ beliefs about chocolate milk.
RAGALIE-CARR: When we asked them, where does chocolate milk come from, they indicated that they thought it came from brown cows.
SHAPIRO: Seven percent of Americans thought that.
CORNISH: Jean Ragalie-Carr is president of the National Dairy Council, which commissioned the survey. She says they put that question to a thousand people and gave them several options for how to answer.
RAGALIE-CARR: Well, there was brown cows or black-and-white cows, or they didn’t know.
A careful listener’s ear may have perked up at this exchange. How exactly was the question phrased? Were those the only three options – two cow colors or “I don’t know”? And did this mean that even someone who plainly knew that chocolate milk was simply any milk that had been mixed with chocolate and sugar was not given the option of choosing anything resembling the correct response? “Comes from” is heavy-handed phrasing in and of itself, implying the chocolate milk emerges as is, without human intervention.
ICYMI: The story behind “one of the best reported pieces of the year”
And the kicker: In this context, if only 7 percent of respondents thought chocolate milk came from brown cows, shouldn’t the real story be that 93 percent thought it either came only from black-and-white cows or had no idea how chocolate milk was made?
It may feel a little silly to quibble with something so unimportant. This shouldn’t be a big deal – a little click-bait as a way into a deeper conversation, and a momentary distraction from the barrage of grim political news.
But particularly now, journalists don’t have the luxury of playing fast and loose with the facts.
We’ve become accustomed to seeing these kinds of poorly phrased survey questions pop up and go viral because of some bonkers statistic they claim to support. The build-up/tear-down cycle is exhausting, and “the media” come off looking either lazy and gullible, or malicious for trying to mislead the public.
Those of us concerned with news literacy and public trust in media feel let down when one of these stories fools us across such a wide array of platforms. The problem isn’t this one survey and subsequent coverage. The evergreen problem is that if we feel like we can’t trust journalists to vet the small stuff for us, we worry that we can’t trust them with the big stuff, either. And we don’t need to be reminded that public trust in media is incredibly fragile right now.
Like so many stories, it seems this one was originally hatched as a PR pitch. An advocacy organization, the Innovation Center for U.S. Dairy, commissioned the survey via a marketing firm, Edelman Intelligence, to kick off a campaign called Undeniably Dairy. A spokesperson for the Innovation Center told me the purpose of the survey was to “gauge some interesting and fun facts about consumers’ perceptions of dairy,” and the chocolate milk stat was apparently a winner. (She declined to respond to my queries about the wording of the questions, and said the full results of the survey were not intended to be published.) Food & Wine magazine was the first to bite, on June 1 (World Milk Day), but the ball didn’t really get rolling until the Post jumped in two weeks later.
ICYMI: The New York Times reporter who tweets like it’s going out of style
As far as intent versus reception, the Post article uses the statistic as a hook to talk about food production and agriculture literacy, though presumably some people stopped reading after the funny lede. The NPR spot didn’t go into great depth about anything besides the chocolate milk tidbit. VICE followed suit. CNN doubled down on highlighting the outrageous stupidity of those making the claim, and then spoon-fed its audience other stats from the survey, all of which may be equally dubious. The distinction between the 7 percent of 1,000 survey respondents and 7 percent of adult Americans was murky at best – the Post did the math and deduced that 16.4 million Americans must believe that chocolate milk comes from brown cows.
(Amazingly, the Post followed up the next day with a piece about how this particular example isn’t a very worrying aspect of public ignorance. It too, however, failed to ask any questions about the origins of the data it was citing, and added that 48 percent of survey respondents – a number cited by Food & Wine, too, but whose origin I haven’t been able to ascertain – claim not to know where chocolate milk comes from. If we add this number to the 7 percent and recall Ragalie-Carr’s three-option response set, I guess we’re supposed to learn that 45 percent of those surveyed think chocolate milk comes from black-and-white cows, making the 7 percent pointing to brown cows officially the least impressive statistic in the bunch. That a piece intending to reassure us about ignorance didn’t pause to ask questions about the quality of information it was citing may be the most concerning aspect of the entire exercise.)
There were a few hints of healthy skepticism. The HuffPost noted that it’s difficult to gauge reliability when you don’t have any context. There’s also the grain of salt that some respondents are certainly trolling the pollsters with a knowingly ridiculous answer.
In the course of trying to dig into this simple clarification – the wording of one survey question – I kept thinking about how unreasonable it was to place such a burden on the news consumer. Over the course of several days, I spent dozens of hours reading every version of this reporting I could get my hands on, repeatedly went down rabbit holes following links I hoped would lead me to the raw source material, attempted to get spokespeople from three separate entities with ties to the story to respond to my request for clarification, and knew the entire time that none of it actually mattered in any big-picture way – an absurd waste of time, even for someone who works in journalism. This is not a process we should expect the average citizen to undertake every time they’re puzzled by a fact put forth by a major news outlet.
This is usually the part of the media critique where we’d look to assign blame, lament the decline of journalism training and the impossible speed of the online era, note that there are five PR specialists for every journalist, etc.
But we’ve been here so many times before, and blame seems almost moot when the scale of the problem looms so large in America’s psyche. This particular version of “fake news” is pervasive and insidious.
The News Literacy Project has been working on this issue for years, and has identified a number of questions students can ask themselves as they try to evaluate the veracity of a new story. What are the sources? Does the language seem geared to provoke outrage? Have other hard news publications reported the information in the same way?
It’s that last point that rankles in this particular situation. That serious, respected outlets like The Washington Post and NPR ran with this story feels like a failure. Genuine “fake news” – stories drawn from thin air – may have had its 15 minutes even before the president co-opted the term to mean anything he personally disliked, but these subtler versions continue to haunt anyone invested in our collective ability to parse fact from fiction. As a parent might say to a conscientious teenager who screwed up, “I’m not mad, I’m just disappointed.” To make progress on news and information literacy, both news producers and news consumers need to do their part.
As we slog through the question of how to better prepare the next generation to read past the headlines, it’s worth a plea to our most trusted news outlets to help us get there.
ICYMI: Equipment you’ll need to start your own podcast
Has America ever needed a media defender more than now? Help us by joining CJR today.
