Sign up for The Media Today, CJR’s daily newsletter.

Algorithm-based news recommendations on social media and search engines shape the news people see. Social media is the main source of news for 42% of adults in the United States — representing a 55% increase since 2013. Worldwide, 28% of users use social media as their main gateway to news, for the first time surpassing direct access to publishers. Such “side-door” routes like social media, search engines, and mobile aggregators are especially popular among younger audiences.

But what news do algorithms show us? With little transparency on how these services determine what information will be presented to users, many are concerned that today’s algorithmic gatekeeping feeds filter bubbles, facilitates the spread of disinformation, and worsens polarization. These fears remain widespread even though the empirical evidence is still mixed, as again and again, studies on exposure to news on digital platforms find little support for these concerns.

To get a sense of how algorithmic recommendations perform as news intermediaries, we asked actual internet users to complete real-world searches of topics in the news and list the recommendations they got. In other words, we conducted a crowdsourced audit of search results, the full findings of which were recently published in Mass Communication & Society.

We examined several major platforms that are key portals to news today: Facebook, Google (Search), Google News, Twitter, and YouTube. Some of these services offer multiple ways of accessing content, but we chose to focus on search-based news discovery. We were particularly interested in the top results, since users are most likely to interact with them.

First, we wanted to understand the news diets that individual participants were offered. Are users being given results that appear to be personalized to their political preferences and social characteristics?

Second, at the macro level, we were curious to see which organizations tend to receive the most exposure in these environments. Do different platforms produce diverging levels of homogeneity in search results and/or highlight different types of sources in searches? Similarly, do different search terms yield different levels of homogeneity and/or different sources?

WHAT WE DID

We asked real users to sign in to their online accounts and perform searches using a given search term across five platforms within a short window of time. This was intended to reduce potential confounders, or variables we couldn’t account for, like the daily rhythms of news production and other algorithmic parameters. The five platforms were: Facebook, Google (Search), Google News, Twitter, and YouTube.

The search terms were ‘crime’, ‘immigration’, ‘state Republicans’, and ‘Trump’ — a mix of general interest and political topics. Each term was searched by a different group of participants on a different date, essentially replicating the study four times to increase the robustness of our findings.

After conducting each search, the participant was asked to paste the links for the top three results they encountered into an online form (with guidance provided for each platform). Participants were then asked to self-report their geographical location, political leanings, and demographic characteristics.

A total of 1,598 ideologically, geographically, and demographically diverse U.S. participants completed the searches, providing 28,475 valid links (link validity rate: 99%). The number of respondents per each search term was roughly the same. Demographically, the median age was 35, 51.1% of respondents self-identified as “Male,” and 76% self-identified as White (only) and non-Latinx. The median income level was $50,000 to $59,999, and 48.3% of the respondents self-reported having a bachelor’s degree or higher. Ideologically, the sample skewed left of center, with Liberals receiving the highest representation (44.6%), followed by Moderates (40.8%) and Conservatives (14.5%). The geographical distribution of respondents across the four top-level U.S. Census Regions was in line with recent U.S. American Community Survey estimates.

This approach offers a more realistic examination of how a diverse set of actual human users was served by these search functions, which are among the main gateways to news and information today. 

WHAT WE FOUND

Despite concerns about fragmentation of the public sphere, we found that the recommendation algorithms on platforms did not promote filter bubbles, but rather homogenized exposure to information, at least among the top results. This effect was evident in every platform studied and across all search terms used. We can’t determine if this has always been the case or if it’s new, but clearly, the information diets that platforms deliver are more homogenized than is often assumed.

The information diets offered by these major platforms promote a centralized winner-takes-most structure. Who were the winners? Our results generally pointed to news organizations with national reach in search results, which were featured at a much higher rate than local and regional news outlets.

But each platform employs its own set of algorithmic logics, which has editorial impacts. Google News and Google’s (Search) Top Stories — two services that specifically focus on professional news — tended to recommend legacy news organizations, primarily major print-based publications. This is in line with previous findings on Google News. On YouTube, also owned by Alphabet, the top performers were national TV networks, including both cable news and traditional broadcasters. Some digitally native outlets did break through, such as BlazeTV, True Crime Daily, and Vox

A different picture emerged on Facebook, Twitter, and the results below the Top Stories on Google (Search), where most top results led to non-journalistic sources. On Google (Search), information repositories like Wikipedia and the websites of organizations relevant to the search topic were granted the most visibility. On Twitter, many of the highlighted sources were individual journalists with large followings (but top results also included a mix of legacy, digital-born, and TV news organizations). Facebook’s search results offered an even mix of links to Facebook Groups, professional news sources, entertainment, and official sites of relevant government agencies. This heterogeneity is perhaps not surprising, given Facebook’s announcements in recent years that its services would prioritize strengthening interpersonal relationships over circulating professional news.

At the same time, digital-born conservative outlets, notably the Washington Examiner and the Daily Caller, were more salient on Facebook searches than on the other platforms studied. This is consistent with other accounts showing that right-wing news performs particularly well on Facebook.

(Follow this link for PDF showing breakdowns of: Proportion of participants who were exposed to a source, by political ideology; Most salient sources by platform; and Proportion of participants who were exposed to a source, by geographical location.)

WHY THIS MATTERS

Our results challenge the conventional wisdom that online platforms’ search algorithms push users into filter bubbles through personalization, and that conservative voices, as Republicans often claim, are being intentionally stifled by Big Tech. Instead, the results support concerns about a winner-takes-most paradigm, with a small number of predominantly national content producers — Fox News prominently among them — receiving much of the attention.

Our results point to another kind of risk, though. Offering increasingly homogenized results that promote incumbent organizations when users search for news could lead to less diversity and plurality in public discourse — not to mention the further entrenchment of a handful of publishers at the expense of opportunities for other news organizations to be discovered by audiences.

While polarization has been much discussed in recent years, concentration and homogenization of news markets may come with its own costs. Algorithmic distribution seemingly benefits a small set of mostly national winners at the expense of smaller sources. This increased homogenization is occurring at a time of increasing market concentration, at least in the United States. Marginalizing local news media and diminishing their opportunities for visibility contributes to the crisis in local news, suggesting the need to support the types of news organizations that have consistently underperformed in searches.

We noticed a substantial change in the performance of news sources identified as politically conservative, most notably Fox News. In a previous study conducted in 2016 that followed similar methods, Fox News was hardly represented in Google News recommendations. In the current study, Fox News repeatedly appeared among the top results on Google News and Google’s (Search) Top Stories, and it was especially salient on YouTube. Across all platforms, Fox News came up nearly as often as The New York Times and The Washington Post — with those being the three most salient news organizations. While our sample sizes and research design preclude us from generalizing at the organizational level, it does provide further evidence against claims that Big Tech erases conservative viewpoints by design — at least when it comes to search. The representation of slanted news sources on digital platforms is a complex issue, and clear principles are hard to form, operationalize, and explain. However, conservative sources — particularly Fox News — were consistently prominent in our findings. In fact, the findings illustrate that they can be far more influential than their limited direct consumership would suggest.

Platforms are not a monolith, and their distinct editorial outcomes may reflect both philosophical and commercial considerations. For example, Facebook in recent years has announced changes to its algorithms to deprioritize news and emphasize social and family connections, while Google News prioritizes journalistic legitimacy. So, news consumers may want to diversify their media diets not only in terms of the news sources they engage with but also with respect to the platforms and services they use as gateways to news.

Click here to read the journal article in full at Mass Communication & Society.

Has America ever needed a media defender more than now? Help us by joining CJR today.