Study: Americans struggle to distinguish factual claims from opinions amid partisan bias

How well do Americans succeed at distinguishing statements of fact from statements of opinion? The answer: Not very well at all, according to new research co-written by a team of University of Illinois Urbana-Champaign scholars.

Americans struggle to tell the difference between statements of fact and statements of opinion – a troubling trend that has grave implications for civic discourse and for navigating the torrent of political information that citizens receive every day, said Jeffery J. Mondak, a professor of political science and the James M. Benson Chair in Public Issues and Civic Leadership at Illinois.

“The capacity to differentiate between a statement of opinion and a statement of fact is vital for citizens to manage the flood of political information they receive on any given day,” said Mondak, a co-author of the research and an affiliate of the Center for Social and Behavioral Science. “There’s a huge amount of research on misinformation. But what we found is that, even before we get to the stage of labeling something misinformation, people often have trouble discerning the difference between statements of fact and opinion.”

“We also see a lot of research on misinformation that comes at the problem from the angle of, ‘How are we doing in terms of playing whack-a-mole with misinformation? Are we able to fact check them and rebut these claims?’ Well, that isn’t necessarily a useful way of getting at the root cause of the problem,” said Matthew Mettler, a U of I graduate student and co-author of the paper.

The study, published by the Harvard Kennedy School Misinformation Review, examined whether Americans can differentiate statements of fact (2 + 2 = 4, for example) versus statements of opinion (“Green is the most beautiful color”), with a particular focus on political statements.

Respondents were asked to categorize 12 statements about current events as statements of fact or statements of opinion. The researchers found that 45.7 percent of respondents performed no better than a coin flip at the task.

“What we’re showing here is that people have trouble distinguishing factual claims from opinion, and if we don’t have this shared sense of reality, then standard journalistic fact-checking – which is more curative than preventative – is not going to be a productive way of defanging misinformation,” Mondak said. “How can you have productive discourse about issues if you’re not only disagreeing on a basic set of facts, but you’re also disagreeing on the more fundamental nature of what a fact itself is?”

Four factors were associated with modest improvements in success at fact-opinion differentiation: civics knowledge, current events knowledge, education, and cognitive ability. 

But partisan bias played a strong role in the root cause of error, the scholars said.

“As partisan political views grow more polarized, Democrats and Republicans both tend to construct an alternate reality in which they report that their side has marshalled the facts and the other side merely has opinions,” Mondak said.

Unbiased error, such as mistakes resulting from random guessing, decreased due to the four factors, but errors linked to partisan bias didn’t budge, Mondak said.

“It’s not merely the case that there were a lot of incorrect responses, but that many of the errors were not random,” he said. “They were systematic errors because many respondents formed their answers to fit their partisan narrative. For example, the statement ‘President Barack Obama was born in the U.S.’ is a statement of fact that could be incorrectly redefined as a statement of opinion, depending on your partisan lens.”

“Although people with greater political sophistication were better at distinguishing fact from opinion, affective partisan polarization tends to promote systematic partisan error,” Mettler said. “It distorts people’s capacity to reason their way through these statements.”

For citizens who are concerned about misinformation, the findings suggest not only that people are resistant to corrections of misinformation, but also that they’re susceptible to manipulation, the researchers said.

“Our analyses show that the problem of misinformation includes an underappreciated dimension in that people do not just disagree on the facts, they also disagree on the more fundamental matter of what facts are,” Mondak said. “The results also suggest that faulty fact-opinion differentiation can severely complicate the correction of misinformation because a consensus of ‘We can agree to disagree’ can emerge even for questions of indisputable fact. Well, you can’t just ‘Agree to disagree’ that 2 + 2 = 22.

“It’s possible that news media can help improve fact-opinion differentiation by highlighting the distinction between statements of fact and statements of opinion in their stories or broadcasts. But the trend nowadays, especially on cable news, is more of a blurring of opinion and fact.”

The implications of the research ought to be highly concerning for political discourse, especially during the 2024 presidential election cycle, the researchers said.

“If you can’t tell if somebody is proposing a statement of fact versus a statement of opinion, you’re doomed as an information consumer,” Mondak said. “It signals a fundamental breakdown in the possibility of meaningful communication between people and political elites, or between journalists and the public.”

The research was supported by the Benson Fund at the U. of I.