University of Texas at Austin communications professor Natalie J. Stroud discussed her research on correcting misperceptions of news spread through social media in the latest installment of the Harvard Kennedy School Shorenstein Center’s virtual misinformation speaker series Wednesday.
Working in collaboration with five fact-checking organizations worldwide, Stroud used Facebook sponsored posts to gauge how changing the forms of news headlines on an article posted to social media could affect the way people perceived the information the article contained. The group studied three types of headlines: “question,” in which the headline only questions but does not assert; “assertion,” which states correct information only; and “refutation,” which states, then refutes, a problematic claim.
In order to evaluate the effectiveness of fact-checking in the real world, “where people are inundated with information,” her study also accounted for the ability of a headline to draw attention.
“We’ve had a lot of research on fact-checking that show people misinformation, and show people a fact check, and it’s in the context of an experimental setting that doesn’t have any of the real-world attributes,” Stroud said.
Explaining the study’s methods, Stroud said the project was “one of the first academic uses” of Brand Lift, a tool typically used commercially to measure the impact of a brand’s Facebook ads.
Facebook users included in her study were asked to complete a survey after viewing the ad, she said. The survey garnered more than 60,000 responses from five countries and recorded information such as the user’s answers to true-false questions about the article and whether the user actually clicked to read the article.
The fact that clicks were roughly the same across the board was “bad news” for the question format, in which the headline contains no assertions at all, Stroud said.
“In terms of attention, there actually aren’t that many differences,” she said.
Stroud found the refutation headline was most effective in terms of learning or knowledge acquisition.
“Refutation delivers a superior performance,” Stroud said. “They’re reducing the ‘don’t know’s and they’re increasing the percentage of people that are correctly answering the question.”
She concluded that “there may be some efficacy to using refutation,” though she added it is important to keep the reasons behind that choice in mind.
“It comes from reducing the number of people who aren’t sure, and it does not come from reducing the number of people who answer these questions incorrectly,” Stroud said.
In addition to using more refutation headlines, Stroud emphasized the importance of conducting more research on reducing misinformation on social media.
“It’s doing these sorts of studies to really find out how people are responding in real-world contexts. Looking at headline format is one component of a fact check, but there are so many others, so I think that the method is also something that could be employed more generally, and that would ultimately help to have social media correct misperceptions,” Stroud said.
