New Research Shows Online Ads Have Limited Impact on Consumer Valuations for Facebook

Policymakers are increasingly debating the relative social costs and benefits of digital advertising. While ads typically benefit platforms and advertisers, do they disadvantage consumers? In a new study, researchers analyzed advertising on Facebook, comparing the effects on consumers of seeing ads versus not seeing ads. The study found no significant differences in users’ valuations between the two groups, suggesting that either the harmful effects of ads are relatively small or that certain benefits offset the harms.

The study, by researchers at Carnegie Mellon University, Stanford University, Northwestern University, and Meta, appears as a new NBER working paper.

“In theory, the effect of online ads on consumer welfare could go either way,” explains Avinash Collis, assistant professor of management science and economics at Carnegie Mellon’s Heinz College, who led the study. “Ads may annoy users with irrelevant content or they may help them find useful products and services.”

Research on the causal effects of online advertising on consumer welfare is limited due to the challenges of running large-scale experiments and tracking effects over extended periods. In this study, researchers analyzed a long-running field experiment of online advertising on Facebook, which began in 2013 and continues today. In that work, a random 0.5% subset of the entire user base (nearly 3 billion people) was assigned to a group that never sees ads while the rest of users experience their usual levels of advertising.

In 2022, researchers recruited a representative sample of more than 53,000 Facebook users in the ads and no-ads groups. Participants were invited to take part in a survey, which included an incentivized online choice experiment that measured users’ valuations of Facebook.

The study found that users’ valuations did not differ statistically between the two groups. This indicates that users did not experience a significant difficulty from current levels of advertising, which suggests that a no-ads environment may not be particularly beneficial for consumers, the authors concluded. Any negative impact of ads on user satisfaction is likely to be smaller than 10% of how much they value Facebook.

In determining why this occurred, researchers suggested that it could be because the beneficial effects of some well-targeted ads (e.g., helping users locate products and services of interest) cancelled out the annoyance of irrelevant ads. Or, they posited, it could be because both effects were so small that they did not reach statistical significance. The users in the ads group did spend 9.4% less time on Facebook than users in the no-ads group, but that didn’t seem to affect their overall valuations.

Among the study’s limitations, the authors say they evaluated only the usefulness or harm of ads on Facebook, not on other platforms. In addition, within Facebook, the no-ads group prevented users from seeing most but not all ads. More than half of the authors on the paper are from Meta, the parent company of Facebook. 

“In the United States and Europe, policymakers have proposed regulations which focus on the ability of online platforms to show targeted ads to consumers,” says Erik Brynjolfsson, a professor at Stanford’s Institute for Human-centered AI, who co-led the study. “Our study provides evidence from the largest and longest-running experiment to estimate of the utility and disutility of ads.”

###

Summarized from an NBER working paper, The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment, by Brynjolfsson, E (Stanford University), Collis, A (Carnegie Mellon University), Liawat, A (Meta), Kutzman, D (Meta), Garro, H (Meta), Deisenroth, D (Meta), and Wernerfelt, N (Northwestern University). Copyright 2024. All rights reserved.

About Heinz College of Information Systems and Public Policy
The Heinz College of Information Systems and Public Policy is home to two internationally recognized graduate-level institutions at Carnegie Mellon University: the School of Information Systems and Management and the School of Public Policy and Management. This unique colocation combined with its expertise in analytics set Heinz College apart in the areas of cybersecurity, health care, the future of work, smart cities, and arts & entertainment. In 2016, INFORMS named Heinz College the #1 academic program for Analytics Education. For more information, please visit www.heinz.cmu.edu.