Do Social Media Ads Matter for Political Behavior? A Field Experiment

(with George Beknazar-Yuzbashev)

Journal of Public Economics, 214 (2022): 104735 [Link to the Published Article (gated)]

Abstract: We exploit Facebook’s introduction of a filter hiding ads from the feed as a unique opportunity to study the effects of online ads on political behavior. In a pre-registered experiment, we randomly assigned participants to hide political ads (treatment) or alcohol ads (control) for several weeks preceding the 2020 US elections. We report an insignificant intent-to-treat effect of political ads on turnout (2.3 pp.), but we cannot rule out a sizable positive effect, with 95% confidence interval of [-2.8,7.4]. The result may mask important heterogeneity, with political ads making Democrats slightly more motivated to vote and Republicans – substantially less. We explore the reasons for this effect, such as natural variation in ad content: the majority of Facebook ads on users’ feeds skewed Democratic. Lastly, the effect on measures of affective polarization and informedness was negligible.

Toxic Content and User Engagement on Social Media: Evidence from a Field Experiment

(with George Beknazar-Yuzbashev, Rafael Jiménez-Durán, Jesse McCrosky)

Abstract: As much as forty percent of social media users have been harassed online, but there is scarce causal evidence of how toxic content impacts user engagement and whether it is contagious. In a pre-registered field experiment, we recruited participants to install a browser extension, and randomly assigned them to either a treatment group where the extension automatically hides toxic text content on Facebook, Twitter, and YouTube, or to a control group without hiding. As the first stage, 6.6% of the content displayed to users was classified as toxic by the extension relying on state-of-the-art toxicity detection tools, and duly hidden in the treatment group during a six-week long period. Lowering exposure to toxicity reduced content consumption on Facebook by 23% relative to the mean – beyond the mechanical effect of our intervention. We also report a 9.2% drop in ad consumption on Twitter (relative to the mean), where this metric is available. Additionally, the intervention reduced the average toxicity of content posted by users on Facebook and Twitter, evidence of toxicity being contagious. Taken together, our results suggest a trade-off faced by platforms: they can curb users’ toxicity at the expense of their content consumption.

Preempting Polarization: An Experiment on Opinion Formation 

(with Dan Kashner)

Revise and Resubmit - Journal of Public Economics

Abstract: Blind adoption of opinions put forward by political parties and influential figures can sometimes be harmful. Focusing on cases where the partisan gap on policy support has not yet arisen, we investigate whether its formation can be prevented by encouraging prior active engagement with non-partisan information. To address this question, we recruited N=851 Republicans for a study about net neutrality, an issue largely unfamiliar to the electorate. In a pre-registered experiment, we randomly changed the order in which the following two types of information were provided: (i) partisan, underscoring Republicans’ opposition and Democrats’ support, and (ii) non-partisan, where the participants evaluated factual arguments about the pros and cons of the policy. Despite holding total information constant, we find that those who saw the non-partisan block first donated 46% more to a charity advocating for net neutrality (p=0.001). We also report that the partisan information altered which arguments Republicans found convincing when viewing the non-partisan block. Lastly, as a robustness check, we provide evidence that the treatment effect on support for the issue persisted in an obfuscated follow-up study, conducted several weeks after the intervention.

To the Depths of the Sunk Cost: Experiments Revisiting the Elusive Effect

(with George Beknazar-Yuzbashev and Sota Ichiba)

Abstract: Despite being often discussed both in practice and academic circles, the sunk cost effect remains empirically elusive. Our model based on reference point dependence suggests that the traditional way of testing it – by assigning discounts – may not produce the desired effect. Instead, we evaluate the effect across the gain-loss divide by randomizing the price (low, medium, or high) of a ticket to enter a real-effort task and observe its effect on playtime. Despite a strong intervention – we varied the sunk cost by $2 for a 14-minute task – and the sample size of N=1,806, we find only a small effect (0.09 SD or 1.1 minutes). Additionally, we explore the importance of sunk cost in a field experiment on YouTube (N=11,208) in which we randomize whether the time until a pre-video ad becomes skippable is shortened (0 s), default (5 s), or extended (10 s). While the point estimates indicate an insignificant impact of sunk time on engagement, this is driven by a significant effect on the extensive margin – more users left before the video started in the extended treatment (6.3 pp. difference relative to the shortened treatment). Excluding such observations, our results are consistent with the sunk cost effect. Taken together, we offer a cautionary tale that applying even the most intuitive behavioral effects in policy settings might prove challenging.