The File Drawer

Interventions to Reduce Affective Polarization

Columbia University
IRB Protocol: AAAU3946

Executive Summary

This study tested 33 distinct interventions designed to reduce affective polarization. Using a weighted regression model with inverse probability weighting (Offer-Westort, Coppock, & Green, 2021), we compared each intervention against a pure control condition.

Study Design: Adaptive Experiment

This was an exploratory study designed to screen many interventions with limited resources. Assignment probabilities varied across treatments, requiring inverse probability weighting for valid inference. The goal is to identify promising approaches for future research.

Pattern Summary: Grouping interventions by modality and mechanism reveals some suggestive patterns. Video-based interventions showed the most consistent negative effects (7 of 12 reduced AP), while text/article-based interventions were more mixed. Interactive elements (quizzes, forms) showed no clear pattern. However, confidence intervals are wide and most effects are not statistically distinguishable from zero.

Total N
3,825
Complete cases
Survey Period
Nov 2022
Nov 9-25, 2022
Platform
Lucid
Theorem panel
Treatments
33
+ pure control

Treatment Effects: Affective Polarization

Effects relative to pure control using WLS: post_ap ~ treatment + pre_ap with IPW weights and HC2 robust SEs. Negative estimates are better (reductions in affective polarization).

ID Category Description N Effect 95% CI P

Patterns by Intervention Category

Rather than focusing on individual treatments, we can examine patterns across intervention categories:

Video-Based (12 treatments)

Mean effect: -1.06 | Range: [-2.65, +1.31]

7 of 12 showed negative point estimates. Includes the only significant result (T15: perception gap video). Videos may enable greater emotional engagement.

Text/Article-Based (12 treatments)

Mean effect: -0.48 | Range: [-1.76, +0.89]

More mixed results. Factual corrections and quote-based appeals showed minimal effects. Scandal-based interventions trended negative.

Image-Based (4 treatments)

Mean effect: -0.96 | Range: [-2.63, +0.38]

Small sample of treatments. Media critique image (T24) showed large negative effect but wide CI.

Interactive (5 treatments)

Mean effect: +0.14 | Range: [-0.09, +0.86]

Quiz-based interventions showed null or slightly positive (backfire) effects. Active engagement did not outperform passive exposure.

Note: These patterns are exploratory and should not be interpreted as causal claims about modality effects. Sample sizes within categories vary considerably.

Measures

Affective Polarization

Measured via feeling thermometers (0-100 scale):

"We would like you to rate them using something we call the feeling thermometer. Ratings between 50 degrees and 100 degrees mean that you feel favorable and warm toward them. Ratings between 0 degrees and 50 degrees mean that you don't feel favorable toward them and that you don't care too much for them. You would rate them at the 50 degree mark if you don't feel particularly warm or cold toward them."

- How would you rate Republicans?
- How would you rate Democrats?

Affective polarization = inparty rating - outparty rating. Higher values indicate greater polarization.

Support for Undemocratic Practices

Four-item index (1-7 agreement scale):

"Please indicate your agreement and disagreement with the following statements."
  1. [Inparty] should reduce the number of polling stations in areas that support [Outparty]
  2. [Inparty] governors should ignore unfavorable court rulings by [Outparty]-appointed judges
  3. [Inparty] governors should prosecute journalists who accuse [Inparty] politicians of misconduct without revealing sources
  4. [Inparty] should not accept the results of elections if they lose

Items were tailored to respondent's party identification. Index = mean of four items.

Control Condition

Pure control (T0): Respondents saw only "Please continue." with no intervention content.

Methodology

Design

Respondents were recruited via Lucid Theorem (November 9-25, 2022) and randomly assigned to one of 34 conditions. Assignment probabilities varied across treatments based on an adaptive design (Offer-Westort, Coppock, & Green, 2021), requiring inverse probability weighting for valid inference.

Statistical Model

We estimate treatment effects using weighted least squares with robust (HC2) standard errors:

post_outcome ~ treatment + pre_outcome, weights = 1/Pr(treatment)

Each treatment coefficient represents the effect relative to pure control, adjusting for pre-treatment levels.

Exclusions

T33 (GPT-3 chatbot) was excluded due to technical failures (n=5). Analyses use remaining 33 treatments plus control.

Reference

Offer-Westort, M., Coppock, A., & Green, D. P. (2021). Adaptive experimental design: Prospects and applications in political science. American Journal of Political Science, 65(4), 826-844.

Replication Materials

Student Contributors

Interventions were designed by students in the Experimental Design course at Columbia University (Fall 2022):

Cameron Adkins, Caroline Alpi, Matias Alvarez, Matthew Amitay, Faith Andrews, Joseph Beckman, Bekim Bruka, Alex Centeno, Ryan Chapman-Schnupp, Zirui Chen, Aidan Conner, Ralph Diab, Jana ElShabrawishi, Crystal Foretia, Nastasya Generalova, Natalie Goldberg, Kyle Gordon, Sarah Heppen, Jack Holmgren, Kerri Kendrick, Marissa Keogh, Emeline Kuether, Dylan Lee, Bradley Lockett, Cindy Lu, Lauren Maitland, Makennan McBryde, Evan Myrdal, Taylor Ormasen, Aja Saldana, Claire Schweitzer, Scott Slater, Tomas Taaffe, Kate Tadeo, Genesis Vanegas Calvo, Carla Villafane Quinones, Jackson Weinberger