Executive Summary
This study tested 33 distinct interventions designed to reduce affective polarization. Using a weighted regression model with inverse probability weighting (Offer-Westort, Coppock, & Green, 2021), we compared each intervention against a pure control condition.
Study Design: Adaptive Experiment
This was an exploratory study designed to screen many interventions with limited resources. Assignment probabilities varied across treatments, requiring inverse probability weighting for valid inference. The goal is to identify promising approaches for future research.
Pattern Summary: Grouping interventions by modality and mechanism reveals some suggestive patterns. Video-based interventions showed the most consistent negative effects (7 of 12 reduced AP), while text/article-based interventions were more mixed. Interactive elements (quizzes, forms) showed no clear pattern. However, confidence intervals are wide and most effects are not statistically distinguishable from zero.
Treatment Effects: Affective Polarization
Effects relative to pure control using WLS: post_ap ~ treatment + pre_ap with IPW weights and HC2 robust SEs. Negative estimates are better (reductions in affective polarization).
| ID | Category | Description | N | Effect | 95% CI | P |
|---|
Patterns by Intervention Category
Rather than focusing on individual treatments, we can examine patterns across intervention categories:
Video-Based (12 treatments)
Mean effect: -1.06 | Range: [-2.65, +1.31]
7 of 12 showed negative point estimates. Includes the only significant result (T15: perception gap video). Videos may enable greater emotional engagement.
Text/Article-Based (12 treatments)
Mean effect: -0.48 | Range: [-1.76, +0.89]
More mixed results. Factual corrections and quote-based appeals showed minimal effects. Scandal-based interventions trended negative.
Image-Based (4 treatments)
Mean effect: -0.96 | Range: [-2.63, +0.38]
Small sample of treatments. Media critique image (T24) showed large negative effect but wide CI.
Interactive (5 treatments)
Mean effect: +0.14 | Range: [-0.09, +0.86]
Quiz-based interventions showed null or slightly positive (backfire) effects. Active engagement did not outperform passive exposure.
Note: These patterns are exploratory and should not be interpreted as causal claims about modality effects. Sample sizes within categories vary considerably.
Measures
Affective Polarization
Measured via feeling thermometers (0-100 scale):
"We would like you to rate them using something we call the feeling thermometer. Ratings between 50 degrees and 100 degrees mean that you feel favorable and warm toward them. Ratings between 0 degrees and 50 degrees mean that you don't feel favorable toward them and that you don't care too much for them. You would rate them at the 50 degree mark if you don't feel particularly warm or cold toward them."
- How would you rate Republicans?
- How would you rate Democrats?
Affective polarization = inparty rating - outparty rating. Higher values indicate greater polarization.
Support for Undemocratic Practices
Four-item index (1-7 agreement scale):
"Please indicate your agreement and disagreement with the following statements."
- [Inparty] should reduce the number of polling stations in areas that support [Outparty]
- [Inparty] governors should ignore unfavorable court rulings by [Outparty]-appointed judges
- [Inparty] governors should prosecute journalists who accuse [Inparty] politicians of misconduct without revealing sources
- [Inparty] should not accept the results of elections if they lose
Items were tailored to respondent's party identification. Index = mean of four items.
Control Condition
Pure control (T0): Respondents saw only "Please continue." with no intervention content.
Methodology
Design
Respondents were recruited via Lucid Theorem (November 9-25, 2022) and randomly assigned to one of 34 conditions. Assignment probabilities varied across treatments based on an adaptive design (Offer-Westort, Coppock, & Green, 2021), requiring inverse probability weighting for valid inference.
Statistical Model
We estimate treatment effects using weighted least squares with robust (HC2) standard errors:
Each treatment coefficient represents the effect relative to pure control, adjusting for pre-treatment levels.
Exclusions
T33 (GPT-3 chatbot) was excluded due to technical failures (n=5). Analyses use remaining 33 treatments plus control.
Reference
Offer-Westort, M., Coppock, A., & Green, D. P. (2021). Adaptive experimental design: Prospects and applications in political science. American Journal of Political Science, 65(4), 826-844.
Replication Materials
- replication_data.csv — Survey data
- replication_script.R — R script (WLS with IPW weights, HC2 robust SEs)
Student Contributors
Interventions were designed by students in the Experimental Design course at Columbia University (Fall 2022):
Cameron Adkins, Caroline Alpi, Matias Alvarez, Matthew Amitay, Faith Andrews, Joseph Beckman, Bekim Bruka, Alex Centeno, Ryan Chapman-Schnupp, Zirui Chen, Aidan Conner, Ralph Diab, Jana ElShabrawishi, Crystal Foretia, Nastasya Generalova, Natalie Goldberg, Kyle Gordon, Sarah Heppen, Jack Holmgren, Kerri Kendrick, Marissa Keogh, Emeline Kuether, Dylan Lee, Bradley Lockett, Cindy Lu, Lauren Maitland, Makennan McBryde, Evan Myrdal, Taylor Ormasen, Aja Saldana, Claire Schweitzer, Scott Slater, Tomas Taaffe, Kate Tadeo, Genesis Vanegas Calvo, Carla Villafane Quinones, Jackson Weinberger