“What to Do (and Not to Do) with Causal Panel Analysis: Parallel Trends, Heterogeneous Treatment Effects, and Diagnostics,” Yiqing Xu, Stanford University

QUANTITATIVE RESEARCH METHODS WORKSHOP

Abstract: Two-way fixed effects (TWFE) models are a popular choice for causal panel analysis in political science. Recent developments in the methodological literature repudiate the TWFE estimator in the presence of heterogeneous treatment effects (HTE) and violations of the parallel trends assumption. To diagnose or alleviate these issues, researchers have also proposed a variety of new estimators and testing procedures. How much do these problems matter for empirical political science research and how do new proposals compare in practice? We answer these two questions by reviewing, replicating, and reanalyzing 30 papers recently published in American Political Science Review, American Journal of Political Science, and The Journal of Politics that use observational panel data with binary treatments. Leveraging both visual and statistical tests, we find that parallel trends violations are a much bigger threat than HTE and that four newly-proposed HTE-robust estimators yield qualitatively similar findings when a strong pretrend is not observed. Moreover, underestimation of uncertainties, which leads to uncontrolled Type-I errors, remains a prominent issue in empirical panel analysis. Overall, we find that claims in one-third of the studies we review receive strong empirical support. We provide recommendations for empirical researchers to improve practice.

Yiqing Xu an Assistant Professor at the Department of Political Science at Stanford University. His primary research covers political methodology, Chinese politics, and their intersection. He received a PhD in Political Science from Massachusetts Institute of Technology in 2016. His work has appeared in American Political Science Review, American Journal of Political Science, Journal of Politics, and Political Analysis, among other peer-reviewed journals. He has won several professional awards, including the best article award from American Journal of Political Science in 2016 and the Miller Prize (2018, 2020) for the best work appearing in Political Analysis the preceding year.

This workshop is open to the Yale community. To receive announcements and invitations to attend, please subscribe at https://csap.yale.edu/quantitative-research-methods-workshop.

This series is sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.