How to Make Causal Inferences Using Texts

New text-as-data techniques offer a great promise: the ability to discover, measure, and then utilize text-based variables for testing social science theories of interest from large collections of text. We introduce a conceptual framework for making causal inferences with text-based measures as either a treatment or outcome. We argue that nearly all text-based causal inferences depend upon a latent representation of the text and provide a set of sufficient assumptions to identify causal effects when text is used as a treatment or outcome. We provide a framework to learn the latent representation"”justifying the use of popular unsupervised methods such as topic modeling or principal component analysis"”and then estimate causal effects with the same sample used to learn the latent representation. But estimating the latent representation, we show, creates new risks: we may introduce an identification problem or overfit. To address this problem we introduce a split-sample framework. We apply our framework to study whether increasing the proportion of women on Congressional committees leads to more representation of women's ideas during the legislative process and to assess how partisans respond to social media messages from President Trump.
Justin Grimmer is an associate professor of political science at Stanford University. His research examines how representation occurs in American politics using new statistical methods. His first book Representational Style in Congress: What Legislators Say and Why It Matters (Cambridge University Press, 2013) shows how senators define the type of representation they provide constituents and how this affects constituents' evaluations. His second book The Impression of Influence: How Legislator Communication and Government Spending Cultivate a Personal Vote (Under Review, with Sean J. Westwood and Solomon Messing) demonstrates how legislators ensure they receive credit for government actions. His work has appeared in the American Political Science Review, American Journal of Political Science, Journal of Politics, Political Analysis, Proceedings of the National Academy of Sciences, Regulation and Governance, and Poetics. During the 2013-2014 academic year he was a National Fellow at the Hoover Institute.