Kosuke Imai’s Research on Design and Analysis of Randomized Experiments and Program Evaluation

Overview

Randomization of treatment assignment is one of the most powerful tools for the development of modern science. The fields of political science and public policy, which had been long dominated by observational studies, have also begun to witness growing use of experimental studies. Similarly, in public policy, a greater demand for accountability and evidence-based policy-making has led to the use of experiments for program evaluations.Although one may think statistical analysis of randomized experiments is straightforward, this is not usually the case in social science randomized experiments where experimental subjects are human beings. Many complications occur especially when experiments are conducted in the field. In this project, we develop statistical methods for addressing these complications that commonly occur in randomized experiments and program evaluation. In particular, (1) we show that Fisher’s randomization inference can be applied to natural experiments when the randomization of treatment assignment may use a complex scheme, (2) we develop statistical methods for dealing with noncompliance, nonresponse, and measurement error problems, (4) we develop statistical methods for identifying and estimating heterogenous treatment effects and interaction effects, and (5) we develop methods for estimating spillover and contagion effects.

Manuscripts and Publications

Randomization inference:

Ho, Daniel E., and Kosuke Imai. (2006). “Randomization Inference with Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election.” Journal of the American Statistical Association, Vol. 101, No. 475 (September), pp. 888-900.

Ho, Daniel E., and Kosuke Imai. (2008). “Estimating Causal Effects of Ballot Order from a Randomized Natural Experiment: California Alphabet Lottery, 1978-2002.” Public Opinion Quarterly, Vol. 72, No. 2 (Summer), pp. 216-240.

Missing data and measurement error:

Horiuchi, Yusaku, Kosuke Imai, and Naoko Taniguchi. (2007). “Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment.” American Journal of Political Science, Vol. 51, No. 3 (July), pp. 669-687.

Imai, Kosuke. (2008).“Sharp Bounds on the Causal Effects in Randomized Experiments with “Truncation-by-Death”.” Statistics & Probability Letters, Vol. 78, No. 2 (February), pp. 144-149.

Imai, Kosuke. (2009). “Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes: An Application to a Voting Experiment.” Journal of the Royal Statistical Society, Series C (Applied Statistics), Vol. 58, No. 1 (February), pp. 83-104.

Imai, Kosuke, and Teppei Yamamoto. (2010). “Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.” American Journal of Political Science, Vol. 54, No. 2 (April), pp. 543-560.

Imai, Kosuke, and Zhichao Jiang. (2018). “A Sensitivity Analysis for Missing Outcomes under the Matched-Pairs Design.” Statistics in Medicine, Vol. 37, No. 20 (September), pp. 2907-2922.

Estimation of treatment effect heterogeneity and interaction effects:

Imai, Kosuke, and Aaron Strauss. (2011). “Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-out-the-vote Campaign.” Political Analysis, Vol. 19, No. 1 (Winter), pp. 1-19. (lead article) Winner of Political Analysis Editors’ Choice Award.

Imai, Kosuke and Marc Ratkovic. (2013). “Estimating Treatment Effect Heterogeneity in Randomized Program Evaluation.” Annals of Applied Statistics, Vol. 7, No. 1 (March), pp. 443-470. Winner of the Tom Ten Have Memorial Award.

Egami, Naoki, and Kosuke Imai. (2019). “Causal Interaction in Factorial Experiments: Application to Conjoint Analysis.” Journal of the American Statistical Association, Vol. 114, No. 526 (June), pp. 529-540.

de la Cuesta, Brandon, Naoki Egami, and Kosuke Imai. “Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution.” Political Analysis, Vol. 30, No. 1 (January), pp. 19-45.

Imai, Kosuke and Michael Lingzhi Li. “Experimental Evaluation of Individualized Treatment Rules.” Journal of the American Statistical Association, Forthcoming.

Goplerud, Max, Kosuke Imai, Nicole E. Pashley. “Estimating Heterogeneous Causal Effects of High-Dimensional Treatments: Application to Conjoint Analysis.”

Ham, Dae Woong, Kosuke Imai, and Lucas Janson. “Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis.”

Estimation of spillover effects:

Jiang, Zhichao and Kosuke Imai. “Statistical Inference and Power Analysis for Direct and Spillover Effects in Two-Stage Randomized Experiments.”

Imai, Kosuke, Zhichao Jiang, and Anup Malani. (2021). “Causal Inference with Interference and Noncompliance in Two-Stage Randomized Experiments.” Journal of the American Statistical Association, Vol. 116, No. 534, pp. 632-644.

Imai, Kosuke, and Zhichao Jiang. (2020). “Identification and Sensitivity Analysis of Contagion Effects in Randomized Placebo-Controlled Trials.” Journal of the Royal Statistical Society, Series A (Statistics in Society), Vol. 183, No. 4 (October), pp. 1637-1657.

Statistical Software

Imai, Kosuke. “experiment: R Package for Designing and Analyzing Randomized Experiments.” available through The Comprehensive R Archive Network. 2007.

Egami, Naoki, Marc Ratkovic, and Kosuke Imai. “FintIt: R Package for Finding Heterogeneous Treatment Effects.” available through The Comprehensive R Archive Network. 2012-2015.

Funding

National Science Foundation, (2008-2009). “New Statistical Methods for Randomized Experiments in Political Science and Public Policy,” (Political Science Program; SES-0752050).