skip to main content
research-article
Open Access

mRSC: Multi-dimensional Robust Synthetic Control

Authors Info & Claims
Published:19 June 2019Publication History
Skip Abstract Section

Abstract

When evaluating the impact of a policy (e.g., gun control) on a metric of interest (e.g., crime-rate), it may not be possible or feasible to conduct a randomized control trial. In such settings where only observational data is available, synthetic control (SC) methods [2-4] provide a popular data-driven approach to estimate a "synthetic" or "virtual" control by combining measurements of "similar" alternatives or units (called "donors"). Recently, robust synthetic control (RSC) [7] was proposed as a generalization of SC to overcome the challenges of missing data and high levels of noise, while removing the reliance on expert domain knowledge for selecting donors. However, both SC and RSC (and its variants) suffer from poor estimation when the pre-intervention period is too short. As the main contribution of this work, we propose a generalization of unidimensional RSC to multi-dimensional Robust Synthetic Control, mRSC. Our proposed mechanism, mRSC, incorporates multiple types of measurements (or metrics) in addition to the measurement of interest for estimating a synthetic control, thus overcoming the challenge of poor inference due to limited amounts of pre-intervention data. We show that the mRSC algorithm, when using K relevant metrics, leads to a consistent estimator of the synthetic control for the target unit of interest under any metric. Our finite-sample analysis suggests that the mean-squared error (MSE) of our predictions decays to zero at a rate faster than the RSC algorithm by a factor of K and √K for the training (pre-intervention) and testing (post-intervention) periods, respectively. Additionally, we propose a principled scheme to combine multiple metrics of interest via a diagnostic test that evaluates if adding a metric can be expected to result in improved inference. Our mechanism for validating mRSC performance is also an important and related contribution of this work: time series prediction. We propose a method to predict the future evolution of a time series based on limited data when the notion of time is relative and not absolute, i.e., where we have access to a donor pool that has already undergone the desired future evolution. We conduct extensive experimentation to establish the efficacy of mRSC in three different scenarios: predicting the evolution of a metric of interest using synthetically generated data from a known factor model, and forecasting weekly sales and score trajectories of a Walmart store and Cricket game, respectively.

References

  1. 2014. Kaggle: Walmart Recruiting - Store Sales Forecasting. (2014). https://www.kaggle.com/c/ walmart- recruiting- store- sales- forecastingGoogle ScholarGoogle Scholar
  2. A. Abadie, A. Diamond, and J. Hainmueller. 2010. Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program. J. Amer. Statist. Assoc. (2010).Google ScholarGoogle Scholar
  3. A. Abadie, A. Diamond, and J. Hainmueller. 2011. Synth: An R Package for Synthetic Control Methods in Comparative Case Studies. Journal of Statistical Software (2011).Google ScholarGoogle Scholar
  4. A. Abadie and J. Gardeazabal. 2003. The Economic Costs of Conflict: A Case Study of the Basque Country. American Economic Review (2003).Google ScholarGoogle Scholar
  5. Anish Agarwal, Muhammad Jehangir Amjad, Devavrat Shah, and Dennis Shen. 2018. Model Agnostic Time Series Analysis via Matrix Estimation. Proc. ACM Meas. Anal. Comput. Syst. 2, 3 (December 2018), 40:1--40:39.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Anish Agarwal, Devavrat Shah, Dennis Shen, and Dogyoon Song. 2019. Model Agnostic High-Dimensional Error-in-Variable Regression. arXiv preprint (2019).Google ScholarGoogle Scholar
  7. Muhammad Amjad, Devavrat Shah, and Dennis Shen. 2018. Robust Synthetic Control. Journal of Machine Learning Research 19, 17 (2018), 1--50.Google ScholarGoogle Scholar
  8. Muhammad J Amjad. 2018. Sequential Data Inference via Matrix Estimation: Causal Inference, Cricket and Retail. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  9. Michael Bailey and Stephen R Clarke. 2006. Predicting the Match Outcome in One Day International Cricket Matches, while the Game is in Progress. Journal of Sports Science & Medicine 5, 4 (12 2006), 480--487. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3861745/Google ScholarGoogle Scholar
  10. Elias Bareinboim and Judea Pearl. 2016. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences 113, 27 (2016), 7345--7352. arXiv: https://www.pnas.org/content/113/27/7345.full.pdfGoogle ScholarGoogle Scholar
  11. Christian Borgs, Jennifer Chayes, Christina E Lee, and Devavrat Shah. 2017. Thy Friend is My Friend: Iterative Collaborative Filtering for Sparse Matrix Estimation. In Advances in Neural Information Processing Systems. 4718-- 4729.Google ScholarGoogle Scholar
  12. Sourav Chatterjee. 2015. Matrix estimation by Universal Singular Value Thresholding. Annals of Statistics 43 (2015), 177--214.Google ScholarGoogle ScholarCross RefCross Ref
  13. Zhe Chen and Andrzej Cichocki. 2005. Nonnegative matrix factorization with temporal smoothness and/or spatial decorrelation constraints. Laboratory for Advanced Brain Signal Processing, RIKEN, Tech. Rep 68 (2005).Google ScholarGoogle Scholar
  14. Bradley Efron and Carl Morris. 1977. Stein's Paradox in Statistics. 236 (05 1977), 119--127.Google ScholarGoogle Scholar
  15. Seamus Hogan. 2015. Cricket and the Wasp: Shameless self promotion (Wonkish). (January 2015). http://offsettingbehaviour.blogspot.com/2012/11/cricket- and- wasp- shameless- self.htmlGoogle ScholarGoogle Scholar
  16. Christina Lee. 2017. Latent Variable Model Estimation via Collaborative Filtering. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  17. Phil Oliver, Travis Basevi, Will Luke, and Freddie Wilde. 2018. CricViz. (2018). http://cricviz.com/Google ScholarGoogle Scholar
  18. Donald B. Rubin Paul R. Rosenbaum. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 1 (1983), 41--55.Google ScholarGoogle ScholarCross RefCross Ref
  19. Judea Pearl. 2009. Causal inference in statistics: An overview. Statistics Surveys 3 (2009), 96--146.Google ScholarGoogle ScholarCross RefCross Ref
  20. Swati Rallapalli, Lili Qiu, Yin Zhang, and Yi-Chao Chen. 2010. Exploiting Temporal Stability and Low-rank Structure for Localization in Mobile Networks. In Proceedings of the Sixteenth Annual International Conference on Mobile Computing and Networking (MobiCom '10). ACM, New York, NY, USA, 161--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Donald B. Rubin. 1973. Matching to Remove Bias in Observational Studies. Biometrics 29, 1 (1973), 159--183.Google ScholarGoogle ScholarCross RefCross Ref
  22. Donald B. Rubin. 1974. Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology 66, 5 (1974), 688--701.Google ScholarGoogle ScholarCross RefCross Ref
  23. Stephan Shemilt. 2015. Cricket World Cup 2015: India and Pakistan fans usurp the limelight. (February 2015). https://www.bbc.com/sport/cricket/31479487Google ScholarGoogle Scholar
  24. Ishan Tharoor. 2016. These global sporting events totally dwarf the SuperBowl. (February 2016). https://www.washingtonpost.com/news/worldviews/wp/2016/02/05/these-global-sporting-events-totally-dwarf-the-super-bowl/?noredirect=on&utm_term=.6f393cc4e52eGoogle ScholarGoogle Scholar
  25. Christopher Xie, Alex Talk, and Emily Fox. 2016. A Unified Framework for Missing Data and Cold Start Prediction for Time Series Data. In Advances in neural information processing systems Time Series Workshop.Google ScholarGoogle Scholar
  26. Hsiang-Fu Yu, Nikhil Rao, and Inderjit S. Dhillon. 2015. Temporal Regularized Matrix Factorization. CoRR abs/1509.08333 (2015).Google ScholarGoogle Scholar

Index Terms

  1. mRSC: Multi-dimensional Robust Synthetic Control

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!