Match!

Does Regression Produce Representative Estimates of Causal Effects

Published on Jan 1, 2016in American Journal of Political Science4.35
· DOI :10.1111/ajps.12185
M AronowPeter15
Estimated H-index: 15
(Yale University),
Cyrus Samii16
Estimated H-index: 16
(NYU: New York University)
Cite
Abstract
With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The “effective sample” that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the “multiple regression weights” that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.
  • References (52)
  • Citations (28)
Cite
References52
Newest
Published on Jul 24, 2014in National Bureau of Economic Research
Alberto Abadie26
Estimated H-index: 26
(NBER: National Bureau of Economic Research),
Susan Athey37
Estimated H-index: 37
(Stanford University)
+ 1 AuthorsJeffrey M. Wooldridge43
Estimated H-index: 43
(MSU: Michigan State University)
When a researcher estimates the parameters of a regression function using information on all 50 states in the United States, or information on all visits to a website, what is the interpretation of the standard errors? Researchers typically report standard errors that are designed to capture sampling variation, based on viewing the data as a random sample drawn from a large population of interest, even in applications where it is difficult to articulate what that population of interest is and ho...
Published on Mar 1, 2013in The Annals of Applied Statistics1.66
Winston Lin1
Estimated H-index: 1
Freedman [Adv. in Appl. Math. 40 (2008) 180-193; Ann. Appl. Stat. 2 (2008) 176-196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman's model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fi...
Published on Jan 29, 2013
M AronowPeter15
Estimated H-index: 15
(Yale University),
Joel A. Middleton6
Estimated H-index: 6
We derive a class of design-based estimators for the average treatment effect that are unbiased whenever the treatment assignment process is known. We generalize these estimators to include unbiased covariate adjustment using any model for outcomes that the analyst chooses. We then provide expressions and conservative estimators for the variance of the proposed estimators.
Published on Jan 1, 2011
Kosuke Imai1
Estimated H-index: 1
(Princeton University),
In Song Kim1
Estimated H-index: 1
(Princeton University)
Fixed eects regression models are the primary workhorse for causal inference in applied panel data analysis. In this paper, we establish that in the case of a binary treatment variable with no pre-treatment covariate, xed eects estimators are algebraically equivalent to particular matching estimators. At the most basic level, the results suggest that xed eects
Published on Oct 1, 2010in International Organization4.51
Jeff D. Colgan13
Estimated H-index: 13
(AU: American University)
Oil-exporting states, or petrostates, engage in militarized interstate disputes (MIDS) at a much higher rate on average than non-petrostates. Why is this so? Further, what explains the variation among the petrostates in adopting aggressive foreign policies and engaging in MIDS on that basis? This paper investigates these questions by developing and testing a theory that proposes that when revolutionary governments come to power in petrostates, they have a higher propensity to launch MIDS than co...
Published on Aug 6, 2010
B MortonRebecca23
Estimated H-index: 23
(NYU: New York University),
C WilliamsKenneth7
Estimated H-index: 7
(MSU: Michigan State University)
Part I. Introduction: 1. The advent of experimental political science Part II. Experimental Reasoning about Causality: 2. Experiments and causal relations 3. The causal inference problem and the Rubin causal model 4. Controlling observables and unobservables 5. Randomization and pseudo-randomization 6. Formal theory and causality Part III. What Makes a Good Experiment?: 7. Validity and experimental manipulations 8. Location, artificiality, and related design issues 9. Choosing subjects 10. Subje...
Published on Aug 1, 2010in The Review of Economics and Statistics3.64
F. Daniel Hidalgo8
Estimated H-index: 8
(University of California, Berkeley),
Suresh Naidu14
Estimated H-index: 14
(Columbia University)
+ 1 AuthorsNeal Richardson3
Estimated H-index: 3
(University of California, Berkeley)
This study estimates the effect of economic conditions on redistributive conflict. We examine land invasions in Brazil using a panel data set with over 50,000 municipality-year observations. Adverse economic shocks, instrumented by rainfall, cause the rural poor to invade and occupy large landholdings. This effect exhibits substantial heterogeneity by land inequality and land tenure systems, but not by other observable variables. In highly unequal municipalities, negative income shocks cause twi...
Published on Jun 1, 2010in Journal of Economic Literature5.41
A-S Deaton87
Estimated H-index: 87
(Princeton University)
There is currently much debate about the effectiveness of foreign aid and about what kind of projects can engender economic development. There is skepticism about the ability of econometric analysis to resolve these issues or of development agencies to learn from their own experience. In response, there is increasing use in development economics of randomized controlled trials (RCTs) to accumulate credible knowledge of what works, without overreliance on questionable theory or statistical method...
Published on Jan 1, 2010in American Journal of Political Science4.35
S GerberAlan42
Estimated H-index: 42
(Yale University),
A HuberGregory27
Estimated H-index: 27
(Yale University)
Previous research shows that partisans rate the economy more favorably when their party holds power. There are several explanations for this association, including use of different evaluative criteria, selective perception, selective exposure to information, correlations between economic experiences and partisanship, and partisan bias in survey responses. We use a panel survey around the November 2006 election to measure changes in economic expectations and behavioral intentions after an unantic...
Cited By28
Newest
Published on Sep 18, 2018in Political Research Quarterly1.58
Ryan C. Briggs (U of G: University of Guelph)
Foreign aid is thought to be useful, and therefore desirable, to recipient governments because it allows them to increase their support through the provision of goods or services. However, the effe...
Published on May 31, 2019in American Political Science Review3.90
Graeme Blair5
Estimated H-index: 5
,
Jasper Cooper + 1 AuthorsMacartan Humphreys23
Estimated H-index: 23
Published on May 27, 2019in American Journal of Physical Anthropology2.66
Richard J. Smith36
Estimated H-index: 36
(WashU: Washington University in St. Louis)
Published on Apr 1, 2019in American Journal of Political Science4.35
Kosuke Imai36
Estimated H-index: 36
(Harvard University),
In Song Kim3
Estimated H-index: 3
(MIT: Massachusetts Institute of Technology),
In Song Kim1
Estimated H-index: 1
(MIT: Massachusetts Institute of Technology)
Published on Mar 1, 2019in arXiv: Econometrics
Ashesh Rambachan1
Estimated H-index: 1
(Harvard University),
Neil Shephard57
Estimated H-index: 57
(Harvard University)
This paper uses potential outcome time series to provide a nonparametric framework for quantifying dynamic causal effects in macroeconometrics. This provides sufficient conditions for the nonparametric identification of dynamic causal effects as well as clarify the causal content of several common assumptions and methods in macroeconomics. Our key identifying assumption is shown to be non-anticipating treatments which enables nonparametric inference on dynamic causal effects. Next, we provide a ...
Published on Feb 12, 2019in Legislative Studies Quarterly1.66
Robert J. McGrath6
Estimated H-index: 6
(GMU: George Mason University),
Josh M. Ryan3
Estimated H-index: 3
(USU: Utah State University)
Alexander Coppock7
Estimated H-index: 7
(Yale University),
Thomas J. Leeper8
Estimated H-index: 8
(LSE: London School of Economics and Political Science),
Kevin J. Mullinix5
Estimated H-index: 5
(KU: University of Kansas)
The extent to which survey experiments conducted with nonrepresentative convenience samples are generalizable to target populations depends critically on the degree of treatment effect heterogeneity. Recent inquiries have found a strong correspondence between sample average treatment effects estimated in nationally representative experiments and in replication studies conducted with convenience samples. We consider here two possible explanations: low levels of effect heterogeneity or high levels...
Published on Nov 1, 2018in American Political Science Review3.90
Kenneth Lowande2
Estimated H-index: 2
Published on Oct 16, 2018in arXiv: Applications
Magdalena Bennett1
Estimated H-index: 1
,
Juan Pablo Vielma16
Estimated H-index: 16
,
Jose R. Zubizarreta1
Estimated H-index: 1
What is the impact of an earthquake on the educational attainment of high school students? In this paper, we address this question using a unique data set and new matching methods. In particular, we use an administrative census of the same students measured before and after the 2010 Chilean earthquake. We propose and analyze new matching methods that overcome three challenges of existing approaches. These new methods allow us: (i) to handle multi-valued treatments without estimating the generali...
View next paperEconometric Analysis of Cross Section and Panel Data