With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The “effective sample” that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the “multiple regression weights” that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.

When a researcher estimates the parameters of a regression function using information on all 50 states in the United States, or information on all visits to a website, what is the interpretation of the standard errors? Researchers typically report standard errors that are designed to capture sampling variation, based on viewing the data as a random sample drawn from a large population of interest, even in applications where it is difficult to articulate what that population of interest is and ho...

Freedman [Adv. in Appl. Math. 40 (2008) 180-193; Ann. Appl. Stat. 2 (2008) 176-196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman's model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fi...

We derive a class of design-based estimators for the average treatment effect that are unbiased whenever the treatment assignment process is known. We generalize these estimators to include unbiased covariate adjustment using any model for outcomes that the analyst chooses. We then provide expressions and conservative estimators for the variance of the proposed estimators.

Fixed eects regression models are the primary workhorse for causal inference in applied panel data analysis. In this paper, we establish that in the case of a binary treatment variable with no pre-treatment covariate, xed eects estimators are algebraically equivalent to particular matching estimators. At the most basic level, the results suggest that xed eects

Oil-exporting states, or petrostates, engage in militarized interstate disputes (MIDS) at a much higher rate on average than non-petrostates. Why is this so? Further, what explains the variation among the petrostates in adopting aggressive foreign policies and engaging in MIDS on that basis? This paper investigates these questions by developing and testing a theory that proposes that when revolutionary governments come to power in petrostates, they have a higher propensity to launch MIDS than co...

Part I. Introduction: 1. The advent of experimental political science Part II. Experimental Reasoning about Causality: 2. Experiments and causal relations 3. The causal inference problem and the Rubin causal model 4. Controlling observables and unobservables 5. Randomization and pseudo-randomization 6. Formal theory and causality Part III. What Makes a Good Experiment?: 7. Validity and experimental manipulations 8. Location, artificiality, and related design issues 9. Choosing subjects 10. Subje...

This study estimates the effect of economic conditions on redistributive conflict. We examine land invasions in Brazil using a panel data set with over 50,000 municipality-year observations. Adverse economic shocks, instrumented by rainfall, cause the rural poor to invade and occupy large landholdings. This effect exhibits substantial heterogeneity by land inequality and land tenure systems, but not by other observable variables. In highly unequal municipalities, negative income shocks cause twi...

There is currently much debate about the effectiveness of foreign aid and about what kind of projects can engender economic development. There is skepticism about the ability of econometric analysis to resolve these issues or of development agencies to learn from their own experience. In response, there is increasing use in development economics of randomized controlled trials (RCTs) to accumulate credible knowledge of what works, without overreliance on questionable theory or statistical method...

Previous research shows that partisans rate the economy more favorably when their party holds power. There are several explanations for this association, including use of different evaluative criteria, selective perception, selective exposure to information, correlations between economic experiences and partisanship, and partisan bias in survey responses. We use a panel survey around the November 2006 election to measure changes in economic expectations and behavioral intentions after an unantic...

Foreign aid is thought to be useful, and therefore desirable, to recipient governments because it allows them to increase their support through the provision of goods or services. However, the effe...

This paper uses potential outcome time series to provide a nonparametric framework for quantifying dynamic causal effects in macroeconometrics. This provides sufficient conditions for the nonparametric identification of dynamic causal effects as well as clarify the causal content of several common assumptions and methods in macroeconomics. Our key identifying assumption is shown to be non-anticipating treatments which enables nonparametric inference on dynamic causal effects. Next, we provide a ...

(LSE: London School of Economics and Political Science), Kevin J. Mullinix5

Estimated H-index: 5

(KU: University of Kansas)

The extent to which survey experiments conducted with nonrepresentative convenience samples are generalizable to target populations depends critically on the degree of treatment effect heterogeneity. Recent inquiries have found a strong correspondence between sample average treatment effects estimated in nationally representative experiments and in replication studies conducted with convenience samples. We consider here two possible explanations: low levels of effect heterogeneity or high levels...

What is the impact of an earthquake on the educational attainment of high school students? In this paper, we address this question using a unique data set and new matching methods. In particular, we use an administrative census of the same students measured before and after the 2010 Chilean earthquake. We propose and analyze new matching methods that overcome three challenges of existing approaches. These new methods allow us: (i) to handle multi-valued treatments without estimating the generali...