Match!

External Validity in Parallel Global Field and Survey Experiments on Anonymous Incorporation

Published on Jul 1, 2017in The Journal of Politics2.49
· DOI :10.1086/690615
G FindleyMichael18
Estimated H-index: 18
,
Brock Laney1
Estimated H-index: 1
+ 1 AuthorsJ. C. Sharman18
Estimated H-index: 18
Cite
Abstract
By comparing parallel field and survey experiments testing compliance with international standards on corporate transparency, we highlight potential problems in the external validity of survey experimental designs. We performed a field experiment using deception in which we requested an anonymous business incorporation from nearly 4,000 corporate service providers in more than 180 countries. Subsequently, we conducted a survey experiment with the same providers using similar treatment conditions, but with informed consent to participate in a research study. Comparing responses and response rates corroborates—from a new angle and with additional implications—survey researchers’ caveats about selection bias and social desirability. Our conclusions on the relative external validity and different substantive results produced by different experimental designs constitutes an important cautionary note given the increased popularity of survey experiments within international relations and political science more g...
  • References (11)
  • Citations (2)
Cite
References11
Newest
Published on Jan 1, 2016in American Journal of Political Science4.35
M AronowPeter15
Estimated H-index: 15
(Yale University),
Cyrus Samii16
Estimated H-index: 16
(NYU: New York University)
With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The “effective sample” that regression uses to generate the estimate may bear little resemblance to t...
Jens Hainmueller30
Estimated H-index: 30
(Stanford University),
Dominik Hangartner15
Estimated H-index: 15
(UZH: University of Zurich),
Teppei Yamamoto13
Estimated H-index: 13
(MIT: Massachusetts Institute of Technology)
Survey experiments, like vignette and conjoint analyses, are widely used in the social sciences to elicit stated preferences and study how humans make multidimensional choices. However, there is a paucity of research on the external validity of these methods that examines whether the determinants that explain hypothetical choices made by survey respondents match the determinants that explain what subjects actually do when making similar choices in real-world situations. This study compares resul...
Published on Jan 1, 2014
Marilynn B. Brewer65
Estimated H-index: 65
(OSU: Ohio State University),
William D. Crano32
Estimated H-index: 32
Published on May 29, 2012
S GerberAlan42
Estimated H-index: 42
,
P GreenDonald62
Estimated H-index: 62
Written by two leading experts on experimental methods, this concise text covers the major aspects of experiment design, analysis, and interpretation in clear language. Students learn how to design randomized experiments, analyze the data, and interpret the findings. Beyond the authoritative coverage of the basic methodology, the authors include numerous features to help students achieve a deeper understanding of field experimentation, including rich examples from the social science literature, ...
Published on Jul 5, 2011
Diana C. Mutz30
Estimated H-index: 30
List of Tables ix Preface xi Chapter One: Population-Based Survey Experiments A Hybrid Methodology for the Social Sciences 1 PART I: TREATMENTS FOR POPULATION-BASED EXPERIMENTAL DESIGNS 23 Chapter Two: Treatments to Improve Measurement 25 Chapter Three: Direct and Indirect Treatments 37 Chapter Four: Vignette Treatments 54 Chapter Five: Treatments in the Context of Games 68 PART II: EXECUTION AND ANALYSIS 81 Chapter Six: Execution of Population-Based Survey Experiments 83 Chapter Seven: Analysis...
Published on Jun 1, 2011in Journal of the American Statistical Association3.41
Kosuke Imai36
Estimated H-index: 36
(Princeton University)
The item count technique is a survey methodology that is designed to elicit respondents’ truthful answers to sensitive questions such as racial prejudice and drug use. The method is also known as the list experiment or the unmatched count technique and is an alternative to the commonly used randomized response method. In this article, I propose new nonlinear least squares and maximum likelihood estimators for efficient multivariate regression analysis with the item count technique. The two-step ...
Published on Jun 1, 2008in Journal of Policy Analysis and Management3.83
Thomas D. Cook53
Estimated H-index: 53
(NU: Northwestern University),
William R. Shadish55
Estimated H-index: 55
(UCM: University of California, Merced),
Vivian C. Wong8
Estimated H-index: 8
(NU: Northwestern University)
This paper analyzes 12 recent within-study comparisons contrasting causal estimates from a randomized experiment with those from an observational study sharing the same treatment group. The aim is to test whether different causal estimates result when a counterfactual group is formed, either with or without random assignment, and when statistical adjustments for selection are made in the group from which random assignment is absent. We identify three studies comparing experiments and regression-...
Published on Feb 1, 2008in American Political Science Review3.90
S GerberAlan42
Estimated H-index: 42
(Yale University),
P GreenDonald62
Estimated H-index: 62
(Yale University),
W LarimerChristopher11
Estimated H-index: 11
(UNI: University of Northern Iowa)
Voter turnout theories based on rational self-interested behavior generally fail to predict significant turnout unless they account for the utility that citizens receive from performing their civic duty. We distinguish between two aspects of this type of utility, intrinsic satisfaction from behaving in accordance with a norm and extrinsic incentives to comply, and test the effects of priming intrinsic motives and applying varying degrees of extrinsic pressure. A large-scale field experiment invo...
Published on Jan 1, 2005
Robert A. Pape14
Estimated H-index: 14
Published on Apr 1, 1997in American Journal of Political Science4.35
James H. Kuklinski32
Estimated H-index: 32
,
Paul M. Sniderman37
Estimated H-index: 37
+ 4 AuthorsBarbara A. Mellers39
Estimated H-index: 39
Theory: We examine the relationship between blatant racial prejudice and anger toward affirmative action. Hypotheses: (1) Blatantly prejudiced attitudes continue to pervade the white population in the United States. (2) Resistance to affirmative action is more than an extension of this prejudice. (3) White resistance to affirmative action is not unyielding and unalterably fixed. Methods: Analysis of experiments embedded in a national survey of racial attitudes. Some of these experiments are desi...
Cited By2
Newest
Published on Jan 3, 2019in Review of International Organizations1.76
Daniel L. Nielson16
Estimated H-index: 16
(BYU: Brigham Young University),
D HydeSusan13
Estimated H-index: 13
(University of California, Berkeley),
Judith G. Kelley16
Estimated H-index: 16
(Durham University)
When do members of civil society view international election observers as legitimate? Motivated by recent work on the legitimacy of international organizations, we evaluate what type of information affects non-governmental organizations’ (NGOs) beliefs about international election observer groups, which include both intergovernmental organizations (IGOs) and international non-governmental organizations (INGOs) that seek to exercise authority, often regarding the same elections. Specifically, we ...
Published on Jan 1, 2019
Knut Petzold , Tobias Wolbring7
Estimated H-index: 7
,
Tobias Wolbring
Vignettenexperimente werden zunehmend nicht nur zur Messung von Einstellungen, sondern auch zur Erfassung von Verhaltensintentionen eingesetzt. Bei entsprechenden Studien wird meist implizit angenommen, dass die mittels Vignettenexperimenten geschatzten Effekte auf tatsachliches Verhalten und dessen Determinanten schliesen lassen. Die Annahme der Verhaltensvaliditat faktorieller Surveys wurde bisher jedoch nur selten explizit thematisiert und methodisch sauber getestet. In diesem Beitrag werden ...
Published on Jan 1, 2017in American Journal of Political Science4.35
David E. Broockman13
Estimated H-index: 13
(Stanford University),
Daniel M. Butler14
Estimated H-index: 14
(WashU: Washington University in St. Louis)
Influential theories depict politicians as, alternatively, strongly constrained by public opinion, able to shape public opinion with persuasive appeals, or relatively unconstrained by public opinion and able to shape it merely by announcing their positions. To test these theories, we conducted unique field experiments in cooperation with sitting politicians in which U.S. state legislators sent constituents official communications with randomly assigned content. The legislators sometimes stated t...
Published on Jul 1, 2016in American Journal of Political Science4.35
Joshua L. Kalla4
Estimated H-index: 4
(University of California, Berkeley),
David E. Broockman13
Estimated H-index: 13
(University of California, Berkeley)
Concern that donations to political campaigns secure preferential treatment from policy makers has long occupied judges, scholars, and the public. However, the effects of contributions on policy makers’ behavior are notoriously difficult to assess. We present the first randomized field experiment on the topic. The experiment focuses on whether contributions facilitate access to influential policy makers. In the experiment, a political organization attempted to schedule meetings between 191 congr...