Match!

What happened to Popperian falsification? Publishing neutral and negative findings : moving away from biased publication practices

Published on Aug 1, 2016
· DOI :10.1108/CCSM-03-2016-0084
Witteloostuijn van Arjen33
Estimated H-index: 33
(University of Antwerp)
Cite
Abstract
Purpose – Current publication practices in the scholarly (International) Business and Management community are overwhelmingly anti-Popperian, which fundamentally frustrates the production of scientific progress. This is the result of at least five related biases: the verification, novelty, normal science, evidence, and market biases. As a result, no one is really interested in replicating anything. In this essay, the author extensively argues what he believes is wrong, why that is so, and what we might do about this. The paper aims to discuss these issues. Design/methodology/approach – This is an essay, combining a literature review with polemic argumentation. Findings – Only a tiny fraction of published studies involve a replication effort. Moreover, journal authors, editors, reviewers and readers are not interested in seeing nulls and negatives in print. This replication crisis implies that Popper’s critical falsification principle is actually thrown into the scientific community’s dustbin. Behind the facade of all these so-called new discoveries, false positives abound, as do questionable research practices meant to produce all this allegedly cutting-edge and groundbreaking significant findings. If this dismal state of affairs does not change for the good, (International) Business and Management research is ending up in a deadlock. Research limitations/implications – A radical cultural change in the scientific community, including (International) Business and Management, is badly needed. It should be in the community’s DNA to engage in the quest for the “truth” – nothing more, nothing less. Such a change must involve all stakeholders: scholars, editors, reviewers, and students, but also funding agencies, research institutes, university presidents, faculty deans, department chairs, journalists, policymakers, and publishers. In the words of Ioannidis (2012, p. 647): “Safeguarding scientific principles is not something to be done once and for all. It is a challenge that needs to be met successfully on a daily basis both by single scientists and the whole scientific establishment.” Practical implications – Publication practices have to change radically. For instance, editorial policies should dispose of their current overly dominant pro-novelty and pro-positives biases, and explicitly encourage the publication of replication studies, including failed and unsuccessful ones that report null and negative findings. Originality/value – This is an explicit plea to change the way the scientific research community operates, offering a series of concrete recommendations what to do before it is too late.
  • References (105)
  • Citations (7)
Cite
References105
Newest
Published on Sep 1, 2016
Kai J. Jonas16
Estimated H-index: 16
(UvA: University of Amsterdam),
Joseph Cesario15
Estimated H-index: 15
(MSU: Michigan State University)
Comprehensive Results in Social Psychology (CRSP) is a novel journal for preregistered research (so-called registered reports, RR) in the field of social psychology. It offers RR-only publications, with the possibility of adding exploratory analysis and data as well. After submission of introduction, hypotheses, methods, procedure, and analysis plan, submitted manuscripts are reviewed prior to data collection. If the peer review process results in a positive evaluation of the manuscript, an init...
Published on Jul 1, 2016in Journal of Management9.06
Sandy Bogaert7
Estimated H-index: 7
(University of Antwerp),
Christophe Boone7
Estimated H-index: 7
(University of Antwerp)
+ 1 AuthorsWitteloostuijn van Arjen33
Estimated H-index: 33
(University of Antwerp)
In this study, the authors provide an assessment of the ecological theory of organizational form emergence and focus on the positive density effect associated with legitimation. The argument comes in two steps. First, organizational ecology seeks to understand cross-population similarities in search of general patterns in form emergence processes. Using summative meta-analysis, the authors show that the average effect of density dependence on population-level legitimation is positive, but this a...
Published on Mar 1, 2016in Psychometrika2.74
D.H.J. Wigboldus29
Estimated H-index: 29
(Radboud University Nijmegen),
Ron Dotsch22
Estimated H-index: 22
(Radboud University Nijmegen)
Not so long ago, in the early days of experimental psychology, researchers performed their data analyses by hand (or paid people to do these calculations for them). No wonder, they took ample time beforehand to think through which analyses to perform (and especially which not). Exploring data was not really an option. The t value or F value that resulted from the calculations provided a more or less definitive answer. H0 could be rejected or not. During the past decades, we have quickly moved fr...
Published on Mar 1, 2016in Human Relations3.37
Dirk Lindebaum12
Estimated H-index: 12
(University of Liverpool)
In this critical essay, I contend that accelerating demands for novel theories in management studies imply that new methodologies and data are sometimes accepted prematurely as supply of these novel theories. This point is illustrated with reference to how neuroscience can inform management research. I propose two demand forces that foster the increased focus on neuroscience in management studies, these being (i) the direction of public research funding, and (ii) publication bias as a boost for ...
Published on Feb 1, 2016
Anne Tsui1
Estimated H-index: 1
(Mendoza College of Business)
Purpose – Value-free science is an ideal that is neither possible nor desirable, especially for social sciences. The subject of social sciences is individuals and groups; hence social, moral, ethical, or political values are inherent and unavoidable in all steps of the scientific process. Further, the authority of science requires the scientist to be responsible experts in ensuring the reliability of knowledge and in assessing the risks in applying the research findings in social policies and pr...
Witteloostuijn van Arjen33
Estimated H-index: 33
(Tilburg University)
Purpose – The purpose of this paper is to argue that the time is ripe to establish a powerful tradition in Experimental International Business (IB). Probably due to what the Arjen van Witteloostuijn refers to as the external validity myth, experimental laboratory designs are underutilized in IB, which implies that the internal validity miracle of randomized experimentation goes largely unnoticed in this domain of the broader management discipline. Design/methodology/approach – In the following p...
Published on Jul 3, 2015
Neve Isaeva1
Estimated H-index: 1
(University of Surrey),
Reinhard Bachmann21
Estimated H-index: 21
(SOAS, University of London)
+ 1 AuthorsMark N. K. Saunders25
Estimated H-index: 25
(University of Surrey)
In this thought piece we take stock of and evaluate the nature of knowledge production in the field of trust research by examining the epistemologies of 167 leading trust scholars, who responded to a short survey. Following a brief review of major epistemological perspectives, we discuss the nature of the prevalent views and their geographical distribution within our field. We call on trust researchers to engage in epistemological reflection, develop their own awareness of alternative epistemolo...
Published on Jul 1, 2015in Journal of Business Venturing Insights
Frédéric Delmar22
Estimated H-index: 22
Abstract Honig and Samuelsson ( 2014 ) recently published an article in this outlet criticizing “Does business planning facilitate the development of new ventures? “a paper I wrote with Scott Shane nearly 15 years ago. They claim that their effort adds to the discussion of (a) the merits of business planning, (b) data replication and extension, (c) sample selection bias, (d) evaluation of normative research and (e) publication standards. However, most of the claims they make are incorrect.
Published on Jun 1, 2015in Review of General Psychology2.79
Michèle B. Nuijten8
Estimated H-index: 8
(Tilburg University),
M.A.L.M. van Assen22
Estimated H-index: 22
(Tilburg University)
+ 1 AuthorsJelte M. Wicherts37
Estimated H-index: 37
(Tilburg University)
Replication is often viewed as the demarcation between science and nonscience. However, contrary to the commonly held view, we show that in the current (selective) publication system replications may increase bias in effect size estimates. Specifically, we examine the effect of replication on bias in estimated population effect size as a function of publication bias and the studies' sample size or power. We analytically show that incorporating the results of published replication studies will in...
Published on Jun 1, 2015in Administrative Science Quarterly8.02
Gerald F. Davis35
Estimated H-index: 35
(UM: University of Michigan)
Organizational research is guided by standards of what journals will publish and what gets rewarded in scholarly careers. This system can promote novelty rather than truth and impact rather than coherence. The advent of big data, combined with our current system of scholarly career incentives, is likely to yield a high volume of novel papers with sophisticated econometrics and no obvious prospect of cumulative knowledge development. Moreover, changes in the world of organizations are not being m...
Cited By7
Newest
Published on Jun 4, 2018in International Journal of Bank Marketing
Claudio Pousa5
Estimated H-index: 5
(Lakehead University),
Timothy Hardie5
Estimated H-index: 5
(Lakehead University),
Xiaodi Zhang1
Estimated H-index: 1
(Lakehead University)
Purpose The purpose of this paper is to test the influence of managerial coaching on frontline employee customer orientation, sales orientation and performance in a Chinese context. Further to this first goal, the authors also aim to compare these results with those obtained with a sample of Canadian bank employees in order to understand to what extent differences between Eastern and Western cultures affect business practices and employee responses in both environments. Design/methodology/approa...
Published on Jun 1, 2018in Philosophy of Management
M. John Foster2
Estimated H-index: 2
(KUL: Kingston University)
This paper examines the proposition that adopting a socially responsible, or philanthropic, management posture is not antithetic to the capitalist business model but rather can be seen as a sound approach to the development of long-term sustainability in business in a modern business environment, wherein a strand of corporate social responsibility is one core aspect of the composite utility function of the modern business. We suggest further that for many of the prominent/significant examples of...
Published on Apr 4, 2018in Comparative Sociology
Michael Minkov1
Estimated H-index: 1
(Yahoo!),
Pinaki Dutt2
Estimated H-index: 2
+ 5 AuthorsBen Mudd2
Estimated H-index: 2
There are few large-scale studies that compare how parents socialize children across the globe and the implications of the different types of socialization. To fill this gap, the authors used data from a new study across 52,300 probabilistically selected respondents from 54 countries. They were asked what advice they would give to their children to instill desirable values and traits in them. Aggregated to the national level, the responses yield two main dimensions of national culture. The first...
Published on Oct 21, 2017in Public Management Review3.16
Richard M. Walker42
Estimated H-index: 42
(CityU: City University of Hong Kong),
JamesOliver22
Estimated H-index: 22
(University of Exeter),
Gene A. Brewer33
Estimated H-index: 33
(UGA: University of Georgia)
Replication is increasingly recognized as an important part of knowledge production in the social sciences, especially for experimental research. However, despite growing use of experiments, replication is little discussed or practiced in public management. We review the approach to replication taken by research in leading public management journals and note its scarcity. We then use a typology developed by Tsang and Kwan to classify the experimental replications undertaken by the articles in th...
Published on Aug 7, 2017
Michael Minkov19
Estimated H-index: 19
,
Pinaki Dutt2
Estimated H-index: 2
+ 5 AuthorsBen Mudd2
Estimated H-index: 2
Purpose The purpose of this paper is to provide an updated and authoritative measure of individualism vs collectivism (IDV-COLL) as a dimension of national culture. Design/methodology/approach Rather than focus solely on Hofstede’s classic work, the authors review the main nation-level studies of IDV-COLL and related constructs to identify the salient cultural differences between rich societies and developing nations. The authors conceptualize and operationalize IDV-COLL on the basis of those di...
Published on Aug 7, 2017
Dessislava Dikova1
Estimated H-index: 1
(WU: Vienna University of Economics and Business),
Witteloostuijn van Arjen33
Estimated H-index: 33
,
Simon C. Parker36
Estimated H-index: 36
(UWO: University of Western Ontario)
Purpose Extant work in international business (IB) involves a partial contingency-theoretic perspective: a holistic view of the impact of bundles of contingencies on an outcome variable is missing. The purpose of this paper is to adopt a contingency approach to study multinational enterprise (MNE) subsidiary performance in the appropriate context of European transition economies at the beginning of the current millennium. Design/methodology/approach Methodologically, the authors introduce abduct...
Published on Oct 3, 2016
Anne-Wil Harzing45
Estimated H-index: 45
Purpose The purpose of this paper is to discuss the pervasive problem of a lack of replication studies in international business based on van Witteloostuijn’s (2016) commentary “What happened to Popperian Falsification?” Design/methodology/approach The author presents two short case studies from her own research, one in which no replication studies took place, and one in which a replication study was conducted shortly after the original study was published. Findings The author shows how the lack...
View next paperScience or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science