Match!

What happened to Popperian falsification? Publishing neutral and negative findings : moving away from biased publication practices

Published on Aug 1, 2016
· DOI :10.1108/CCSM-03-2016-0084
Witteloostuijn van Arjen35
Estimated H-index: 35
(University of Antwerp)
Abstract
Purpose – Current publication practices in the scholarly (International) Business and Management community are overwhelmingly anti-Popperian, which fundamentally frustrates the production of scientific progress. This is the result of at least five related biases: the verification, novelty, normal science, evidence, and market biases. As a result, no one is really interested in replicating anything. In this essay, the author extensively argues what he believes is wrong, why that is so, and what we might do about this. The paper aims to discuss these issues. Design/methodology/approach – This is an essay, combining a literature review with polemic argumentation. Findings – Only a tiny fraction of published studies involve a replication effort. Moreover, journal authors, editors, reviewers and readers are not interested in seeing nulls and negatives in print. This replication crisis implies that Popper’s critical falsification principle is actually thrown into the scientific community’s dustbin. Behind the facade of all these so-called new discoveries, false positives abound, as do questionable research practices meant to produce all this allegedly cutting-edge and groundbreaking significant findings. If this dismal state of affairs does not change for the good, (International) Business and Management research is ending up in a deadlock. Research limitations/implications – A radical cultural change in the scientific community, including (International) Business and Management, is badly needed. It should be in the community’s DNA to engage in the quest for the “truth” – nothing more, nothing less. Such a change must involve all stakeholders: scholars, editors, reviewers, and students, but also funding agencies, research institutes, university presidents, faculty deans, department chairs, journalists, policymakers, and publishers. In the words of Ioannidis (2012, p. 647): “Safeguarding scientific principles is not something to be done once and for all. It is a challenge that needs to be met successfully on a daily basis both by single scientists and the whole scientific establishment.” Practical implications – Publication practices have to change radically. For instance, editorial policies should dispose of their current overly dominant pro-novelty and pro-positives biases, and explicitly encourage the publication of replication studies, including failed and unsuccessful ones that report null and negative findings. Originality/value – This is an explicit plea to change the way the scientific research community operates, offering a series of concrete recommendations what to do before it is too late.
  • References (109)
  • Citations (8)
📖 Papers frequently viewed together
1 Citations
110 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
References109
Newest
#1Kai J. Jonas (UvA: University of Amsterdam)H-Index: 16
#2Joseph Cesario (MSU: Michigan State University)H-Index: 16
Comprehensive Results in Social Psychology (CRSP) is a novel journal for preregistered research (so-called registered reports, RR) in the field of social psychology. It offers RR-only publications, with the possibility of adding exploratory analysis and data as well. After submission of introduction, hypotheses, methods, procedure, and analysis plan, submitted manuscripts are reviewed prior to data collection. If the peer review process results in a positive evaluation of the manuscript, an init...
13 CitationsSource
#1Sandy Bogaert (University of Antwerp)H-Index: 7
#2C.A.J.J. Boone (University of Antwerp)H-Index: 26
Last. Witteloostuijn van Arjen (University of Antwerp)H-Index: 35
view all 4 authors...
In this study, the authors provide an assessment of the ecological theory of organizational form emergence and focus on the positive density effect associated with legitimation. The argument comes in two steps. First, organizational ecology seeks to understand cross-population similarities in search of general patterns in form emergence processes. Using summative meta-analysis, the authors show that the average effect of density dependence on population-level legitimation is positive, but this a...
19 CitationsSource
#1Daniël H. J. Wigboldus (Radboud University Nijmegen)H-Index: 30
#2Ron Dotsch (Radboud University Nijmegen)H-Index: 22
Not so long ago, in the early days of experimental psychology, researchers performed their data analyses by hand (or paid people to do these calculations for them). No wonder, they took ample time beforehand to think through which analyses to perform (and especially which not). Exploring data was not really an option. The t value or F value that resulted from the calculations provided a more or less definitive answer. H0 could be rejected or not. During the past decades, we have quickly moved fr...
18 CitationsSource
#1Dirk Lindebaum (University of Liverpool)H-Index: 13
In this critical essay, I contend that accelerating demands for novel theories in management studies imply that new methodologies and data are sometimes accepted prematurely as supply of these novel theories. This point is illustrated with reference to how neuroscience can inform management research. I propose two demand forces that foster the increased focus on neuroscience in management studies, these being (i) the direction of public research funding, and (ii) publication bias as a boost for ...
15 CitationsSource
#1Anne Tsui (Mendoza College of Business)H-Index: 1
Purpose – Value-free science is an ideal that is neither possible nor desirable, especially for social sciences. The subject of social sciences is individuals and groups; hence social, moral, ethical, or political values are inherent and unavoidable in all steps of the scientific process. Further, the authority of science requires the scientist to be responsible experts in ensuring the reliability of knowledge and in assessing the risks in applying the research findings in social policies and pr...
13 CitationsSource
Purpose – The purpose of this paper is to argue that the time is ripe to establish a powerful tradition in Experimental International Business (IB). Probably due to what the Arjen van Witteloostuijn refers to as the external validity myth, experimental laboratory designs are underutilized in IB, which implies that the internal validity miracle of randomized experimentation goes largely unnoticed in this domain of the broader management discipline. Design/methodology/approach – In the following p...
12 CitationsSource
#1Alexander A. AartsH-Index: 1
#2Joanna E. Anderson (DRDC: Defence Research and Development Canada)H-Index: 3
Last. Kellylynn Zuni (Adams State University)H-Index: 2
view all 270 authors...
Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of ...
2,235 CitationsSource
#1Neve Isaeva (University of Surrey)H-Index: 1
#2Reinhard Bachmann (SOAS, University of London)H-Index: 21
Last. Mark N. K. Saunders (University of Surrey)H-Index: 27
view all 4 authors...
In this thought piece we take stock of and evaluate the nature of knowledge production in the field of trust research by examining the epistemologies of 167 leading trust scholars, who responded to a short survey. Following a brief review of major epistemological perspectives, we discuss the nature of the prevalent views and their geographical distribution within our field. We call on trust researchers to engage in epistemological reflection, develop their own awareness of alternative epistemolo...
11 CitationsSource
Abstract Honig and Samuelsson ( 2014 ) recently published an article in this outlet criticizing “Does business planning facilitate the development of new ventures? “a paper I wrote with Scott Shane nearly 15 years ago. They claim that their effort adds to the discussion of (a) the merits of business planning, (b) data replication and extension, (c) sample selection bias, (d) evaluation of normative research and (e) publication standards. However, most of the claims they make are incorrect.
14 CitationsSource
Organizational research is guided by standards of what journals will publish and what gets rewarded in scholarly careers. This system can promote novelty rather than truth and impact rather than coherence. The advent of big data, combined with our current system of scholarly career incentives, is likely to yield a high volume of novel papers with sophisticated econometrics and no obvious prospect of cumulative knowledge development. Moreover, changes in the world of organizations are not being m...
57 CitationsSource
Cited By8
Newest
Source
Source
In this commentary, I argue why we should stop engaging in null hypothesis statistical significance testing altogether. Artificial and misleading it may be, but we know how to play the p value threshold and null hypothesis-testing game. We feel secure; we love the certainty. The fly in the ointment is that the conventions have led to questionable research practices. Wasserstein, Schirm, & Lazar (Am Stat 73(sup1):1–19, 2019. https://doi.org/10.1080/00031305.2019.1583913) explain why, in their tho...
Source
#1Richard M. Walker (CityU: City University of Hong Kong)H-Index: 43
#2Gene A. Brewer (UGA: University of Georgia)H-Index: 33
Last. Witteloostuijn van Arjen (VU: VU University Amsterdam)H-Index: 35
view all 5 authors...
3 CitationsSource
#1M. John Foster (KUL: Kingston University)H-Index: 1
This paper examines the proposition that adopting a socially responsible, or philanthropic, management posture is not antithetic to the capitalist business model but rather can be seen as a sound approach to the development of long-term sustainability in business in a modern business environment, wherein a strand of corporate social responsibility is one core aspect of the composite utility function of the modern business. We suggest further that for many of the prominent/significant examples of...
Source
#1Claudio Pousa (Lakehead University)H-Index: 5
#2Timothy Hardie (Lakehead University)H-Index: 5
Last. Xiaodi (Freddy) Zhang (Lakehead University)H-Index: 1
view all 3 authors...
Purpose The purpose of this paper is to test the influence of managerial coaching on frontline employee customer orientation, sales orientation and performance in a Chinese context. Further to this first goal, the authors also aim to compare these results with those obtained with a sample of Canadian bank employees in order to understand to what extent differences between Eastern and Western cultures affect business practices and employee responses in both environments. Design/methodology/approa...
3 CitationsSource
#1Michael MinkovH-Index: 20
#2Pinaki DuttH-Index: 2
Last. Ben MuddH-Index: 2
view all 8 authors...
There are few large-scale studies that compare how parents socialize children across the globe and the implications of the different types of socialization. To fill this gap, the authors used data from a new study across 52,300 probabilistically selected respondents from 54 countries. They were asked what advice they would give to their children to instill desirable values and traits in them. Aggregated to the national level, the responses yield two main dimensions of national culture. The first...
1 CitationsSource
#1Richard M. Walker (CityU: City University of Hong Kong)H-Index: 43
#2JamesOliver (University of Exeter)H-Index: 23
Last. Gene A. Brewer (UGA: University of Georgia)H-Index: 33
view all 3 authors...
Replication is increasingly recognized as an important part of knowledge production in the social sciences, especially for experimental research. However, despite growing use of experiments, replication is little discussed or practiced in public management. We review the approach to replication taken by research in leading public management journals and note its scarcity. We then use a typology developed by Tsang and Kwan to classify the experimental replications undertaken by the articles in th...
9 CitationsSource
#1Michael MinkovH-Index: 20
#2Pinaki DuttH-Index: 2
Last. Ben Mudd (University of Nottingham)H-Index: 2
view all 8 authors...
Purpose The purpose of this paper is to provide an updated and authoritative measure of individualism vs collectivism (IDV-COLL) as a dimension of national culture. Design/methodology/approach Rather than focus solely on Hofstede’s classic work, the authors review the main nation-level studies of IDV-COLL and related constructs to identify the salient cultural differences between rich societies and developing nations. The authors conceptualize and operationalize IDV-COLL on the basis of those di...
19 CitationsSource
#1Dessislava Dikova (WU: Vienna University of Economics and Business)H-Index: 1
#2Witteloostuijn van Arjen (Tilburg University)H-Index: 35
Last. Simon C. Parker (UWO: University of Western Ontario)H-Index: 37
view all 3 authors...
Purpose Extant work in international business (IB) involves a partial contingency-theoretic perspective: a holistic view of the impact of bundles of contingencies on an outcome variable is missing. The purpose of this paper is to adopt a contingency approach to study multinational enterprise (MNE) subsidiary performance in the appropriate context of European transition economies at the beginning of the current millennium. Design/methodology/approach Methodologically, the authors introduce abduct...
6 CitationsSource