Match!

Secular trends and evaluation of complex interventions: the rising tide phenomenon

Published on May 1, 2016in BMJ Quality & Safety7.04
· DOI :10.1136/bmjqs-2015-004372
Yen-Fu Chen22
Estimated H-index: 22
(Warw.: University of Warwick),
Karla Hemming5
Estimated H-index: 5
(University of Birmingham)
+ 1 AuthorsRichard Lilford66
Estimated H-index: 66
(Warw.: University of Warwick)
Cite
Abstract
Evaluations of service delivery interventions with contemporaneous controls often yield null results, even when the intervention appeared promising in advance. There can be many reasons for null results. In this paper we introduce the concept of a ‘rising tide’ phenomenon being a possible explanation of null results. We note that evaluations of service delivery interventions often occur when awareness of the problems they intend to address is already heightened, and pressure to tackle them is mounting throughout a health system. An evaluation may therefore take place in a setting where the system as a whole is improving – where there is a pronounced temporal trend or a ‘rising tide causing all vessels to rise’. As a consequence, control sites in an intervention study will improve. This reduces the difference between intervention and control sites and predisposes the study to a null result, leading to the conclusion that the intervention has no effect. We discuss how a rising tide may be distinguished from other causes of improvement in both control and intervention groups, and give examples where the rising tide provides a convincing explanation of such a finding. We offer recommendations for interpretation of research findings where improvements in the intervention group are matched by improvements in the control group. Understanding the rising tide phenomenon is important for a more nuanced interpretation of null results arising in the context of system-wide improvement. Recognition that a rising tide may have predisposed to a null result in one health system cautions against generalising the result to another health system where strong secular trends are absent.
  • References (39)
  • Citations (26)
Cite
References39
Newest
Published on Feb 6, 2015in BMJ27.60
Karla Hemming25
Estimated H-index: 25
(University of Birmingham),
Terry Haines32
Estimated H-index: 32
(Monash University)
+ 2 AuthorsRichard Lilford66
Estimated H-index: 66
(Warw.: University of Warwick)
The stepped wedge cluster randomised trial is a novel research study design that is increasingly being used in the evaluation of service delivery type interventions. The design involves random and sequential crossover of clusters from control to intervention until all clusters are exposed. It is a pragmatic study design which can reconcile the need for robust evaluations with political or logistical constraints. While not exclusively for the evaluation of service delivery interventions, it is pa...
Published on Feb 3, 2015in JAMA51.27
Nicholas H. Osborne18
Estimated H-index: 18
(UM: University of Michigan),
Lauren Hersch Nicholas17
Estimated H-index: 17
(Johns Hopkins University)
+ 2 AuthorsJustin B. Dimick62
Estimated H-index: 62
(UM: University of Michigan)
Importance The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) provides feedback to hospitals on risk-adjusted outcomes. It is not known if participation in the program improves outcomes and reduces costs relative to nonparticipating hospitals. Objective To evaluate the association of enrollment and participation in the ACS NSQIP with outcomes and Medicare payments compared with control hospitals that did not participate in the program. Design, Setting, and...
Published on Feb 3, 2015in JAMA51.27
David A. Etzioni26
Estimated H-index: 26
(Mayo Clinic),
Nabil Wasif25
Estimated H-index: 25
(Mayo Clinic)
+ 5 AuthorsElizabeth B. Habermann29
Estimated H-index: 29
(Mayo Clinic)
(adjusted odds ratio, 1.00; 95% CI, 0.97-1.03), serious complications (adjusted odds ratio, 0.98; 95% CI, 0.94-1.03), or mortality (adjusted odds ratio, 1.04; 95% CI, 0.94-1.14). CONCLUSIONS AND RELEVANCE No association was found between hospital-based participation in the NSQIP and improvements in postoperative outcomes over time within a large cohort of patients undergoing elective general/vascular operations at academic hospitals in the United States. These findings suggest that a surgical ou...
Published on Feb 1, 2013in BMJ Quality & Safety7.04
Julian Bion36
Estimated H-index: 36
(University of Birmingham)
Background: Bloodstream infections from central venous catheters (CVC-BSIs) increase morbidity and costs in intensive care units (ICUs). Substantial reductions in CVC-BSI rates have been reported using a combination of technical and non-technical interventions. Methods: We conducted a 2-year, four-cluster, stepped non-randomised study of technical and non-technical (behavioural) interventions to prevent CVC-BSIs in adult and paediatric ICUs in England. Random-effects Poisson regression modelling...
Published on Dec 1, 2011in BMC Medical Research Methodology2.51
Karla Hemming25
Estimated H-index: 25
(University of Birmingham),
Alan Girling27
Estimated H-index: 27
(University of Birmingham)
+ 2 AuthorsRichard Lilford66
Estimated H-index: 66
(University of Birmingham)
Background Cluster randomised controlled trials (CRCTs) are frequently used in health service evaluation. Assuming an average cluster size, required sample sizes are readily computed for both binary and continuous outcomes, by estimating a design effect or inflation factor. However, where the number of clusters are fixed in advance, but where it is possible to increase the number of individuals within each cluster, as is frequently the case in health service evaluation, sample size formulae have...
Published on Sep 1, 2011in Journal of Clinical Epidemiology4.65
Noreen Dadirai Mdege9
Estimated H-index: 9
(Ebor: University of York),
Mei-See Man15
Estimated H-index: 15
(Ebor: University of York)
+ 1 AuthorsDavid Tappin74
Estimated H-index: 74
(Ebor: University of York)
Abstract Objective To describe the application of the stepped wedge cluster randomized controlled trial (CRCT) design. Study Design and Setting Systematic review. We searched Medline, Embase, PsycINFO, HMIC, CINAHL, Cochrane Library, Web of Knowledge, and Current Controlled Trials Register for articles published up to January 2010. Stepped wedge CRCTs from all fields of research were included. Two authors independently reviewed and extracted data from the studies. Results Twenty-five studies wer...
Published on Feb 3, 2011in BMJ27.60
Amirta Benning2
Estimated H-index: 2
,
Mary Dixon-Woods51
Estimated H-index: 51
+ 12 AuthorsMartin Carmalt3
Estimated H-index: 3
Objective To independently evaluate the impact of the second phase of the Health Foundation’s Safer Patients Initiative (SPI2) on a range of patient safety measures. Design A controlled before and after design. Five substudies: survey of staff attitudes; review of case notes from high risk (respiratory) patients in medical wards; review of case notes from surgical patients; indirect evaluation of hand hygiene by measuring hospital use of handwashing materials; measurement of outcomes (adverse ev...
Published on Feb 3, 2011in BMJ27.60
Amirta Benning2
Estimated H-index: 2
(University of Birmingham),
Maisoon Ghaleb11
Estimated H-index: 11
(University of Hertfordshire)
+ 12 AuthorsMartin Carmalt3
Estimated H-index: 3
(Royal Orthopaedic Hospital)
Objectives To conduct an independent evaluation of the first phase of the Health Foundation’s Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design Mixed method evaluation involving five substudies, before and after design. Setting NHS hospitals in the United Kingdom. Participants Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1);...
Published on Jan 1, 2011
Peter Craig14
Estimated H-index: 14
,
Curtis Cooper163
Estimated H-index: 163
+ 8 AuthorsMatt Sutton35
Estimated H-index: 35
The aim of this guidance is to help producers, users, funders and publishers of evidence understand how and when 'natural experiments' can be used to good effect. While there is growing interest in using natural experiments to evaluate large-scale interventions to improve population health, the relevant methodological literature is dispersed across disciplines and there is a lack of general guidance either on the range of approaches available, or on the circumstances in which they are likely to ...
Cited By26
Newest
Published on May 3, 2019in BMC Pregnancy and Childbirth2.41
Deborah Weiss2
Estimated H-index: 2
,
Deborah Weiss (U of O: University of Ottawa)+ 6 AuthorsMonica Taljaard33
Estimated H-index: 33
In 2002, the MOREOB (Managing Obstetrical Risk Efficiently) obstetrical patient safety program was phased-in across hospitals in Ontario, Canada. The purpose of our study was to evaluate the effect of the MOREOB program on rates of adverse maternal and neonatal outcomes. A retrospective cohort study, using province-wide administrative hospitalization data. We included maternal and neonatal records between fiscal years 2002–2003 and 2013–2014, for deliveries taking place at the 67 Ontario hospita...
Published on Apr 2, 2019in BMC Health Services Research1.93
Michael P. Thompson4
Estimated H-index: 4
(UM: University of Michigan),
Ilana Graetz (Emory University)+ 2 AuthorsTeresa M. Waters1
Estimated H-index: 1
(UK: University of Kentucky)
Background Concerted quality improvement (QI) efforts have been taken to discourage the practice of early elective deliveries (EEDs), but few studies have robustly examined the impact of directed QI interventions in reducing EED practices. Using quasi-experimental methods, we sought to evaluate the impact of a statewide QI intervention to reduce the practice of EEDs.
Abstract Background Non-technical skills (NTS) failures have been implicated in a large proportion of surgical errors. The objective of this meta-analysis was to investigate whether NTS training of theatre staff improves patient outcomes. Methods In a systematic literature search all interventional studies evaluating the effects of NTS training of theatre staff were identified. Primary outcomes included mortality, morbidity, readmission rate and length of hospital stay. Secondary outcomes includ...
Nicholas Mays39
Estimated H-index: 39
,
Eilís Keeble1
Estimated H-index: 1
+ 2 AuthorsMary Alison Durand13
Estimated H-index: 13
Objective To examine whether any differential change in emergency admissions could be attributed to integrated care by comparing Pioneer and non-Pioneer populations from a pre-Pioneer baseline period (April 2010 to March 2013) over two follow-up periods: to 2014/15 and to 2015/16. Design Difference-in-differences analysis of emergency hospital admissions from English Hospital Episode Statistics (HES). Setting Local authorities in England classified as either Pioneer or non-Pioneer. Participants ...
Published on Aug 1, 2019in Health Services and Delivery Research
Paul R Mouncey7
Estimated H-index: 7
,
Dorothy Wade6
Estimated H-index: 6
(UCLH: University College London Hospitals NHS Foundation Trust)
+ 15 AuthorsDavid Howell9
Estimated H-index: 9
(UCLH: University College London Hospitals NHS Foundation Trust)
Published on Jun 20, 2019in BMJ27.60
Geraldine M Clarke , Stefano Conti2
Estimated H-index: 2
(NHS England)
+ 1 AuthorsAdam Steventon16
Estimated H-index: 16
### What you need to know Interventions to transform the delivery of health and social care are being implemented widely, such as those linked to Accountable Care Organizations in the United States,1 or to integrated care systems in the UK.2 Assessing the impact of these health interventions enables healthcare teams to learn and to improve services, and can inform future policy.3 However, some healthcare interventions are implemented without high quality evaluation, in ways that require onerous ...
Published on May 16, 2019in Stroke6.05
Sandy Middleton19
Estimated H-index: 19
(ACU: Australian Catholic University),
Simeon Dale8
Estimated H-index: 8
(ACU: Australian Catholic University)
+ 25 AuthorsRichard Gerraty9
Estimated H-index: 9
(Monash University)
Published on May 7, 2019in BioMed Research International2.20
Raúl Murillo18
Estimated H-index: 18
(IARC: International Agency for Research on Cancer),
Claudia Robles6
Estimated H-index: 6
Implementation of evidence-based cancer prevention and early detection in low- and middle-income countries (LMIC) is challenging. Limited and inappropriate introduction of novel alternatives results in an equity gap whereby low-income populations receive a lower benefit. Implementation research represents an opportunity to foster the adoption and expansion of evidence-based cancer control strategies; however, scientific development in high-income countries does not necessarily fulfill the partic...
Published on Dec 1, 2018in Implementation Science4.53
Justin Presseau19
Estimated H-index: 19
(Newcastle University),
Joan Mackintosh13
Estimated H-index: 13
(Newcastle University)
+ 9 AuthorsEileen Kaner36
Estimated H-index: 36
(Newcastle University)
National diabetes audits in the UK show room for improvement in the quality of care delivered to people with type 2 diabetes in primary care. Systematic reviews of quality improvement interventions show that such approaches can be effective but there is wide variability between trials and little understanding concerning what explains this variability. A national cohort study of primary care across 99 UK practices identified modifiable predictors of healthcare professionals’ prescribing, advising...
Published on Aug 1, 2018in Resuscitation4.57
Filip Haegdorens2
Estimated H-index: 2
(University of Antwerp),
Peter Van Bogaert11
Estimated H-index: 11
(University of Antwerp)
+ 4 AuthorsKoenraad G. Monsieurs32
Estimated H-index: 32
(University of Antwerp)
Abstract Aim Deterioration of hospitalised patients is often missed, misinterpreted, and mismanaged. Rapid Response Systems (RRSs) have been proposed to solve this problem. This study aimed to investigate the effect of an RRS on the incidence of unexpected death, cardiac arrest with cardiopulmonary resuscitation (CPR), and unplanned intensive care unit (ICU) admission. Methods We conducted a stepped wedge cluster randomised controlled trial including 14 Belgian acute care hospitals with two medi...
View next paperResistance to Changes in Diet