Phoebe Nguyen is a PhD candidate with the Methods in Evidence Synthesis Unit at Monash University. Her research focuses on improving the reporting and methodological conduct of interrupted time series (ITS) studies. She is passionate about developing practical tools and resources to incentivize and support researchers in adopting best research practices.
ABSTRACT:
An investigation of selective outcome reporting in interrupted time series studies: a methodological study
Nguyen PY [1], Turner SL [1], Korevaar E [1], Page MJ [1], McKenzie JE [1]
- Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia 3004
Background: Selective reporting bias occurs when authors report outcomes or results based on the results’ nature. Selective reporting bias has not been examined in interrupted time series (ITS) studies.
Methods: We systematically searched for published peer-reviewed protocols of ITS studies in 22 databases, and their corresponding results reports addressing primary research question (PRQ) in the protocol. For each PRQ, we identified matching pairs of outcomes in the protocol and results report. We defined a discrepancy as any outcome that was added, omitted, or had its primacy reclassified (e.g. from primary to secondary) in the results reports compared to its protocol.
Results: Our search returned 4,590 abstracts. After excluding ongoing studies and protocols without published results for the PRQ, we identified 44 ITS protocols (published between 2010-2022) with corresponding results reports. Non-reporting of outcomes was prevalent, with 24% (132/553) of outcomes defined in the protocol not reported in the results report, and 57% (25/44) of studies having at least one unreported outcome. Among the outcomes assessed, 52% (202/388) had a discrepancy in reporting, affecting 74% (31/42) of studies. The most common type of discrepancy was outcome specified in the protocol but was not reported in the report of results (34%, 132/388). Justifying changes in outcome reporting was infrequent; only 28% (56/202) of discrepancies were justified.
Conclusion: Non-reporting of outcomes and discrepancies in reported outcomes was common in our sample of ITS studies. This is concerning given the important role ITS studies have in informing public health and policy decision making.