Advertisement

Randomization, blinding, data handling and sample size estimation in papers published in Veterinary Anaesthesia and Analgesia in 2009 and 2019

Published:September 28, 2021DOI:https://doi.org/10.1016/j.vaa.2021.09.004

      Abstract

      Objective

      To evaluate reporting of items indicative of bias and weak study design.

      Study design

      Literature survey.

      Population

      Papers published in Veterinary Anaesthesia and Analgesia.

      Methods

      Reporting of randomization, blinding, sample size estimation and data exclusion were compared for papers published separated by a 10 year interval. A reporting rate of more than 95% was considered ideal. The availability of data supporting results in a publicly accessible repository was also assessed. Selected papers were randomized and identifiers removed for review, with data from 59 (57 in 2009, two in 2008) and 56 (52 in 2019, four in 2018) papers analyzed. Items were categorized for completeness of reporting using a previously published operationalized checklist. Two reviewers reviewed all papers independently.

      Results

      Full reporting of randomization increased over time from 13.6% to 85.7% [95% confidence interval (CI), 57.8–86.6%; p < 0.0001], as did sample size estimation (from 0% to 20%; 95% CI, 7.6–32.4%; p = 0.002). Reporting of blinding (49.2% and 50.0%; 95% CI, –18.3% to 20.0%; p = 1.0) and exclusions of samples/animals (39.0% and 50.0%; 95% CI, –8.8% to 30.8%; p = 0.3) did not change significantly. Data availability was low (2008/2009, zero papers; 2018/2019, two papers). None of the items studied exceeded the predetermined ideal reporting rate.

      Conclusions and clinical relevance

      These results indicate that reporting quality remains low, with a risk of bias.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Anon
        A fair share: the concept of sharing primary data is generating unnecessary angst in the psychology community.
        Nature. 2006; 444: 653-654
        • Anon
        Reducing our irreproducibility.
        Nature. 2013; 496: 398
        • Axiak Flammer S.M.
        • Trim C.M.
        ARRIVE and CONSORT guidelines: do they have a place in Veterinary Anaesthesia and Analgesia?.
        Vet Anaesth Analg. 2016; 43: 2-4
        • Avey M.T.
        • Moher D.
        • Sullivan K.J.
        • et al.
        The devil is in the details: incomplete reporting in preclinical animal research.
        PLoS One. 2016; 11e0166733
        • Begley C.G.
        • Ioannidis J.P.A.
        Reproducibility in science: improving the standard for basic and preclinical research.
        Circ Res. 2015; 116: 116-126
        • Blanco D.
        • Biggane A.M.
        • Cobo E.
        • MiRoR network
        Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?.
        Trials. 2018; 19: 80
        • Boutron I.
        • Page M.J.
        • Higgins J.P.T.
        • et al.
        Considering bias and conflicts of interest among the included studies.
        in: Higgins J.P.T. Thomas J. Chandler J. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (updated February 2021). 2021. Cochrane, 2021 (Available from)
        • Brown D.C.
        Control of selection bias in parallel-group controlled clinical trials in dogs and cats: 97 trials (2000–2005).
        J Am Vet Med Assoc. 2006; 229: 990-993
        • Collins F.S.
        • Tabak L.A.
        NIH plans to enhance reproducibility.
        Nature. 2014; 505: 612-613
        • Cramond F.
        • Irvine C.
        • Liao J.
        • et al.
        Protocol for a retrospective, controlled cohort study of the impact of a change in Nature journals’ editorial policy for life sciences research on the completeness of reporting study design and execution.
        Scientometrics. 2016; 108: 315-328
        • Curtis M.J.
        • Alexander S.
        • Cirino G.
        • et al.
        Experimental design and analysis and their reporting II: updated and simplified guidance for authors and peer reviewers.
        Br J Pharmacol. 2018; 175: 987-993
        • Di Girolamo N.
        • Giuffrida M.A.
        • Winter A.L.
        • Meursinge Reynders R.
        In veterinary trials reporting and communication regarding randomisation procedures is suboptimal.
        Vet Rec. 2017; 181: 195
        • Equator Network
        Enhancing the QUAlity and Transparency Of health Research: search for reporting guidelines.
        2021
        • Fisher M.
        • Feuerstein G.
        • Howells D.W.
        • et al.
        Update of the stroke therapy academic industry roundtable preclinical recommendations.
        Stroke. 2009; 40: 2244-2250
        • Freedman L.P.
        • Cockburn I.M.
        • Simcoe T.S.
        The economics of reproducibility in preclinical research.
        PLoS Biol. 2015; 13e1002165
        • Giuffrida M.A.
        Type II error and statistical power in reports of small animal clinical trials.
        J Am Vet Med Assoc. 2014; 244: 1075-1080
        • Grindlay D.J.C.
        • Dean R.S.
        • Christopher M.M.
        • Brennan M.L.
        A survey of the awareness, knowledge, policies and views of veterinary journal Editors-in-Chief on reporting guidelines for publication of research.
        Bio Med Central Vet Res. 2014; 10: 10
        • Han S.
        • Olonisakin T.F.
        • Pribis J.P.
        • et al.
        A checklist is associated with increased quality of reporting preclinical biomedical research: a systematic review.
        PLoS One. 2017; 12e0183591
        • Hofmeister E.H.
        • King J.
        • Read M.R.
        • Budsberg S.C.
        Sample size and statistical power in the small-animal analgesia literature.
        J Small Anim Pract. 2007; 48: 76-79
        • Iqbal S.A.
        • Wallach J.D.
        • Khoury M.J.
        • et al.
        Reproducible research practices and transparency across the biomedical literature.
        PLoS Biol. 2016; 14e1002333
        • Kilkenny C.
        • Browne W.J.
        • Cuthill I.C.
        • et al.
        Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research.
        PLoS Biol. 2010; 8e1000412
        • Landis S.C.
        • Amara S.G.
        • Asadullah K.
        • et al.
        A call for transparent reporting to optimize the predictive value of preclinical research.
        Nature. 2012; 490: 187-191
        • Leung V.
        • Rousseau-Blass F.
        • Beauchamp G.
        • Pang D.S.J.
        ARRIVE has not ARRIVEd: support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia.
        PLoS One. 2018; 13e0197882
        • Lu M.J.
        • Zhong W.H.
        • Liu Y.X.
        • et al.
        Sample size for assessing agreement between two methods of measurement by Bland–Altman method.
        Int J Biostat. 2016; 12: 20150039
        • Lund E.M.
        • James K.M.
        • Neaton J.D.
        Veterinary randomized clinical trial reporting: a review of the small animal literature.
        J Vet Intern Med. 1998; 12: 57-60
        • Macleod M.R.
        • van der Worp H.B.
        • Sena E.S.
        • et al.
        Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality.
        Stroke. 2008; 39: 2824-2829
        • Macleod M.R.
        • Lawson McLean A.
        • Kyriakopoulou A.
        • et al.
        Risk of bias in reports of in vivo research: a focus for improvement.
        PLoS Biol. 2015; 13e1002273
        • Moher D.
        • Hopewell S.
        • Schulz K.F.
        CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials.
        Br Med J. 2010; 340: c869
        • Percie du Sert N.
        • Hurst V.
        • Ahluwalia A.
        • et al.
        The ARRIVE guidelines 2.0: updated guidelines for reporting animal research.
        PLoS Biol. 2020; 18e3000410
        • Pilat D.
        • Fukasaku Y.
        OECD principles and guidelines for access to research data from public funding.
        Data Sci J. 2007; 6: OD4-OD11https://doi.org/10.2481/dsj.6.OD4
        • PLoS One
        Data availability.
        2019
        • Reidpath D.D.
        • Allotey P.A.
        Data sharing on medical research: an empirical investigation.
        Bioethics. 2001; 15: 125-134
        • Rufiange M.
        • Rousseau-Blass F.
        • Pang D.S.J.
        Incomplete reporting of experimental studies and items associated with risk of bias in veterinary research.
        Vet Rec Open. 2019; 6e000322
        • Sargeant J.M.
        • Thompson A.
        • Valcour J.
        • et al.
        Quality of reporting of clinical trials of dogs and cats and associations with treatment effects.
        J Vet Intern Med. 2010; 24: 44-50
        • The NCQIP Collaborative Group
        Did a change in Nature journals’ editorial policy for life sciences research improve reporting?.
        BMJ Open Science. 2019; 3e000035
        • Totton S.C.
        • Cullen J.N.
        • Sargeant J.M.
        • O’Connor A.M.
        The reporting characteristics of bovine respiratory disease clinical intervention trials published prior to and following publication of the REFLECT statement.
        Prev Vet Med. 2018; 150: 117-125
        • Turner L.
        • Shamseer L.
        • Altman D.G.
        • et al.
        Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals.
        Cochrane Database Syst Rev. 2012; 11MR000030https://doi.org/10.1002/14651858.MR000030.pub2
        • Wagg C.R.
        • Kwong G.P.S.
        • Pang D.S.J.
        Application of confidence intervals to data interpretation.
        Can Vet J. 2016; 57: 547
        • Wellcome
        Use of animals in research policy.
        2020
        • Wicherts J.M.
        • Bakker M.
        • Molenaar D.
        Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results.
        PLoS One. 2011; 6e26828
        • Wilkinson M.D.
        • Dumontier M.
        • Aalbersberg I.
        • et al.
        The FAIR guiding principles for scientific data management and stewardship.
        Sci Data. 2016; 3: 160018