Advertisement

Misconceptions surrounding the relationship between journal impact factor and citation distribution in veterinary medicine

  • Daniel S.J. Pang
    Correspondence
    Correspondence: Daniel SJ Pang, Department of Clinical Sciences, Faculty of Veterinary Medicine, Université de Montréal, 3200 Rue Sicotte, Saint-Hyacinthe, QC, J2S 2M2, Canada.
    Affiliations
    Department of Clinical Sciences, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, QC, Canada

    Groupe de recherche en pharmacologie animale du Québec (GREPAQ), Université de Montréal, Saint-Hyacinthe, QC, Canada

    Department of Veterinary Clinical and Diagnostic Sciences, Faculty of Veterinary Medicine, University of Calgary, Calgary, AB, Canada
    Search for articles by this author
Published:December 11, 2018DOI:https://doi.org/10.1016/j.vaa.2018.11.004

      Abstract

      Objective

      To define the relationship between journal impact factor (JIF) and citation distribution in veterinary journals. Citation distribution is a summary of the number of citations of individual papers published in a defined period, and JIF is said to represent the mean number of citations received by a paper published in a given journal. JIF is criticized for promoting unimportant differences between journals, exaggerating small differences in journal citation distributions by misrepresenting a skewed citation distribution. The hypothesis was that veterinary journals have a skewed citation distribution and that median citation rates between journals would be smaller than that indicated by JIF.

      Study design

      Bibliometric study.

      Animals

      None.

      Methods

      A published method was used to generate journal citation reports from a commercial database, with search limits set for document (‘article’ and ‘review’) and the 2 year citation window of interest. Citation distributions [median (range)] and cumulative citations were calculated for Veterinary Anaesthesia and Analgesia (Vet Anaesth Analg, 2007–2017), 11 preselected subject- and species-specific and general veterinary journals (2016) and veterinary journals from the top (n = 10) and bottom (n = 10) of the Veterinary Sciences category ranking (2016) with a 10 year publication record.

      Results

      Citation distributions were right-skewed for all journals, with 15–20% of papers contributing approximately 50% of citations. For Vet Anaesth Analg, the median citation distribution [1 (0–2)] did not change despite JIF ranging from 1.044 to 2.064 between 2007 and 2017. Calculated median citation rates revealed minimal differences between journals, with only three groups identified: bottom (median citation 0), preselected (median citation 1) and top (median citation 2) journals. These groups represent over 100 places in the JIF (0.316–3.148) ranking.

      Conclusions

      Ranking veterinary journals according to JIF is misleading, exaggerating differences while concealing minimally different citation distributions.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Veterinary Anaesthesia and Analgesia
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Adam D.
        The counting house.
        Nature. 2002; 415: 726-729
        • Adler R.
        • Ewing J.
        • Taylor P.
        Citation statistics.
        Statistical Science. 2009; 24: 1-14
        • Alberts B.
        Impact factor distortions.
        Science. 2013; 340: 787
        • Allen J.F.
        • Missirlis F.
        Queen Mary: nobody expects the Spanish Inquisition.
        Lancet. 2012; 379: 1785
        • Anon
        Wellcome Trust Open access policy.
        2005
        • Anon
        The impact factor game. It is time to find a better way to assess the scientific literature.
        PLoS Med. 2006; 3: e291
        • Anon
        The EASE Statement on inappropriate use of impact factors.
        2007
        • Anon
        San Francisco Declaration on Research Assessment.
        2012
        https://sfdora.org
        Date accessed: April 23, 2018
        • Anon
        Natural Sciences and Engineering Research Council of Canada Peer Review Manuals.
        2017
        • Archambault E.
        • Larivière V.
        History of journal impact factor: contingencies and consequences.
        Scientometrics. 2009; 79: 635-649
        • Bagioli M.
        Watch out for cheats in citation game.
        Nature. 2016; 535: 201
        • Bornmann L.
        • Daniel H.-D.
        The state of h index research. Is the h index the ideal way to measure performance?.
        EMBO Reports. 2009; 10: 2-6
        • Campbell P.
        Escape from the impact factor.
        Ethics Sci Environ Polit. 2008; 8: 5-7
        • Casadevall A.
        • Bertuzzi S.
        • Buchmeier M.J.
        • et al.
        ASM Journals eliminate Impact Factor information from journal websites.
        MBio. 2016; 7 (e01150–16)
        • Christopher M.
        Weighing the impact (factor) of publishing in veterinary journals.
        J Vet Cardiol. 2015; 17: 77-82
        • Clark C.
        Let me count the ways.
        Science. 1993; 261: 145
        • Colquhoun D.
        Challenging the tyranny of impact factors.
        Nature. 2003; 423: 479
        • Colquhoun D.
        How to get good science.
        Physiology News. 2007; 69: 12-14
        • DiBartola S.P.
        • Hinchcliff K.W.
        Metrics and the scientific literature: deciding what to read.
        J Vet Intern Med. 2017; 31: 629-632
        • Falagas M.E.
        • Kouranos V.D.
        • Michalopoulos A.
        • et al.
        Comparison of the distribution of citations received by articles published in high, moderate, and low impact factor journals in clinical medicine.
        Intern Med J. 2010; 40: 587-591
        • Ferreira R.C.
        • Antoneli F.
        • Briones M.R.S.
        The hidden factors in impact factors: a perspective from Brazilian science.
        Front Genet. 2013; 11: 130
        • Garfield E.
        The Clarivate Analytics Impact Factor.
        1994
        • Garfield E.
        How can impact factors be improved.
        BMJ. 1996; 313: 411-413
        • Garfield E.
        Der Impakct Faktor und seine richtige Anwendug [The Impact Factor and using it correctly].
        Der Unfallchirurg. 1998; 48: 413
        • Garfield E.
        The history and meaning of the journal impact factor.
        JAMA. 2006; 295: 90-93
        • Hicks D.
        • Wouters P.
        • Waltman L.
        • et al.
        Bibliometrics: the Leiden manifesto for research metrics.
        Nature. 2015; 520: 429-431
        • Larivière V.
        • Kiermer V.
        • MacCallum
        • et al.
        A simple proposal for the publication of journal citation distributions.
        bioRxiv. 2016; https://doi.org/10.1101/062109
        • Lawrence P.A.
        The mismeasurement of science.
        Curr Biol. 2007; 17: R583-R585
        • Martin B.R.
        Editors’ JIF-boosting stratagems - which are appropriate and which are not?.
        Res Policy. 2016; 45: 1-7
        • Mooney C.T.
        Journal impact or usage.
        J Small Anim Pract. 2013; 54: 59-60
        • Pulverer B.
        Impact fact-or fiction.
        EMBO J. 2013; 32: 1651-1652
        • Pulverer B.
        Dora the brave.
        EMBO. 2015; 34: 1601-1602
        • Rossner M.
        • Van Epps H.
        • Hill E.
        Show me the data.
        J Cell Biol. 2007; 179: 1091-1092
        • Schekman R.
        • Patterson M.
        Reforming research assessment.
        eLife. 2013; 2e00855
        • Seglen P.O.
        The skewness of science.
        J Am Soc Information Sci. 1992; 43: 628-638
        • Seglen P.O.
        Causal relationship between article citedness and journal impact.
        J Am Soc Information Sci. 1994; 45: 1-11
        • Seglen P.O.
        Why the impact factor of journals should not be used for evaluating research.
        BMJ. 1997; 314: 498-502
        • Strathern M.
        “Improving ratings”: audit in the British University system.
        Eur Rev. 1997; 5: 305-321
        • Van Noorden R.
        Metrics: a profusion of measures.
        Nature. 2010; 465: 864-866
        • Van Noorden R.
        Brazilian citation scheme outed.
        Nature. 2013; 500: 510-511
        • Wilsdon J.
        • Allen L.
        • Belfiore E.
        • et al.
        The Metric Tide: report of the independent review of the role of metrics in research assessment and management.
        2015
        https://doi.org/10.13140/RG.2.1.4929.1363
        Date accessed: April 23, 2018
        • Zhang L.
        • Rousseau R.
        • Sivertsen G.
        Science deserves to be judged by its contents, not by its wrapping: revisiting Seglen’s work on journal impact and research evaluation.
        PLoS One. 2017; 12: e0174205