Introduction
The novel Coronavirus 2019 disease (COVID-19) pandemic has influenced not only the clinical practice of orthopaedic surgery, but also the types of peer-reviewed literature published in the field.1 Though the pandemic’s academic impact was anticipated, the degree to which this pandemic has affected orthopaedic surgery literature has not been described. The aim of this study was to determine if articles related to the COVID-19 pandemic were more widely disseminated and had a higher impact than other articles published during the same time period. This was determined by assessing bibliometric data such as the number of citations and Altmetric Attention Score (AAS) of articles.
The number of citations a paper receives has been used as a metric for an article’s impact on the literature, which is a well-established surrogate to measure the impact of a publication, demonstrated across many specialties including orthopaedics.2, 3 AAS, introduced in 2010 as a tool to complement other bibliometrics, assesses the influence and dissemination of an article through the degree of attention received online and through social media. Both metrics have their limitations, in this case limiting the amount of journals and subsequent articles analyzed, in order to maintain a sample size manageable for individual review, while still adequate enough to capture a wide breadth of orthopaedic surgical journals. Analyzing both AAS and subsequent citations can help reveal how the COVID-19 pandemic influenced the orthopaedic surgery literature throughout 2020. We hypothesized that COVID-19 related articles would amass greater influence and dissemination, as well as generate a greater impact within the orthopaedic community, as represented by AAS and number of citations, respectively.
Materials and Methods
The twenty orthopaedic surgery journals with the highest impact factor in 2019 were selected using the Journal Citation Reports.4 Of these journals, a total of 7,562 articles published in 2020 were identified. COVID-19 related articles were identified by searching titles for "COVID", "SARS", "pandemic", "corona", "COVID-19", "SARS-CoV-2", “2019 nCoV”, or “2019 novel corona virus,” yielding a total of 223 (3%) articles. These articles were designated to the COVID-19 related article cohort, while the remaining articles made up the non-COVID-19 related article cohort. For each article, the number of times the article was subsequently cited in the literature was recorded using the I Cite tool from the National Institute of Health.5 The AAS, which is a weighted indicator of the influence an article has based on the attention it receives online, was also recorded. Both data points were assessed in March 2021.6 The AAS has been demonstrated to be correlated with more traditional bibliometrics such as impact factor and h-index.7 The Mann-Whitney U test was used to assess AAS and citations for COVID-19 versus non-COVID-19 related articles. For COVID-19 related articles, Kruskal-Wallis test was used to assess whether these AAS and citations varied by sub specialty of orthopaedic surgery (arthroplasty, general, trauma, spine, other), type of article (editorial, original article, or guidelines), study type (survey, review, case report, cross-sectional, other), and the quarter in which the article was published.8
Figure 1
AAS in COVID-19 articles and non-COVID-19 articles from orthopaedic journals included in the study. *** = p<0.001
Results
The 223 COVID-19 related articles had a combined AAS of 3,261 and an average AAS of 15 (range 0-957), whereas the 7,339 non-COVID-19 articles had a combined AAS of 41,596 and an average AAS of 6 (range 0-678) (p=0.019, Figure 1). The percentage of COVID-19 articles with an AAS of over 100 was nearly four times greater than the percentage of non-COVID-19 articles with an AAS of over 100 (5 out of 223 vs. 44 out of 7,339). For both COVID-19 and non-COVID-19 related orthopaedic articles, the most common AAS was zero (46% and 50%, respectively), meaning they were not disseminated at all across online news, blogs, and social media platforms.6 Dividing COVID-19 articles amongst subspecialty of orthopaedics (arthroplasty, general, trauma, spine, other), articles categorized as other had a significantly higher mean AAS than those pertaining to spine or trauma (51 vs. 20 & 6, respectively. p<0.001, Figure 2). There was no association between AAS and article type, study type, or quarter in which it was published for COVID-19 related articles, (p≥0.9 for all).
The 223 COVID-19 related articles accrued a total of 1,753 citations, with an average of eight citations per article (range, 0-95), whereas the 7,339 non-COVID-19 articles had a total of 7,787 citations with an average of one per article (range, 0-37) (p<0.001, Figure 3). Among the COVID-19 related articles, 10 articles accounted for 35% of the total number of citations, while 59 had no subsequent citations in the literature. With respect to the types of COVID-19 articles, original articles received significantly more citations than editorial articles (10 vs. 5, p<0.001, Figure 4). Furthermore, articles published in the second quarter of the year accrued more citations than those published in quarters three and four (10 vs. 2 & <1, respectively. p<0.001, Figure 5). For COVID-19 related articles, there was no association between the number of citations and the orthopaedic subspecialty (p=0.45) or study type (p=0.76).
Discussion
The COVID-19 pandemic has left an indelible mark on the practice of orthopaedic surgery. Given the impact of the global pandemic, we hypothesized that the orthopaedic surgery literature would reflect this influence. No study to date has attempted to assess the impact of the COVID-19 pandemic on orthopaedic surgery literature. With the uncertainty brought about by the pandemic, it is natural that many orthopaedic surgeons turned to the literature to both share their concerns and educate the global orthopaedic surgery community of best practices for continuing to care for patients with orthopaedic injuries, despite widespread strains on healthcare resources and risk of viral transmission to both patient and health care providers.
AAS has been established as a useful metric for assessing the influence and dissemination of scholarly work within the orthopaedic surgery literature.7, 9, 10, 11 Our analysis suggests that articles related to COVID-19 published in orthopaedic journals in 2020 were more influential and widely disseminated than those not related to COVID-19 as suggested by a significantly higher mean AAS. This finding is not unique to the orthopaedic literature, with similar trends being demonstrated in other fields.12, 13 Furthermore, when differentiating COVID-19 articles by subspecialty of orthopaedics, our analysis suggests articles that did not focus on one specific subspecialty had the highest average AAS.
Articles related to the COVID-19 pandemic have been significantly more impactful than other articles published concurrently, as assessed by subsequent citations in the literature. This finding is particularly telling as articles typically require a lag-time of several years to accrue subsequent citations in the literature. This suggests the immediacy and magnitude of the impact of these COVID-19 articles on orthopaedic surgery community.14, 15 Articles published in the second quarter of the year amassed more citations than those published later in the year. While the second quarter of 2020 represents the peak of the COVID-19 pandemic in many regions of the world including Europe and the United States and thus likely represents heightened interest in the COVID-19 pandemic, this finding is also confounded by time as these articles have been published longer and have had more time to accumulate subsequent citations. Further insight and time are required to assess how articles pertaining to the COVID-19 pandemic will affect clinical practice of orthopaedic surgery as healthcare systems transition to post-pandemic practice. Still, these articles may be important in guiding clinicians that continue to serve patients in areas where the condition of the COVID-19 pandemic is still severe.
This study has several limitations. Our review of the literature was limited to the 20 journals with the highest impact factors. This number was selected in order to capture a wide breadth of orthopaedic surgical journals, while maintaining a sample size manageable for individual review of articles for cohort division. Furthermore, this study is limited to analysis of articles within one year after publication. As article citation count requires several years of lag-time to plateau and equilibrate, subsequent analysis should be performed to compare these cohorts several years from now.16, 17
In conclusion, the orthopaedic surgery literature was significantly impacted by the COVID-19 pandemic. Articles pertaining to COVID-19 were more immediately influential, disseminated, and impactful than other articles published concurrently in orthopaedic surgery journals.
