Less than 1 in 10 Phase 3 Clinical Trials Are Cited at POC

A study has characterized the factors that influence the adoption of clinical research into practice by comparing the citation rates of medical research papers between 18 different specialties, article type, geography, and time. The results were published recently in eLife.

The work reveals that most clinical trials are never cited by the time they reach the point of care—where clinical trials are embedded into regular medical practice. 

Furthermore, less than one in two Phase 3 clinical trials are ever cited across nine out of 18 medical specialties. Phase 3 clinical trials typically involve a large number of participants and are designed to evaluate the safety and effectiveness of a new medical intervention across the general population. 

The results represent a preliminary step to understanding why increased research funding in the US has not yielded equivalent results in health outcomes. The findings could be used to monitor the efficiency of research investments and evaluate health policies influencing the translation of biomedical studies to clinical practice.

The lack of evidence to assess the effectiveness and efficiency of treatments is a contributing factor that underpins the unsustainable expenditure of the US health care system. This is worsened by a limited understanding of the factors that influence the translation of clinical research into practice. 

“This may partially explain why, in the US health care systems, many patients do not receive the recommended services and many receive treatment that is not necessary or appropriate for them,” says Moustafa Abdalla, MD, PhD, resident in the Department of Surgery at Massachusetts General Hospital, a fellow at Harvard Medical School, Boston, MA, and co-author of the study 

Practice guidelines, clinical trials most cited across specialties

The team used the resource tool UpToDate, which provides current, evidence-based clinical information at the point of medical practice, to assess what clinical research makes it into practice. By leveraging a dataset of more than 10,000 UpToDate articles, sampled every three months between 2011–2020, they were able to trace the path of research from initial publication to medical practice, compared to the 2.4 million uncited studies published during the same time window across 18 medical specialties. 

The analysis revealed substantial variation in how different medical specialties adopt research with regard to the frequency of literature citations and the quality of evidence incorporated. Across all specialties, only 0.4 percent–2.4 percent of published clinical research is ever cited in UpToDate. Amongst the 18 specialties included in the analysis, neurology had the highest citation rate at 2.4 percent (2,057 of 85,843 research articles), and pathology had the lowest rate at 0.4 percent (317 of 69,343 research articles). 

The proportion of citations varied substantially by the type of article the research was presented in. Practice guidelines represented the most likely article type to be cited, with nine of the 18 specialties receiving citations at a rate of over 13 percent. Clinical trials were the second most likely to be cited but were also the most variable. 

In nine of the 18 specialties, the team observed that less than one in 10 Phase 3 clinical trials ever gets cited at the point of care. 

Can NIH funding have an impact on citations?

The team next sought to characterize the effects of National Institutes of Health (NIH) funding on the number of citations a paper receives, primarily focusing on the US. They found that, across all specialties in the past decade, average annual NIH funding correlated strongly with the total number of citations. 

Given this strong correlation, the team then calculated the effective cost per new citation at the point of care. This was used to represent the approximate indirect cost of bringing clinical research to medical practice in NIH funding dollars. In urology and nephrology, the cost of bringing a new citation to the point-of-care was estimated at $48,086.18 per article (not including the intercept of the model which reflects setup and startup costs). On the other side of the spectrum, in pathology, it cost $874.85 for every additional citation. 

Whilst UpToDate represents the largest point-of-care resource, its relevance varies by specialty and training status. Therefore, while the study uses UpToDate as a metric of translation of research, the authors say a citation does not necessarily indicate actual changes in practice. Instead, citations represent the adoption of knowledge to support current approaches, inform new changes in practice, or highlight points of controversy. 

“While the findings of our study are largely descriptive and explanatory, we have provided a baseline for monitoring the efficiency of research investments. This may eventually lead to the development of strategies to quantify the effectiveness of proposed reforms to the biomedical scientific process,” says Mohamed Abdalla, PhD, a scientist at Institute for Better Health, Trillium Health Partners, ON, Canada, and co-author of the study.

- This press release was originally published on the eLife website