International Guidelines Aim to Reduce Potentially Critical Scientific Fraud

The number of retractions issued for scientific research articles in 2023 exceeded 10,000—smashing annual records. To date, publishers have struggled to clean up a slew of papers with serious integrity concerns.

Now a world-first framework for assessing research integrity in clinical guidelines, has been published today in the prominent international medical journal, The Lancet’s eClinical Medicine.

Monash University researchers in Australia developed the Research Integrity in Guidelines and evIDence synthesis (RIGID) framework.

Clinical trials that lack integrity, or present concerns over trustworthiness, can compromise patient care, both directly through unnecessary or harmful treatments, or indirectly through wasted resources and misdirected future medical research. “It is estimated that at least 25 percent of clinical trials informing clinical guidelines may not be trustworthy” according to RIGID co-lead author and research integrity expert, professor Ben Mol.

In 2023, the RIGID framework was applied to the International Evidence-Based Guidelines for Polycystic Ovary Syndrome (PCOS) in collaboration with 39 national and international societies and with input from 80 multidisciplinary experts, as well as consumers. The PCOS Guidelines have been downloaded more than 35,000 times, and the framework is now being applied in several other national and international guidelines. 

The RIGID framework and accompanying checklist uses a simple six-step approach to determine integrity risk, providing guideline developers, policy-makers, clinicians, and scientists with a roadmap to assess research integrity and to exclude untrustworthy research during evidence synthesis and clinical guideline development.

In 2009, the UK-based Committee on Publication Ethics (COPE) released a now widely-adopted policy for handling publication retractions. However, according to Mol, "these policies relate to editorial and publishing processes for retraction and do not provide guidance for researchers and guideline developers on how to handle suspicious or untrustworthy evidence which remains in the public domain, unretracted." Retraction practices are also often insufficient, relying on whistleblowers to flag integrity concerns and then authors, their institutions, and publishers to respond to these queries, where there is little incentive, processes, or systems to do so. “Queries to journals and authors' institutions are frequently met with silence or defensive threats of legal action, and official conclusions are seldom reached or formulated euphemistically, with limited retractions.”

According to professor Helena Teede, the other co-lead author “the perpetuation of problematic research is underpinned by complex systemic shortcomings, including inadequate application of quality research reporting processes or detection systems; lack of time and resources to investigate claims; lack of incentives for journals, institutions and whistleblowers; and barriers around reputational or legal implications. Most importantly, there is a lack of standardized procedures or protocols with appropriate oversight to manage integrity concerns.”

Key to the RIGID framework is transparency, where studies ranked by the integrity committee as having a moderate or high risk for integrity concerns are clearly documented, and authors are contacted to highlight the identified concerns. This limits their inclusion in the evidence synthesis, pending clarification. Authors are provided with the opportunity to engage in processes to address these issues, generally within two weeks of contact. This is usually met with little to no response, and the research evidence in question is then not considered in formulating conclusions or guideline recommendations. 

In the 2023 International Evidence-Based Guideline for PCOS, the framework weeded out problematic studies to ensure only trustworthy evidence informed clinical practice. With the use of RIGID, it turned out that no less than 45 percent of the RCTs assessed had moderate or high risk of integrity concerns, and these could not be trusted to guide practice.

The lead developer of the PCOS guideline, Teede noted, "it has been approved by the National Health and Medical Research Council of Australia, including the RIGID process, and has been viewed over 130,000 times and presented at over 100 conferences globally. 

“During guideline development, patient representatives and health professionals highly prioritized the importance of only relying on trustworthy research to guide clinical practice. The RIGID framework is now being applied to other international guidelines, including the Premature Ovarian Insufficiency (POI) International Guideline and the Australian adaptation of the European Society of Human Reproduction and Embryology (ESHRE) Unexplained Infertility Guideline,” she said.

The RIGID six-step approach:

The six steps include: 

1. Review: standard systematic review processes.
2. Exclude: studies which have been retracted are excluded, and those with expressions of concern are flagged for further evaluation.
3. Assess: remaining studies are assessed for integrity using an appropriate tool such as the Research Integrity Assessment (RIA) tool or Trustworthiness of Randomised Controlled Trials (TRACT) checklist and allocated an initial integrity risk rating of low, moderate or high risk for integrity concerns.
4. Discuss: integrity assessment results are discussed among integrity committee members with votes to determine final integrity risk rating allocations for each study.
5. Establish contact: low risk studies are included without author contact, whereas authors of studies ranked as moderate or high risk are contacted for clarification. 
6. Reassess: studies are reassessed for inclusion using the RIGID author response algorithm (reclassified as “included” where authors have provided a satisfactory response, “awaiting classification” where authors have engaged but time is needed to address concerns, or “not included” where authors have not responded to contact attempts).

- This press release was originally published by Monash University.