The Positive Impact of Sharing Negative Research Results
Investigators present studies on rare disease research that achieved negative or irreproducible results
AMSTERDAM, NETHERLANDS — In the era of “publish or perish,” research results that disprove the hypothesis tested often go unpublished. In addition, many researchers who have been unable to reproduce results from other laboratories have found it difficult to publish their findings. In this special issue on the “Null Hypothesis” of the Journal of Neuromuscular Diseases investigators illustrate the importance and value of sharing well-conducted studies with negative or irreproducible results.
“Negative results are just as useful as positive ones,” explained guest editors Virginia Arechavala-Gomeza, PhD, Neuromuscular Disorders, Biocruces Bizkaia Health Research Institute, Barakaldo; and Ikerbasque, Basque Foundation for Science, Bilbao, Spain; and Annemieke Aartsma-Rus, PhD, Department of Human Genetics, Leiden University Medical Center, Leiden, the Netherlands. “Research results that prove an original hypothesis to be incorrect advance knowledge as much as those that confirm it. If the experiments are done and analyzed correctly, negative results let us advance in our research and sharing this information avoids duplication of work. A publication bias towards therapeutic approaches that appear to have beneficial effects and a lack of published studies where therapies are either not useful or toxic, or where published results are not reproducible, not only impedes science and therapy development, but also leads to unnecessary duplication of effort and a waste of resources.”
“The need to share negative information is gaining backers, but in a rare disease field such as neuromuscular disorder research, not having all the information is a tragedy,” added Arechavala-Gomeza.
This special issue marks the inauguration of a new section of the Journal of Neuromuscular Diseases dedicated to negative results that nevertheless provide valuable contributions to the scientific record. It presents eight articles on a variety of subjects, from animal models to biomarkers and clinical trials.
One contribution re-examines the effect of simvastatin in Duchenne muscular dystrophy (DMD). A study in 2015, led by Nick Whitehead, PhD, University of Washington, Seattle, Washington, USA, provided evidence that simvastatin, a widely used cholesterol lowering drug, had potential to be a novel treatment for DMD. Two independent labs led by Aartsma-Rus in the Netherlands and Dominic J. Wells, VetMB, PhD, Department of Comparative Biomedical Sciences, Neuromuscular Diseases Group, Royal Veterinary College, London, UK, were unable to replicate the results in DMD model mice (mdx mice) although the treatment protocol was similar. They concluded that simvastatin did not improve DMD pathology, suggesting this could either be due to the ineffectiveness of simvastatin itself or low simvastatin plasma levels following oral administration in the mice’s chow.
In a rebuttal published in this issue, Whitehead noted that his lab and many others have conducted many studies, some unpublished, using several batches of simvastatin in chow, on at least three colonies of mdx mice. All of these showed an improvement in mdx skeletal and cardiac muscle health and function. He pointed out that his lab achieved much higher drug exposure. “Replication of results by an independent lab is the foundation of scientific inquiry,” he stated. “We welcome a careful effort to replicate our results. Unfortunately, they failed to meet the most critical aspect of animal drug studies—levels of drug exposure in the therapeutic range.”
In their response, Aartsma-Rus and colleagues explained that the batch of chow they fed to the mdx mice contained the same dose of simvastatin and was prepared in an identical manner as the original study and queried the results of similar studies that either did not mention the dose or used a higher dose. “While these studies confirm the therapeutic effect of simvastatin treatment in mdx mice, they do not confirm that this can be achieved at doses that are in the range of what humans use.”
Commenting on this productive exchange among researchers, the guest editors added, “Failure to replicate the finding does not mean that simvastatin does not work; it means that in different lab settings the same dose of simvastatin in chow resulted in different plasma levels. This is something that needs to be studied further and for now clinical trials should be held off. These kinds of studies are often not ‘sexy’ enough for regular funders. The Duchenne UK patient association funded the replication studies because they felt it important to only plan clinical trials if a robust treatment effect could be replicated.”
The issue also highlights reporting on a clinical trial on edasalonexent, a randomized, double-blind, placebo-controlled, global Phase 3 study in pediatric patients four–eight years old with DMD, the results of the PolarisDMD trial, by Richard S. Finkel, MD, St. Jude Children’s Research Hospital, Memphis, TN, and Nemours Children’s Hospital, Orlando, FL, USA, and colleagues. While edasalonexent was generally well tolerated with a manageable safety profile at the dose of 100 mg/kg/day, it did not achieve statistically significant improvement of DMD in the study population. Results did suggest edasalonexent may slow disease progression in patients under six.
“We of course regret that this did not work out,” noted the guest editors. “However, we really appreciate that the clinical experts involved and the industry sponsor pushed for timely publication so the field can benefit from these results.” Studies such as this are important to publish, especially when the results fail to confirm the preliminary favorable findings in the Phase 2 study. They provide an opportunity to explore the reasons for a failure to replicate the earlier findings.
“While most researchers consider it important to share negative results many themselves do not actually do this. This mindset needs to change,” commented Aartsma-Rus. “In a field such as neuromuscular disorders, which are rare diseases, not having all the information is a big problem. If a potential therapeutic approach is not effective in cells, mice, or patients, the field needs to know this as soon as possible to avoid unnecessary duplication of work.”
“Sharing results of a research question that turned out to be incorrect gives less ‘credit’ than sharing results of a research question that was correct,” added Arechavala-Gomeza. “We need to adopt a new mindset in which sharing well conducted research is valued regardless of whether the research question was correct or not.”
“The only really ‘negative’ results are those that end up in a drawer and are never shared!” concluded the guest editors.
- This press release was originally published on the IOS Press website