Melissa Rayhill’s study of headache clinical trials found discrepancies between reported outcomes and trial registry data, suggesting possible pitfalls in the peer review process.
Neurology Professor Finds Selective Reporting in Headache Literature
Published November 18, 2015 This content is archived.
story based on news release by ellen goldbaum
Only a quarter of headache clinical trials published in the field’s core journals over a nine-year period were registered in an approved clinical trial registry, according to a study led by Melissa L. Rayhill, MD, and published in Neurology.
Even of the registered trials, 38 percent had published on outcomes that did not match those posted on the registry website — suggesting that investigators changed what they planned to measure during the course of the trial.
As these results fall outside the scope of the study’s original design, much more caution should be used when interpreting them, says Rayhill, a University at Buffalo clinical assistant professor of neurology.
“Whether or not a paper answers a question the study was designed to address is critical, but that issue may not even be considered in the peer review process,” she notes.
Trial Data Published Despite Editorial Decision
Rayhill and her colleagues evaluated trials published from 2014 back to 2005 — the same year editors at major medical journals decided to only consider publishing the results of registered clinical trials.
During that period, they found that of the 225 trials published in Headache, Cephalalgia and the Journal of Headache and Pain, only 26 percent complied with trial registration requirements.
Ramifications for Clinical Guidelines
Rayhill — a neurologist in the UBMD practice plan who specializes in headache medicine — was compelled to scrutinize the quality of the published trials because physicians develop clinical guidelines and make clinical decisions based on their data.
“Our study suggests that selective reporting remains a problem in the headache medicine literature and should bring into question the quality of similar reporting in other disciplines throughout the medical literature,” she says.
She became interested in potential pitfalls in the peer review process while serving as an ad hoc reviewer for several journals during her fellowship training in headache medicine at Harvard Medical School/Brigham and Women’s Hospital.
“Although the peer review process is rigorous, if the few people who are asked to review a paper fail to bring up a potential bias or study design flaw, it may be published without addressing the issue at all,” she says.
Another problem, Rayhill notes, is that reviewers work to meet deadlines while receiving little to no compensation for their time.
Seeking Greater Awareness of Trial Registries
In recent years, more published headache trials have complied with trial registration requirements, Rayhill says.
Selective reporting is not unique to headache medicine, she believes. The percentage of registered trials published in other fields, such as geriatrics and surgery, ranges from comparable to somewhat higher.
“I hope our findings can positively impact the medical research process, increase awareness of trial registration and its rationale, and indirectly improve the quality of data available to physicians to make good clinical decisions for patients,” she says.
Rayhill’s study is titled “Registration Status and Outcome Reporting of Trials Published in Core Headache Medicine Journals.”