For years, doctors have often turned to meta-analyses to make sense of conflicting research.
Considered the highest form of scientific analysis, these summary studies use meticulous statistical research to synthesize the current literature and give the most comprehensive final word on a given topic.
Yet, recent reports have begun to reveal the pitfalls of this kind of research — leaving many to wonder if there’s any kind of scientific study you can trust.
One researcher in particular — John Ioannidis — has been sounding the alarm that meta-analysis can easily be full of hidden biases and conflicts of interest.
According to a report in Quartz, Ioannidis’ most recent study of meta-analysis led him to conclude that “the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, or conflicted.”
One of the forces pushing the research world toward skewed research is the pressure to publish quickly and often. For example, Ioannidis looked at a group of studies involving antidepressant drugs.
“There are 185 of them published in the literature within seven years, which means about 25 of them published every year for the very same drugs and the very same indication, major depression,” Ioannidis said in a recent NPR report.
And like other scientific studies, Ioannidis argues that the authors of meta-analysis often have financial interests tied to the outcome of the research.
After examining those 185 systematic reviews on antidepressants, Ioannidis found that among the 58 that included a negative statement about the drugs, only one had an author who worked at a pharmaceutical company at the time. All told, when an industry author contributed to a meta-analysis, it was 22 times less likely to generate a negative outcome about the drug.
And when authors have a bias, they can not only set up the study to get the answer they want, they can also bury negative information, such as side effects for drugs.
Peter Kramer, a professor interviewed by NPR, also blames ideological biases as a reason a study can become tainted. For example, he said he’s seen biases in academics working on studies about antidepressants who already believe placebos are just as useful for patients.
What’s being done to improve the situation?
Groups like the AllTrials initiative are pushing for greater transparency in the research community by getting pharmaceutical companies and health organizations to publish trials regardless of the outcome. The UN is also pressuring governments to ensure all clinical trial results are made public.
Ioannidis has also pitched the idea of making meta-analysis reviews living documents, like Wikipedia pages, that researchers can update and manage.
But for now, it’s partially up to doctors to read between the lines of every study — including meta-analyses — to understand the biases that could be distorting the bottom line.
