Reviews Of Medical Studies May Be Tainted By Funders' Influence

Oct 12, 2016
Originally published on October 14, 2016 10:43 am

When doctors want to help untangle confusing and sometimes contradictory findings in the scientific literature, they often turn to specially crafted summary studies. These are considered the gold standard for evidence. But one of the leading advocates for this practice is now raising alarm about them, because they are increasingly being tainted by commercial interests.

For many years, these studies — called meta-analyses and systematic reviews — seemed to solve a big problem. Doctors who had once relied on each other's expert opinions to select the best treatments gradually turned to careful scientific studies instead.

But the number of studies mushroomed and often came to different conclusions. So in the 1990s, doctors and medical advisory committees started relying on studies that combined results from many different research projects to streamline the search for answers.

These kinds of studies are "extremely important," says Dr. John Ioannidis, a professor of medicine health research and policy at Stanford University. He has conducted many of these types of studies over the course of his career. "They're trying to make some sense out of a very convoluted scientific and medical literature."

But Ioannidis says unfortunately things have gotten out of hand. First, "the problem is that there are just too many meta-analyses," Ioannidis says.

In a recent study on the subject, titled "The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-Analyses," he chose as an example studies involving antidepressant drugs. "There are 185 of them published in the literature within seven years, which means about 25 of them published every year for the very same drugs and the very same indication, major depression," Ioannidis said.

What's worse, they're increasingly being generated by scientists who have financial interests in the outcome, Ioannidis found.

"About 80 percent of them have been funded or have some other conflicts of interest with manufacturers of these drugs," he says. "If you look at what their conclusions are, those that have been authored by industry employees, practically all of them, with one exception, have claimed that there are no caveats about antidepressants."

You have to read deep into the studies to find warnings about potential suicide risks, for example — if downsides are mentioned at all.

That's potentially misleading to doctors who turn to this kind of analysis to get a quick take on what works and what doesn't. Ioannidis says the drug industry has started using meta-analysis for commercial purposes, rather than as a disinterested look at the evidence.

"They can get the results or at least the interpretation that fits their needs. So you have the most powerful and most prestigious design in current medical evidence, and it can be easily manipulated as an advertisement, as a marketing tool."

And that defeats one of the main purposes of these studies, which is to make an overflowing scientific literature more manageable.

Peter Kramer, a clinical professor emeritus at Brown University and author of Listening to Prozac, took a deep dive into meta-analyses when he was writing his latest book, Ordinarily Well. He found the situation even worse than Ioannidis suggests.

"In some ways my doubts are stronger than his," Kramer told Shots.

Some analyses he looked at were trying to parse very subtle differences — for example, comparing two very similar antidepressants. You could get any answer you want, depending on how you set up the study, Kramer says.

And the problems he found aren't simply commercial conflicts of interest. For example, he saw biases among academics who were wedded to the notion that placebos are just as good as actual drugs for depression.

"Even on that side of the equation, where there's no adverse sponsorship but just an allegiance to an idea, I thought it wasn't always the case that everybody's hands were on the table," Kramer said.

But financial conflicts are easier to identify. Five years ago, the National Academy's Institute of Medicine (now called the National Academy of Medicine) suggested dozens of standards that should apply to systematic reviews of the research literature.

The influence of funders is a concern, says Dr. Alfred Berg, a professor emeritus of family medicine at the University of Washington who chaired one of the committees.

"If there's obvious sponsorship from an organization that might have a conflict of interest, that's probably not a good idea," he says. It's reasonable to suggest that scientists with clear financial conflicts of interest should not be producing these studies. "Is it going to happen in my lifetime?" Berg says. "Probably not!"

Pfizer, one of the many drug companies that fund meta-analyses involving their own products, declined an interview request. But the company noted in an emailed statement that the drug company "shares data from its clinical trials with outside members of the research community for them to conduct their own independent meta-analyses."

Fortunately, scientific journals are starting to do a better job of making sure researchers disclose their financial interests. It's up to readers now to take heed.

Email Richard Harris at rharris@npr.org.

Copyright 2017 NPR. To see more, visit http://www.npr.org/.

AUDIE CORNISH, HOST:

When you show up at the doctor's office with a serious medical condition, chances are your doctor will decide what treatment to offer by drawing on a specific kind of scientific study, one that pulls together information from many individual studies. Now, this technique helps sort through research that's often confusing and sometimes contradictory. But for a long time it's been considered the gold standard for scientific evidence. Researchers are now raising concerns that many of these studies are tainted by commercial interests. NPR's Richard Harris explains.

RICHARD HARRIS, BYLINE: Doctors used to rely on each other's opinions to figure out what worked and didn't work in medicine. More recently they turned to evidence from controlled clinical trials to guard treatments. But these days there are so many clinical trials; medical researchers have come to rely on techniques the combine evidence from many studies to come up with a sharper answer. These are called meta-analyses and systematic analyses.

JOHN IOANNIDIS: They are extremely important. They're trying to make some sense out of very convoluted scientific and medical literature.

HARRIS: John Ioannidis at Stanford is a big fan of this type of analysis, but he says unfortunately things have gotten out of hand. The scientific literature is now overflowing with these kinds of studies. To illustrate this point, Dr. Ioannidis recently looked at meta-analyses of antidepressant drugs.

IOANNIDIS: The problem is that there's just too many meta-analyses. There's 185 of them published in the literature within seven years, which means about 25 of them published every year for the very same drugs and then the very same indication, major depression.

HARRIS: What's worse, they're increasingly being generated by scientists who have financial interests in the outcome.

IOANNIDIS: About 80 percent of them have been funded or have some other conflicts of interest with manufacturers of these drugs. If you look at what their conclusions are, those that have been authored by industry employees - practically all of them with one exception - have claimed that there's no caveats about antidepressants.

HARRIS: You have to read deep into the studies to hunt for warnings about potential suicide risks, for example. That's potentially misleading doctors who turn to this kind of analysis to get a quick take on what works and what doesn't. Ioannidis says the drug industry has started using meta-analysis for commercial purposes rather than as a disinterested look at the evidence.

IOANNIDIS: They can get the results or at least the interpretation that fits their needs. So you have the most powerful and most prestigious design in current medical evidence, and it can be easily manipulated as an advertisement, as a marketing tool.

HARRIS: And it's defeating one of the main purposes of these studies, which is to make an overflowing scientific literature more manageable. Brown University psychiatry professor Peter Kramer, famous for his book "Listening To Prozac," took a deep dive into meta-analysis when he was writing his latest book, "Ordinarily Well." He found the situation was even worse than Ioannidis suggests.

PETER KRAMER: In some ways my doubts are stronger than his.

HARRIS: Some analyses he looked at were trying to parse differences that were simply too fine - for example, comparing two very similar antidepressants. You could get any answer you want depending upon how you set up the study.

Dr. Kramer says the problems aren't simply commercial conflicts of interest. For example, he saw biases among academics who were committed to the idea that placebos are just as good as actual drugs for depression.

KRAMER: Even on that side of the equation where there's no adverse sponsorship - it's just an allegiance to an idea - I thought, you know, it wasn't always the case that everybody's hands were on the table.

HARRIS: But financial conflicts are easier to identify. Five years ago, the National Academies Institute of Medicine suggested dozens of standards that should apply to systematic reviews of the research literature. Dr. Alfred Berg from the University of Washington chaired one of the committees. Money was one big concern.

ALFRED BERG: If there's obvious sponsorship from an organization that might have a financial conflict, that's probably not a good idea.

HARRIS: Would you say that people who have financial conflicts of interest should actually just stop doing systematic reviews and meta-analysis? Is that a reasonable suggestion?

BERG: (Laughter) Of course it's a reasonable suggestion. Is it going to happen in my lifetime - probably not.

HARRIS: Fortunately scientific journals are doing a better job of making sure researchers disclose their financial interest. It's up to readers now to take heed. Richard Harris, NPR News. Transcript provided by NPR, Copyright NPR.