If you cared to follow some of the more controversial threads here at Longecity, this should not come as a surprise - most research results are junk science. Many astute posters here at Longecity are quick to point out flaws in various studies and it turns out they are probably correct about the validity of the results. MOST cancer and health study results cannot be reproduced. Nine out of ten cancer study results cannot be reproduced!!
This reminds me of the yogurt thread where the mice lived but a fraction of the days that well-cared for lab mice live, yet it was touted as a breakthrough lifespan study by its authors.
And it dovetails nicely with this thread about mouse lifespan studies.
Some choice quotes from the article:
The two note that they are not alone in finding academic biomedical research to be sketchy. Three researchers at Bayer Healthcare published an article [PDF] in the September 2011 Nature Reviews: Drug Discovery in which they assert “validation projects that were started in our company based on exciting published data have often resulted in disillusionment when key data could not be reproduced.” How bad was the Bayer researchers’ disillusionment with academic lab results? They report that of 67 projects analyzed “only in 20 to 25 percent were the relevant published data completely in line with our in-house findings.”
Perhaps results from high-end journals have a better record? Not so, say the Bayer scientists. “Surprisingly, even publications in prestigious journals or from several independent groups did not ensure reproducibility. Indeed, our analysis revealed that the reproducibility of published data did not significantly correlate with journal impact factors, the number of publications on the respective target or the number of independent groups that authored the publications.”
Given all the brouhaha [PDF] over how financial interests are allegedly distorting pharmaceutical company research, it’s more than a bit ironic that it is pharmaceutical company scientists who are calling academic researchers to account. Back in 2004, an American Medical Association report [PDF] on conflicts of interest noted that reviews comparing academic and industry research found, "Most authors have concluded that industry-funded studies published in peer-reviewed journals are of equivalent or higher quality than non-industry funded clinical trials.” In an email, Begley, who was an academic researcher for 25 years before joining Amgen, agrees, “My impression, I don't have hard data, is that studies from large companies is of higher quality. Those companies are going to lay down millions of dollars if a study is positive. And they don't want to terminate a program prematurely so a negative study is more likely to be real.”
These results strongly suggest that the current biomedical research and publication system is wasting scads of money and talent. What can be done to improve the situation? Perhaps, as some Nature online commenters have bitterly suggested, researchers should submit their work directly to Bayer and Amgen for peer review? In fact, some venture companies are hedging against “academic risk” when it comes to investing in biomedical startups by hiring contract research organizations to vet academic science.
The FDA/big Pharma nexus is a legitimate concern, however, big pharma has A LOT at stake when they pursue new drugs, and thus (like Amgen) absolutely need to reproduce academic studies to make sure they are legit. This is a good thing.
It goes to show how difficult it is to conduct a robust relevant study.
