Altman made that statement in a 1994 BMJ article entitled “The scandal of poor medical research.” Here we are, 24 years later, and nothing has changed. In fact, thanks to the rise of predatory journals, things are much worse.
Altman lamented research containing flaws such as “the use of inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation” and felt many poor studies were the result of pressure on researchers to publish.
Should every physician do research? Altman said, “A common argument in favor of every doctor doing some research is that it provides useful experience and may help doctors to interpret the published research of others. Carrying out a sensible study, even on a small scale, is indeed useful, but carrying out an ill-designed study in ignorance of scientific principles and getting it published surely teaches several undesirable lessons.”
In 2005, noted statistician John Ioannidis said 90% of all medical research is flawed. When interviewed a few days ago by Eric Topol on Medscape, he had not changed his mind saying, “one can see it as the glass half empty or half full, or 10% full or maybe a little bit more.”
Edward Tufte, another famous statistician, tweeted the other day that every statistical model should come with a warning “similar to those accompanying every prescription drug” because mistakes in data analysis can shorten thousands of lives.
 |
| Click on the figure to enlarge it |
Thanks to @michelaccad for letting me know about the Tufte warning
6 comments:
As a student I was involved in research which even at the time I knew was bullshit. Poorly designed, underpowered studies were the rule for my classmates and I, who for the most part were encouraged to churn out crap to advance the careers of our bosses.
One solution is to significantly de-emphasise the requirement for students and junior staff to do research to get on to training programs. In one move you cut off the supply of free and easily exploitable labour to professors, and free up a huge amount of time for already overworked people.
Anon, I agree with you. The question is how does one make the change you suggest happen?
I have been thinking about this post for several days. I was hoping there would be more discussion / comments, I am curious to hear other opinions. I am a junior general surgery resident at a community hospital with little to no research support. And yet there is this unrealistic expectation that we will generate well-designed studies and we are oddly congratulated when we publish a case report to the "Internet Journal of Something" for an insane publishing fee. The only reason I see to participate is for something to show for fellowship interviews. We are taught that certain specialties are "research heavy" which drives the machine. I think I speak for most residents - the work is not great and feels like the modern day equivalent of peddling Dr. Poppy's Wonder Elixir. But I see this more and more becoming a means of measuring resident performance
Anon, thanks for the comment. I agree. Research has gone from something meaningful to getting one's ticket punched. I don't see how it can be fixed.
Things are worse in one respect: existing research, not necessarily settled science, is the basis for determining what kind of medical care will be covered under the ACA, and perhaps what will simply be available to some patients. This has led to some radical changes in some types of primary care, not just expensive and dangerous procedures.
Hypothyroid patients have fallen through the cracks. Because it took a long time to reduce the error in the TSH assay (although it was reasonably good by 1990), that meant a dramatic reduction in the upper limit of the TSH reference range from the initial starting point of 10. Alas, this moving target made clinical trials to determine T4 dosing standards unfeasible, so the USPSTF told endocrinologists to go back to the drawing board and set that limit at 10.00. Though at least study suggested a strong relationship between TSH and triglycerides, 0 was narrowly in the confidential interval for the calculated odds ratio used to make the decision, so the authors concluded that there was no relationship.
Although our government considers reducing medical care costs to be a top objective, does the quality of that care have to suffer this way?
Sorry for the delay in posting your comments. Thanks.
Post a Comment
Note: Only a member of this blog may post a comment.