Here
are the first three paragraphs of a story from the medical news site, MedPage
Today.
"ORLANDO
– Patients on kidney dialysis who are infected with Clostridium difficile
appeared to have a greater risk of infection relapse and also appeared to have
a higher all-cause mortality that patients who do not have kidney disease,
researchers said here.
"Mortality
related to C. difficile infection was 3.8% among the 104 patients with
end-stage renal disease (ESRD) and 1.46% among 300 controls without ESRD, said
Massini Merzkani, MD, resident in internal medicine at the Albert Einstein
School of Medicine's Jacobi Medical Center in Bronx, N.Y. (No data as to
significance were presented.)
"In
his poster presentation at the National Kidney Foundation 2013 Spring Clinical
Meetings, Merzkani told MedPage Today that the relapse rate in severe C.
difficile infection was 34.7% in the controls and 45.2% in the patients
with ESRD. (No data as to significance were presented.)"
Does it make you nervous that "No data as to significance were
presented"?
It should.
The authors didn't analyze the data for statistical
significance. Would there be any way to do it yourself?
Yes, if you knew which statistical test to use.
Should a science reporter know something about statistics?
Yes, and the story was reviewed by an emeritus professor of
medicine at an Ivy League medical school who should have known too.
In addition, there is a rather interesting math error. The
mortality rate of 1.46% for the 300 controls doesn't compute. [300 x 0.0146 =
4.38] Unless 4.38 people died, the figure must be wrong.
Since both the mortality and relapse rates are categorical
(yes or no) variables, the correct statistical test to use is Fisher's exact
test.
The p value for mortality is 0.21 and for relapse is 0.061.
Neither difference is statistically significant which means that based on this
study, one cannot say that "C. diff is dangerous in ESRD."
You might point out that a p of 0.061 is pretty close to the
magical value of 0.05. That is true, but there is another major flaw in the
study. The article says the ESRD patients "were compared with patients
without chronic kidney disease who were admitted with C. difficile infection
during the same time period. The researchers calculated that randomly selecting
300 of the 2,400 control patients would produce a valid comparison of
outcomes."
Despite that "calculation," the comparison is
invalid. One cannot simply compare ESRD patients to random patients. They would
at least need to be matched for age, sex, co-morbidities other than ESRD and
perhaps other variables to eliminate confounding.
It is possible that ESRD patients will have worse outcomes if
they contract C. diff colitis. But this study doesn't prove that, and the story
is misleading.
It's 2013. I agree with The Guardian's Observer column which
says that Nate Silver's accurate predictions highlight "the importance of
statistical literacy in our data-heavy age."
