Here are the first three paragraphs of a story from the medical news site, MedPage Today.
"ORLANDO – Patients on kidney dialysis who are infected with Clostridium difficile appeared to have a greater risk of infection relapse and also appeared to have a higher all-cause mortality that patients who do not have kidney disease, researchers said here.
"Mortality related to C. difficile infection was 3.8% among the 104 patients with end-stage renal disease (ESRD) and 1.46% among 300 controls without ESRD, said Massini Merzkani, MD, resident in internal medicine at the Albert Einstein School of Medicine's Jacobi Medical Center in Bronx, N.Y. (No data as to significance were presented.)
"In his poster presentation at the National Kidney Foundation 2013 Spring Clinical Meetings, Merzkani told MedPage Today that the relapse rate in severe C. difficile infection was 34.7% in the controls and 45.2% in the patients with ESRD. (No data as to significance were presented.)"
Does it make you nervous that "No data as to significance were presented"?
The authors didn't analyze the data for statistical significance. Would there be any way to do it yourself?
Yes, if you knew which statistical test to use.
Should a science reporter know something about statistics?
Yes, and the story was reviewed by an emeritus professor of medicine at an Ivy League medical school who should have known too.
In addition, there is a rather interesting math error. The mortality rate of 1.46% for the 300 controls doesn't compute. [300 x 0.0146 = 4.38] Unless 4.38 people died, the figure must be wrong.
Since both the mortality and relapse rates are categorical (yes or no) variables, the correct statistical test to use is Fisher's exact test.
The p value for mortality is 0.21 and for relapse is 0.061. Neither difference is statistically significant which means that based on this study, one cannot say that "C. diff is dangerous in ESRD."
You might point out that a p of 0.061 is pretty close to the magical value of 0.05. That is true, but there is another major flaw in the study. The article says the ESRD patients "were compared with patients without chronic kidney disease who were admitted with C. difficile infection during the same time period. The researchers calculated that randomly selecting 300 of the 2,400 control patients would produce a valid comparison of outcomes."
Despite that "calculation," the comparison is invalid. One cannot simply compare ESRD patients to random patients. They would at least need to be matched for age, sex, co-morbidities other than ESRD and perhaps other variables to eliminate confounding.
It is possible that ESRD patients will have worse outcomes if they contract C. diff colitis. But this study doesn't prove that, and the story is misleading.
It's 2013. I agree with The Guardian's Observer column which says that Nate Silver's accurate predictions highlight "the importance of statistical literacy in our data-heavy age."