Don't trust the abstract; read the whole paper

Above are the results and conclusion from an abstract of a paper called "Single-Incision Laparoscopic Cholecystectomy: Will It Succeed as the Future Leading Technique for Gallbladder Removal?" It appears online in the journal Surgical Laparoscopy, Endoscopy & Percutaneous Techniques.

It is another great example of why you need to read the entire paper and not just the abstract.

The methods section says that the study involved 875 patients with prospectively collected data. Don’t be fooled by prospectively collected data. This research is retrospective.

Here are some issues.

Antibiotics instead of surgery for appendicitis? I don’t think so.

Like the villain in a bad horror movie, the idea of treating appendicitis with antibiotics refuses to die.

A meta-analysis published yesterday on line in the British Medical Journal claims that treating uncomplicated appendicitis with antibiotics is better than surgical appendectomy. Four studies involving 900 patients were included in the paper. [Full text here.]

Treatment with antibiotics was said to have resulted in fewer complications [relative risk reduction of 31%] and similar hospital lengths of stay of just over 3 days for each group.

Medical news outlets such as MedPage Today “Antibiotics May Be Enough for Appendicitis,”  Eurekalert “Antibiotics a safe and viable alternative to surgery for uncomplicated appendicitis, say experts” and the BBC “Appendicitis: Antibiotics may be better than surgery” touted the study without much criticism. That will not be the case here.

The authors state, “Diagnosis of acute appendicitis at admission was confirmed by ultrasonography in one study and by computed tomography in two studies, although this was done only in some patients in the study by Hansson et al.” In the fourth study, the diagnosis was based on clinical factors only. Translation: An unknown number of patients in the group treated with antibiotics may not have actually had appendicitis.

There were issues concerning the methods of randomization in the four studies. The paper says, “Randomisation methods were reported as computer generated, external randomization [not explained, my comment], and by date of birth. The randomisation method was not clear in one study.” Date of birth is a notoriously poor way to randomize subjects in a study because the treating physicians can know which group the subject is assigned to before entering him in the experiment. This means that two of the four studies had questionable randomization schemes.

In one of the four studies, almost half of the patients [96/202] in the antibiotic group required appendectomy. And 20% of all patients treated with antibiotics required appendectomy within one year of entry into their respective studies.

The mean length of stay for both treatments was just over 3 days for both antibiotics and surgery. Maybe that is true in Europe where these four studies were done. Only one of those four studies included patients with complicated appendicitis. But here in the US, the median length of stay for almost 17,000 appendectomy patients with uncomplicated appendicitis is 1 day [Advani et al, Am J Surg]. In my personal series of 171 appendectomies during 2009-2011, the mean length of stay for all patients, including those with complicated appendicitis, was 2.4 days with a median of 1 day.

The complication rate comparison raises a “straw man.” The 25% rate of complications for the appendectomy group in the meta-analysis is more than twice that quoted in the series by Advani. Laparoscopic appendectomy results in fewer complications than open appendectomy. One of the studies used in the meta-analysis is from 1995, when very few laparoscopic appendectomies were being done.

The success rates for the two treatments are compared. According to the paper’s Table 2, 58.3% patients were successfully treated with antibiotics vs 92.6% successfully treated with surgery. Unsuccessful surgical treatment is defined as removal of a normal appendix.

The recurrence rate of appendicitis in those who underwent surgery was 0.

If I proposed treating you with drugs that had a 58.3% rate of success in curing your illness for one year with the possibility that you could still suffer another attack of the illness two or twenty years later, would you choose that treatment? Or would you opt for a treatment which would keep you in the hospital for less than 24 hours with no risk of recurrence of the problem?

Fun with Statistics: Straw Man, Hawthorne Effect & Power Debunk Cholangiogram Study

Major bile duct injuries can be prevented by implementation of routine intraoperative cholangiography [an x-ray of the bile ducts] say the authors of a paper published in the August 2011 issue of the Journal of the American College of Surgeons. The Dutch researchers established a policy of routine intraoperative cholangiography during laparoscopic cholecystectomy and looked at the incidence of bile duct injuries three years before and three years after it was instituted. Selective [at the discretion of the surgeon] intraoperative cholangiography was performed in 421 patients and routine intraoperative cholangiography was to have been in 435. Bile duct injury occurred in 1.9% of patients before the routine use of intraoperative cholangiography and in no patients after. The difference was statistically significant, p = 0.004.

Therefore, everyone having a laparoscopic cholecystectomy should undergo routine intraoperative cholangiography, right?

Not so fast. There are a few problems with the study. Let’s take the “Straw Man” issue first. A “Straw Man” is the establishing of a false premise and then defeating it with an argument. I have blogged about this before [here, here, and here]. The “Straw Man” in this case is the bile duct injury rate of 1.9% in the pre-routine intraoperative cholangiography cohort. Many  large series of laparoscopic cholecystectomies report rates of bile duct injury of well under 1%. An Egyptian study of 2,714 laparoscopic cholecystectomies found only 5 [0.18%] bile duct injuries. A Swiss study of 31,000 laparoscopic cholecystectomy patients noted a similar rate of bile duct injury of 0.3%. A third study reviewed 234,220 laparoscopic cholecystectomies done in Florida over a 10-year period and found that 0.25% resulted in a bile duct injury.

The next problem is called the “Hawthorne Effect,” which is the well-known finding that behavior improves when subjects know they are being watched. It is named for a factory near Chicago where several such experiments were done 80 years ago. Workers’ productivity increased no matter what changes were made in their environment. The surgeons in the routine intraoperative cholangiography study were given extra training in a skills laboratory and were aware that their performance was being monitored.

Despite the policy, only 59.8% of the patients in the routine intraoperative cholangiography group actually underwent routine intraoperative cholangiography. Even at the end of the three years, more than 23% of patients were still not undergoing routine intraoperative cholangiography. This suggests that the surgeons themselves were not totally convinced that the procedure was worth the extra time involved to complete the x-ray.

Another problem that is true of all so-called “before-and-after” studies is the fact that the “after” group has the benefit of the surgeons becoming more proficient simply because they have been performing the procedure longer.

The correct way to perform this investigation would be to randomize patients with one group having mandatory cholangiography and compare them to patients randomized to not have cholangiography. Such a study would be very difficult to do because the incidence of bile duct injury is so small. In order to achieve adequate statistical power, one would need more than 1000 patients in each group. 

Meanwhile, I will continue to perform selective intraoperative cholangiography.

Is Robotic Surgery All Hype and No Substance

I am unable to contain myself for another minute. Let’s talk about robotic surgery.

This is from How Stuff Works, a Discovery Company, but it could have been written by the makers of the surgical robot:

“Most surgeries require nearly a dozen people in the room. As with all automation, surgical robots will eventually eliminate the need for some personnel.”

This is another example of a “straw man.” [Establish a false premise and defeat it with your argument.] Just about all operations require five people—surgeon, assistant, scrub nurse, circulating nurse, anesthesiologist. Some complex procedures might require another assistant. A dozen people might be needed for separation of Siamese twins or bilateral leg transplants.

There are many theoretical advantages of robotic surgery over conventional laparoscopy such as elimination of hand tremors, availability of more flexible instruments, more precise movements, easier learning and execution of intra-corporeal suturing techniques and possibly others. Disadvantages include the costs associated with robotic technology with the robotic itself selling for $1-2 million, yearly service contracts of >$300,000 and the high cost of the specialized instruments. Also, robotic surgery takes much longer than conventional surgery.

The problem with robotic surgery is that it has never been shown to improve patient outcomes for any procedure. Let’s look at the literature. The review articles cited below are almost exclusively based on non-randomized studies.

Laparoscopic Cholecystectomy. A review by the noted Cochrane Group involving five studies and 453 patients showed no differences in any outcome measure when comparing robotic surgery to conventional laparoscopic surgery.

Esophageal Reflux Disease. A review of 11 papers comparing standard laparoscopic anti-reflux surgery to Robot-Assisted Laparoscopic Fundoplication [acronym “RALF”] in 533 patients showed no differences in peri-operative complication rates or length of hospital stay. The robotic procedure took significantly longer. Another recent paper demonstrates the lack of quality research on this topic.

Colorectal Surgery. A review of 17 studies, one of which was randomized and controlled, showed no difference in the rates of complications and cancer outcomes. Robotic procedures took longer an cost more than conventional laparoscopic colon surgery. Despite the results, the authors felt that “Robotic colorectal surgery is a promising field and may provide a powerful additional tool…”

Gynecologic Surgery. A review of 22 non-randomized studies found that robotic surgery resulted in less blood loss [statistically significant but not clinically significant differences] and shorter hospital stays but no differences in overall complication rates when compared to conventional laparoscopic or open surgery. The authors commented that the methods used in the papers reviewed were poor and better studies are needed before concluding that robotic surgery offered any true advantages.

Abdominal Surgery. A paper from 2010 looked at 31 studies of nine different abdominal operations [robotic vs. conventional laparoscopic], 6 of which were randomized, controlled trials [RCTs]. The total number of patients included in all the studies was 2166. The number of patients who were participants in RCTs was 230. No RCT involved more than 50 patients. Not surprising was that the results were mixed with robotic surgery offering no clear advantage. These authors also called for larger and better designed studies.

Prostate Cancer. To date, there are no good RCTs comparing robotic to open or standard laparoscopic prostatectomy. This quote from a recent review of the literature on prostate cancer surgery says it all:

“Robotic prostatectomy is definitely here to stay and although a randomized, controlled trial comparing the open to robotic techniques would be ideal, it is clear that this is unlikely to occur.”


UPDATE [9/17/2011]
Robotic hysterectomy outcomes are no better than standard procedures reports a new study. It also suggests the procedures are driven by marketing.


UPDATE [12/29/2011]
Add gastrectomy to the list of procedures that do not show better outcomes when done robotically. 

UPDATE [1/3/2012]
ENT surgeon, @FauquierENT, dismisses robotic surgery for sleep apnea as "way overkill akin to using a $50,000 sniper rifle to kill an ant on the wall."

The issue may already be settled. According to the New York Times, patients are voting with their feet, preferring to have robotic surgery when it is available. This appears to be true in Wisconsin as well. A recent paper reveals that when hospital purchase a robot, their volume of prostate cancer surgery doubles.

Questions. Can anything be done about this? Should Medicare and private insurance companies pay for expensive, unproven treatments?

Robotic Single-Port Cholecystectomy Fails to Convince

I have resisted the urge to blog again about single-port [a.k.a. single-incision] laparoscopic cholecystectomy because I was afraid I would start swearing a lot, but I can contain myself no longer. The stakes have escalated with the appearance of a publish-ahead-of-print paper on robotic single-port laparoscopic cholecystectomy in Archives of Surgery.

One of the co-authors of this paper is the chief medical advisor for the company that makes the robotic surgery device and the other is a consultant. The paper is a pilot study of the feasibility of using the robot to perform single-port laparoscopic cholecystectomy. It attempts to retrospectively compare 10 patients done robotically to 10 patients who had standard, four-port laparoscopic cholecystectomies. Patients with acute cholecystitis were excluded from both groups.

One robotic case had to be converted first to a four-port laparoscopic approach and then to an open cholecystectomy. This patient was excluded from the analysis of the results, a flagrant violation of the “intention to treat” principle which is that all patients should remain in whatever group they originally were assigned to.

The robotic cases [minus, of course, the case that was converted] averaged 105 minutes in duration which was not significantly different than the four-port cases, which averaged 106 minutes. The authors concluded that this showed that the duration of robotic single-port laparoscopic cholecystectomy was equivalent to four-port laparoscopic cholecystectomy. Sounds great, right? Not really. This is a nice example of what is known in the statistics business as a “straw man.” A straw man is defined as establishing a control that is not representative of real life and then comparing the experimental group [favorably] to it. Most four-port laparoscopic cholecystectomies for non-inflamed gallbladders can be done in well under 60 minutes. For example, another recent paper comparing non-robotic single-port laparoscopic cholecystectomy to standard, four-port laparoscopic cholecystectomy noted mean case durations of 88 and 45 minutes respectively.

Of the nine analyzed robotic patients, two suffered urinary retention, one of whom had to be discharged with a catheter. This is a very rare complication of four-port surgery.

The authors state, “This approach [robotic single-port] appears to be safe, even in difficult cases with inflammation, and has a high degree of satisfaction with the patients.” It is difficult to see how they arrived at that conclusion since patients with inflammation were specifically excluded from the study. Patient satisfaction was only briefly discussed and not compared to the standard surgery group.

Not mentioned in the paper is the cost of the robot which is at least $1.3 million plus yearly maintenance fees of hundreds of thousands of dollars. All of this is being advocated without any evidence that the robot is safer or more efficacious for any type of surgery, let alone standard laparoscopic cholecystectomy. And it is being promoted to avoid three 5 mm incisions which in most patients are not painful and barely visible if at all. 

All I can say is "Klaatu barada nikto," which some have interpreted to mean “This escalation is unnecessary.”

The “Straw Man” Is Back

A rather breathless posting on Science Daily today extols the virtues of the “scarless” or single incision laparoscopic cholecystectomy compared to the standard four small incision technique. Single incision, or laparoendoscopic single-site surgery [abbreviated LESS (a catchy acronym is mandatory)], utilizes one incision in the navel through which the entire surgical dissection and removal of the gallbladder are done. LESS cannot usually be done when the surgery is for an acute gallbladder attack or if the patient has had previous upper abdominal surgery. The study was done at Mt. Sinai Hospital in New York.

According to the article “The Mt. Sinai group did find two advantages to the LESS procedure: these patients required less pain medicine after the operation than their counterparts who had the traditional minimally invasive operation; and LESS patients typically reported higher satisfaction scores: —4.7 on a scale of 1 to 5 (5 equals highest score) versus 3.6 for the conventional laparoscopic surgery group.”

Available in the abstract of the paper but not reported by Science Daily were the following: the study was retrospective and involved only 26 LESS patients and 50 conventional laparoscopic cholecystectomy patients; 31% of the LESS patients required additional incisions; the average age of the LESS patients was significantly younger than the conventional group [37 vs. 49 years respectively]; follow-up data were unavailable for over half of the conventional group.

The Science Daily piece quotes one of the authors. "What's really exciting is how these patients would recommend the procedure to a friend or family member," Dr. Chin said. "Seventy-four percent of the patients who had the single-incision operation would strongly recommend the procedure to someone else versus 36 percent of those who had laparoscopic surgery."

Here is where the “straw man” is introduced. A “straw man” is defined [see The Skeptic's Dictionary] as creating a fallacious argument and then refuting it with one’s own position. If you believe this article, only 36% of those who had standard four-incision laparoscopic surgery would recommend it to someone else. However, in the early days of laparoscopic cholecystectomy, papers reported patient satisfaction rates of 94-95% after conventional laparoscopic cholecystectomy.

Patients in both groups had obviously undergone only one of the two procedures making the recommendation data rather difficult to interpret. If 64% of patients who had undergone conventional laparoscopic cholecystectomy would not recommend it to someone else, what then would they recommend? Keep your gallbladder despite the pain? Old fashioned large incision open surgery? Suicide?

The straw man is an old friend. It’s good to see that he is still around.