Here’s a problem with that new 6-step hand hygiene study

A 6-step alcohol-based hand hygiene technique is significantly superior to the standard 3-step technique in reducing bacteria colony counts.

So says a randomized trial with 78 nurse and 42 doctor participants recently published online in the journal Infection Control and Hospital Epidemiology. The full text of the study is available here.

At 42.5 seconds, the 6-step process took significantly longer than the 35 second 3-step.

Multiple media outlets, including the New York Times, published stories about this study.

The study was done properly. But after reading it, I had an issue.

How to get the answers you want from a survey

This isn't about religion or politics, two subjects I tend to avoid. This is about surveys and how they can mislead.

I received this survey in the mail last week. It is from CatholicVote.org and is touted as the "largest survey of Catholics ever conducted on the issue of ObamaCare."

CatholicVote.org promises that the results will "send a strong and clear message to every politician running for election or reelection in the 2014 midterm congressional elections, that the overwhelming majority of Catholic voters demand ObamaCare be repealed."

Judging from the way the questions are framed, I think the message will be clear.

Here are a few examples:

From Section B "ObamaCare's War on Christianity and Morality"

Question #2: Do you think ObamaCare is violating the Constitution's First Amendment protections for freedom of religion and freedom of conscience by forcing pro-life Americans to purchase health coverage that includes abortion inducing drugs?

A) Yes, this is certainly a violation of the Constitution's First Amendment protections.
B) No, this is not a violation of the Constitution
C) Not Sure
D) Other

Question #4: As a state lawmaker in Illinois, Barack Obama voted twice to deny lifesaving medical care to babies born in botched abortions. What is your reaction to this fact?

A) I support President Obama on this.
B) I am horrified and angered by this.
C) Not Sure
D) Other

From Section C "ObamaCare's War on Freedom"

Question #5: Do you think President Obama knew about the crushing cost of ObamaCare for families across America, and was just lying about the cost to get ObamaCare passed into law? Or do you think he shares our shock and dismay at the staggering cost of ObamaCare?

A) I believe President Obama knew about the crushing cost of ObamaCare for families across America, and was just lying about the cost to get ObamaCare passed into law.
B) I think he shares our shock at the staggering cost of ObamaCare and was just unaware of it.
C) Not Sure
D) Other

Question #6: How do you think the mass exodus of doctors from medicine will impact your ability to see a doctor and get the medical treatments you need?

A) A doctor shortage on this scale will certainly drive healthcare costs up dramatically and make it far more difficult for me to see a doctor and get the medical care I need.
B) I don't think we'll see much impact from this doctor shortage.
C) Not Sure
D) Other

Had enough?

I look forward to seeing the results.

Ultrasound selfies? How surveys can mislead

Do you believe that traditional hospitals will be obsolete in the future? A recent survey found that 57% of those polled believed that would happen.

The survey, sponsored by the Intel Corporation, involved 12,000 subjects from the United States and seven other countries around the world.

Here are some other revelations from that survey:

• 84% said they would be willing to share their personal health information to advance and lower costs in the health care system.
• 70% said they were receptive to using toilet sensors, prescription bottle sensors, and swallowed health monitors.
• 53% said they would trust a test they personally administered as much or more than if that same test was performed by a doctor
• 30% of people would trust themselves to perform their own ultrasound. 

That made me laugh. Ultrasonography is one of the most operator-dependent tests in use today. It is not easy to perform, nor is it easy to interpret.
I then began to wonder about the credibility of this survey. Before I retired, I practiced in a typical small town in the northeastern United States. Some patients googled me, and a few searched the Internet for information about their illnesses. But for the most part, it was a technologically unsophisticated population.

I just can't envision most of my patients wanting to share their personal health information, use toilet sensors, or trust tests they did at home. Do their own ultrasounds? Not likely. Many of them did not even know what medications they were on.

After rereading the article about the survey, it occurred to me that the sample may have been flawed.

This sentence stood out. "[The] Intel Health Innovation Barometer was conducted online by Penn Schoen Berland in Brazil, China, France, India, Indonesia, Italy, Japan and the United States." The key word is "online."

This reminded me of a famous survey conducted by a magazine called The Literary Digest, which polled 10 million people and had a response of 2.4 million just before the 1936 presidential election. The magazine had correctly called the previous four presidential winners.

The names of the 10 million people queried were drawn from lists of the magazine's subscribers, owners of automobiles, and those with telephones. The survey predicted a crushing defeat for President Franklin D. Roosevelt at the hands of the Republican nominee, Alf Landon.

Of course, the opposite occurred. Roosevelt won all but 8 electoral vote, a huge landslide. What went wrong? Unlike the prior years, 1936 was the middle of the Great Depression, and this time those who had enough money to subscribe to The Literary Digest, own cars, and have telephones were not a representative sample of those who voted.

Do you think maybe the 12,000 people polled online might not be a representative sample of the general population of the world?

I'm not expecting patients to do their own ultrasounds anytime soon. I think hospitals will be around for a while too.

The best general surgery residency programs for clinical training?

I've received a couple of emails from Doximity [A closed medical "community" of > 280,000 doctors] reminding me to complete a survey which they are sponsoring jointly with U.S. News & World Report. They are asking members, possibly only surgeons, to name the best general surgery training programs in the country.

Not mentioned in the email but stated at the beginning of the survey is that they want respondents to name the 5 best programs for clinical training.

I have a feeling that not everyone will notice the part about clinical training, and we will get a list of the usual suspects just as we do every year with the U.S. News best hospitals survey.

For several reasons, the survey is fundamentally flawed.

There are 240 general surgery residency programs in the country. Unless one is personally involved with a program, it is impossible to judge the competency of its graduates. How would I or anyone else who does not work there know whether residents training at UCLA or Baylor or Lehigh Valley are clinically competent?

There are no accepted ways to judge the clinical skills of any surgeon. Video recording of procedures with judging by peers can assess technical ability, and as shown in the recent New England Journal paper from Michigan, there is some correlation with outcomes.

The American Board of Surgery publishes first attempt board passage rates for all programs, but passing the boards does not necessarily equate to clinical skill.

Most surgeons have probably encountered only one or two graduates of any of surgical residency. Even if the ones we have seen were great, they may not represent the majority of graduates.

I'll bet I can name most of the top 5 programs right now. These are not necessarily the programs that produce the best clinically trained residents.

Here are my guesses: Massachusetts General, Johns Hopkins, Mayo Clinic-Rochester MN, New York Presbyterian-Columbia, Cleveland Clinic.

In the past, some institutions on my list were rumored to be terrible places to learn to perform surgery because the residents did a lot of watching and retracting but not much operating. Whether that is true today or was so in the past, I could not tell you.

I guarantee you that no community hospital will rank in the top 20 [maybe top 50] despite the fact that such hospitals produce many fine clinical surgeons.

I have no idea which programs produce the best clinically trained surgeons. After the Doximity-U.S. News survey results are published, you won't know either.

Cows or Sharks? Which are more likely to kill you?

Give me a minute, and I'll get to the cows and sharks.

You would be surprised at how few doctors are familiar with even the most basic statistics. Medical journal articles often have statistical errors which are missed by manuscript peer reviewers and readers alike.

Most medical students have taken a course in statistics, but it is usually taught in the first or second year of school. By the time they start residency training when they could really use the information, they have forgotten most of it. Statistics should be taught during the clinical years of medical school and reinforced throughout residency training.

Hospital administrators are even more clueless than physicians. I have blogged before (here and here) about the irrational responses of administrators to miniscule changes in poorly constructed surveys of patient satisfaction. When scores go down by insignificant percentages, all hell breaks loose with task forces, ad hoc committees and browbeating of staff.

Here’s a fun exercise involving statistics. It's OK. No formulas will be discussed.

Which animal kills more people per year in the United States, cows or great white sharks?

Although not long ago a German tourist was killed by a shark in Hawaiian waters, the answer is overwhelmingly "cows."

How can this be? You rarely hear about a cow killing a human but it happens about 20 times every year. Between 2003 and 2008, 108 people died from injuries caused by cattle across the United States, according to the Centers for Disease Control and Prevention. That's 27 times the whopping 4 people killed in shark attacks in the United States during the same time period, according to the International Shark Attack File.

Guess how many cows there are in the US.

According to the Drovers Cattle Network, there were 96.5 million head of cattle here as of mid-2013. The cattle population dwarfs the number of great white sharks. The New Ecologist estimates that the number of great whites in the entire world is about 3500.

The Guardian recently reported that there have been 1,085 recorded shark attacks in the US since the year 1670 for an average of only 3.5 shark attacks each year for the last 342 years.

Although not as dramatic or as newsworthy as a shark attack, it is far more likely that a person will be killed by a cow than a shark.

So keep your statistical radar turned on. Be skeptical.

And if you see an udder in the water, get to shore as fast as you can.

Televised live surgery: Would you want to be the patient?

"Defibrillation, defibrillation, where is the defibrillator? We did a mistake while talking," said a cardiologist when a patient went into ventricular fibrillation during a live broadcast of a percutaneous coronary intervention at a meeting in Europe last year.

Before it was recognized by the operators, the complication was identified by members of a panel and an audience who were watching the procedure remotely.

The Medscape article with a link to the video of the procedure is available here.

Live broadcasting of procedures—is it really educational; is it just self-promotion; is it marketing?

Several societies have published guidelines for live broadcasts of operations and procedures, and some papers have discussed the ethics of the practice. I won't get into those areas.

Here are some things to consider.

More problems with patient satisfaction surveys

Here are some updates on the patient satisfaction front.

A paper in last month's JAMA Surgery journal noted that patient satisfaction ratings have very little to do with the quality of care provided by a hospital.

The study analyzed data from 31 hospitals that were participated in patient satisfaction surveys, the CMS Surgical Care Improvement Project (SCIP) and employee safety attitudes questionnaires. 

They found that patient satisfaction did not correlate at all with the rates of hospital compliance with SCIP process measures or the opinions of employees about the culture of the institution for half of the categories questioned.

They concluded that "patient satisfaction may provide information about a hospital's ability to provide good service as a part of the patient experience; however, further study is needed before it is applied widely to surgeons as a quality indicator."

What about patient satisfaction and the quality of medical care provided by doctors? 

This is only an anecdote but it does say volumes about the subject.

A New York area cardiologist admitted to defrauding government and private insurers of $19 million. This was described as the largest healthcare scam by a single physician ever recorded in New York or New Jersey. 

Thousands of patients underwent unnecessary and possibly dangerous tests and treatments. He also employed unlicensed and unqualified personnel who treated patients.

As noted by Dan Diamond, managing editor of the Daily Briefing, the Healthgrades patient satisfaction scores for Dr. Katz all ranged from very good to excellent.

In fact, Dr. Katz has received not one, not two, but three Healthgrades Quality Awards, which are still in evidence on their website. I guess $19 million worth of fraud is not enough to impact one's Healthgrades ratings.

Although this next vignette is about customer satisfaction and has nothing to do with patients, it too illustrates the folly of basing one's opinion on satisfaction scores alone.

According to the Consumerist blog, an subsidiary of the magazine Consumer Reports, certain well-known companies have based employee pay raises and promotions on the results of customer satisfaction surveys.

Apparently, the companies considered anything less than a perfect "5" rating as failure. This resulted in employees telling patrons to either give them a "5" rating or if they could not do so, decline to take the survey. 

I have seen this phenomenon in hospitals too. Staff were coached about what to say to patients to help persuade them to give higher scores. 

I think it's called "gaming the system."

For lots more on the subject, type "patient satisfaction" in my blog's search field (upper right corner).

ADDENDUM 5/9/2013

A friend emailed me this comment: "When I take my car to the dealer for service, they tell me they will be sending me a survey in the mail. Then they tell me if I cant give them all '5's, I shouldn’t fill out the survey, instead I should call them and speak to the manager so they can do better next time"

Docs are not happy: Medscape’s survey of physicians

MedScape recently surveyed over 24,000 US physicians regarding their compensation. In general, incomes are down somewhat since the 2011 survey. Most news outlets focused on the finding that only 11% of doctors considered themselves “rich,” which was not defined. Many docs pointed out that although their incomes were high, they had many expenses and debts.

Not receiving any notice were some other interesting results.

Only 54% of the respondents said that they would choose medicine as a career if they had a chance to “do it all over again.” Think about that. How sad. Anecdotally, I know a lot of unhappy doctors, but for almost half to admit that they made a poor career choice is shocking. Only 41% would choose the same specialty, and a mere 23% would practice in the same setting.

Another outcome of note is that of satisfaction by specialty. With 64% saying they were satisfied, dermatologists led the list. Specialties which had fewer than 50% of their ranks expressing satisfaction were the core groups, family medicine, internal medicine, obstetrics/gynecology and my own specialty, general surgery. Fewer than half of the general surgeons would choose medicine as a career again or general surgery as a specialty.

These findings will likely not be cited by medical schools looking to recruit applicants, who apparently don’t read surveys like this anyway. It’s not very flattering for most specialties either. How bad is it when even one-third of the dermatologists aren’t satisfied? So-called “organized medicine” will ignore all of this as they have done for years. The government will continue to put the squeeze on.

I’ve said it before; the future of medicine is indeed bleak.

The quest for medical error transparency suffers setbacks

Two recent articles in American Medical News highlight the difficulties in moving toward a blameless culture in medicine.

First we have the results of an Agency for Healthcare Research and Quality survey of 600,000 staff from over 1100 hospitals. Half of them “believe their mistakes are held against them, and 54% said that when an adverse event is reported, ‘it feels like the person is being written up, not the problem.’" And two-thirds are concerned that records of errors are maintained in their personnel files. Despite lip service by medical thought leaders, the reported responses have not changed since the last AHRQ survey in 2007.

Much more alarming is the second amednews.com piece, which involves a family secretly recording a conversation with the chief medical officer [CMO] of an Ohio hospital.

Two days after knee surgery, a man died of cardiac arrest apparently secondary to an overlooked high serum potassium level. Before he died, the patient’s children met with the CMO, who had not personally been caring for the patient. Unaware that the conversation was being recorded, the CMO “made sympathetic and apologetic comments and admitted fault on the part of the hospital for Smith's condition, according to court records.”

The Ohio appellate court ruled that the secret recording was admissible and not protected by peer review privilege, as had been argued by the CMO’s lawyers. They had claimed that the CMO had learned of the error via the peer review process. Ohio law states that as long as one party consents, conversations can be recorded.

Thus we are left with pie-in-the-sky appeals for greater transparency and candor regarding medical errors [the so-called “Just Culture”] vs. a litigious society with stealth technical capabilites and a workplace atmosphere of fear and distrust.

Somehow I do not foresee a major change in the way doctors and hospital staffs approach this issue.

Survey finds med students view primary care and all specialties negatively

In a previous blog, I discussed some of the issues facing primary care. Specifically, I have pointed out that medical students simply do not want to go into primary care as a career. I even facetiously [although some thought I was serious] suggested that the way to lure students into primary care was to not provide them with clerkship experience.

In the January 2012 issue of the journal Family Medicine [full text available here], a survey of medical students’ attitudes toward primary care and specialties reveals some sobering information about primary care and the overall practice of medicine.

At three medical schools, Michigan, Michigan State and Brown, 1533 students were sent surveys during the years 2006-2008, with 983 [64%] responding, an excellent rate of return for a survey. My theory about exposure to primary care may be wrong because third and fourth year students were significantly more inclined to choose primary care as a career than first or second year students. But the overall percentage of students who said they would opt for primary care was only 14.8%. This is consistent with matching program data, which indicate that 14% of US medical graduates match in primary care.

The students had negative opinions about the work life of all physicians, but they were particularly down on primary care. Senior students were more negative about primary care than juniors [maybe my theory about exposing them to a primary care clerkship is right] and more positive about the work life of specialists. Despite the increased negativity of senior students, the clerkship in primary care did not really have much of an impact on the students’ career choices.

The paper has some limitations, most notably that the students were only polled once. It is well-known that many change their minds as they progress through school. Also, this study involved students from only three medical schools.

The authors conclude: “Our learners’ negativity about their future work lives reflects and portends a pessimistic culture of medicine. Student views of primary care work life are particularly negative, but some students indicate an interest in primary care despite negative perceptions.”

The authors feel that to attract more students into primary care, the work life of primary care doctors needs to be improved. However, they did not offer any suggestions on how this could be done. And the website FierceHealthcare reports that the government is trying to lure students into primary care. Other than mentioning a $877,000 grant to The University of Maryland Medical School for development of a “primary care track,” which to me seems somewhat short of innovative, the article does not describe any “lures.” They might have been better off giving the money directly to the students as a bribe.

Of even more concern is that the surveyed students perceived the work life of all physicians, primary care MDs and specialists alike, negatively. As I have stated before, I agree with the authors that the future of medicine is not bright.

Surgeons and alcohol use

“Prevalence of alcohol use disorders among American surgeons” appeared in the February, 2012 issue of Archives of Surgery.

A survey of 7197 surgeons, all members of the American College of Surgeons [ACS], had a 28.7% response rate and revealed that 15.4% had scores on an alcohol use assessment test that indicated abuse of or dependence on alcohol. This is consistent with the rate of such alcohol problems in the general public.

Factors significantly associated with alcohol abuse or dependence were age [younger surgeons had more problems], being in a relationship, burnout, depression, fewer hours worked, fewer call nights and not having children [I would have thought otherwise]. Women surgeons had twice the incidence of alcohol abuse or dependence than men, which was statistically significant, p = 0.001.

The most distressing result of this survey is that 77.7% of surgeons who had problems with alcohol said they had committed a major medical error within the three previous months. This is three times more often than those who reported no alcohol problems and statistically significant, p = 0.001.

The survey may not be that accurate because it had such a low response rate. In addition, the ACS had 65,844 members at the time of the survey but only 27,457 had provided the organization with e-mail addresses, 25,073 of which were valid. Although the rate of return of those with valid email addresses was 28.7%, the actual percentage of ACS members responding was 10.9% [7197/65,844]. There are also the usual problems of trying to sort out whether those who responded were more or less likely to have alcohol issues.

My opinion: It is not reassuring that surgeons abuse or are dependent on alcohol at the same rate as everyone else, nor is it acceptable. We should be held to a higher standard. Not one surgeon should take the responsibility of operating on a patient while dependent on alcohol. The rate of medical errors is chilling. Now that we have this information, what is going to be done about it?

Lying to patients: Part II of a two-part series

In Part I, we established that not only do physicians lie to patients; they lie about how frequently they lie to patients. So why do they lie to patients?

Apparently, it’s because they lie about other things too. Two recent papers illustrate the point.

A group from the M. D. Anderson Cancer Center in Houston reviewed 243 applications to their gynecologic oncology fellowship and found the following:

Applicants listed over 400 articles published but only 83% of these could be verified. And 30% of applicants who listed published papers had at least one unverifiable paper.

Hard to believe, but male gender was statistically significantly more likely to be associated with the deception.

The results reported in the above paper were remarkably similar to those found with applicants to a general surgery residency program in a paper from 2008.

The authors, from Duke University, looked at almost 500 applications to their program. They found that of 596 publications listed, 33% could not be verified. And of the 150 applicants who listed publications, 33% had one or more unverifiable publications.

In this paper, factors associated with unverifiable publications were older applicants and graduation from a foreign medical school.

If these papers are accurate [Can we believe them or anything else? After all, the two papers were written by doctors.], inflating one’s curriculum vitae is very common.

It is no wonder that doctors lie to patients.

Lying to patients: Part I of a two-part series

A survey just published in Health Affairs found that 20% of the nearly 2000 physician respondents said that “they had not fully disclosed mistakes to patients for fear of being sued” in the past year. When I first heard about this the other day, I tweeted the following, “And 80% lied.”

Now that I’ve had a chance to think about it, I still feel that way. Is it possible that 80% of the surveyed doctors did not make a single mistake over the course of a year? I would guess that many doctors who practice full time would have been likely to have made more than one error that led to significant harm to a patient. Do you think 80% of them owned up to it to the patient or his family?

I don’t think so. As much as it would be nice to live in an ideal world, we don’t. Despite research and assurances to the contrary, most doctors I know are not ready to confess their sins to patients because they do fear lawsuits. And that fear is well-founded. But it is unfortunate because in my limited experience [hard as it is to believe, I’ve made my share of mistakes], patients and families generally do take it well when you admit that you made a mistake and more importantly, say you are sorry.

One bright spot. I’m not surprised at this. The paper said, “General surgeons and pediatricians were most likely to completely agree about needing to disclose all serious medical errors to patients, while cardiologists and psychiatrists were least likely to report this attitude (p < 0:001).”

Having read the entire Health Affairs paper, which is entitled “Survey Shows That At Least Some Physicians Are Not Always Open or Honest with Patients,” I can point out another finding that did not receive enough attention. That is, 89% of physicians said that in the past year they never told a patient something that wasn’t true.

I am equally skeptical of that statement. Think about it. We have many opportunities to lie to patients. Many of them can at least be partially justified. In the so-called “informed consent” discussion, we don’t disclose every possible complication that has ever been associated with a procedure. If we did, no patient would ever consent to anything. What about injecting some local anesthesia and saying, “This won’t hurt. It’s just like a mosquito bite.” There are many more.

Many of the lies we tell patients are not intended to be malicious or deliberately deceiving, but they are still lies in the strict sense of the word. The bigger problem of disclosing errors in an honest and forthright way will happen more often when the punitive culture of medicine changes.

I do not expect to see that happen soon.

Coming up in Part II: Why do physicians lie?

Surgical Residents Surveyed; Concerns about Experience & Confidence

A recent survey of general surgery residents was published online by the Journal of the American College of Surgeons yesterday. More than 4200 residents who took the 2008 American Board of Surgery In-ServiceTraining Exam (ABSITE) responded. This accounts for 82% of all categorical (five-year) general surgery trainees. The authors of the paper represented the American Board of Surgery, Yale University and Memorial Sloan Kettering Cancer Center.

Compared to those in university programs, residents in community hospital programs had more positive responses to questions about satisfaction with their operative experience, didactic teaching and support from their programs.

But a significant number (27%) of all residents surveyed worried that they would not feel confident to perform surgery by themselves when they finished training. A similar number were not satisfied with their operative experience. Almost half of all residents were not satisfied with the level of didactic teaching being offered.

Something is wrong if over 25% of surgical residents are uncertain that they will be able to operate independently when they finish training. And just how does one identify those surgeons? Don’t say, “Check to see if they’re board-certified.” The boards don’t test operative skill.

Not long ago, I blogged about the coming shortage of general surgeons. The paper discussed above would indicate that expanding existing general surgery residency programs may not be the answer. If a quarter of all residents feel they are not getting enough operative experience and are not confident in their skills, how can programs be expanded?

Training Surgeons and Informed Consent

A rather bold new survey reveals that patients are willing to let residents operate as long as they operate on someone else. The study, published on line in Archives of Surgery, queried 316 patients at a tertiary care US Army hospital.

There were some positives. When given generic questions on resident participation in their surgery, 94% of all respondents said they would “consent to the involvement” of a resident in their operation. Of those respondents who had a preference, 91% felt that their care would be as good or better in a teaching vs. a non-teaching hospital. 

However, when presented with specific descriptions of the degree of resident participation in an operation, patients’ enthusiasm waned significantly. Just over half (57%) said they would consent to a junior resident assisting the staff surgeon; 32% would agree to having the staff surgeon assist the resident; 25% would consent to having the resident perform the surgery with the staff surgeon observing; only 18% would allow the resident to operate without the staff surgeon in the room. This survey clearly shows that if given a choice, most patients would not allow surgical residents to perform any part of their surgery.

This research is bold because it addresses a topic that most surgeons in teaching hospitals would rather not talk about.

You see, to learn how to be a surgeon, one must actually perform surgery. It can’t be learned by watching and regardless of what you may have read or heard, it can’t be learned on simulators alone.

With the pressure on to reduce work hours and alleged fatigue, surgical residents already are grappling with their levels of confidence upon graduation from training. The American College of Surgeons is so concerned about this that it is surveying its members to assess their opinions on whether today’s residents are adequately trained [Survey Regarding Competency and Confidence of Current General Surgical Trainees, link not available].

 

Since everyone likes the surgeons/pilots analogy, are you aware that co-pilots, some with far less experience than senior pilots, often are at the controls during take-offs and landings without the knowledge or consent of the passengers? The airline industry knows that flying is another skill that has to be learned by doing.

In the old days, surgeons learned by doing. Yes, we made some mistakes. And we still do. We are human. But how is the next generation of surgeons to be trained if they cannot operate under supervision?