Patient satisfaction surveys are flawed in many ways. Here are just a few.
Sampling is a huge problem. A description of why sampling is an issue can be found here. It’s a bit complex. To summarize, the validity of a survey is strongly related to the size of the sample and the rate of response of the survey. If you have a patient base of 1000 and elect to survey 500 of them and receive responses from 100, the sample is really only 10% [100/1000] of the population in question. This would result in an accuracy of 95% ± 20%. See chart.
Most patient satisfaction surveys include far fewer than 50% of the population in question and have even lower response rates than 20%.
Thankfully, I no longer directly participate in the quarterly hysteria that occurs when Press Ganey scores are received by hospitals. Press Ganey is a company that is hired by hospitals to perform patient satisfaction surveying. They send out small numbers of questionnaires and have a very low rate of response. In addition, they use only a five point scale as a basis for their ratings and report the results as percentiles. [Note: I am not a statistician, but I don’t think it is kosher to report a five point scale in percentiles ranging from 1 to 100.] Usually there are modest up and down variations in these scores which are almost never statistically significant, especially when you consider the margin of error of well over ± 20%. Upon receipt of lower scores, task forces are established, multiple meetings are held, policies are changed and staffs are browbeaten. Many times the scores improve on the next cycle and the task force is congratulated. Lost in the euphoria is the fact that there is a three-month lag between the institution of any policy changes and the receipt of the next group of survey responses. In other words, the policy changes probably were not the cause of the uptick in the scores.
Note that the Medicare Hospital Consumer Assessment of Healthcare Providers and Systems [HCAHPS] survey suffers from many of the ills of Press Ganey, such as small sample size, poor response rates and way too many questions.
Other issues with patient satisfaction scores include the following:
There is no correlation between patient satisfaction scores and complaints.
Surveys are more reliable if they are completed as close to the time of the encounter as possible. Most are not done that way.
They do not necessarily correlate with quality of care as is shown in papers involving medical patients and patients with heart attacks. Other thoughtful essays on this topic can be found here and here.
No doubt the facts will not deter the bean counters from mandating that all physicians survey patients for satisfaction no matter how meaningless the data may be. American Medical News recently reported that the AMA and Press Ganey will be happy to help you with this for a mere $65.00 per month for AMA members and $85.00 per month for non-members.
I suggest you start your own patient satisfaction surveying company. As for me, I am happy that I work for a hospital.