The J.D. Power and Associates 2009 National Pharmacy Study shows high average satisfaction at mail-order pharmacies. Yet the National Community Pharmacists Association (NCPA) issued a press release immediately after the J.D. Power results claiming Patient Survey Finds Widespread Problems With Mail Order Pharmacies.
Drug Topics magazine duly reported on the “controversy” of these supposedly “conflicting results.”
Now, I don't want to get off on a rant here, but it’s inaccurate and intellectually dishonest to compare the NCPA’s aggregation of “just over 400” independent pharmacy customers' viewpoints with J.D. Power’s statistically sound survey of 12,215 pharmacy customers. And I will gladly debate any pharmacy school professor who wants to defend the statistical reliability of NCPA’s methodology.
GETTING SOME SATISFACTION
Here are the average pharmacy satisfaction rankings rankings (out of 1,000 points) by pharmacy format from the J.D. Power survey. Click the chart to enlarge it. Note that brick-and-mortar pharmacies were evaluated using different, store-related factors than mail-order pharmacies. (See paragraph 6 of the J.D. Power press release for details.)
Two interesting findings:
- The mail-order pharmacies of the big 3 pharmacy benefit managers (PBMs)—CVS Caremark (CVS), Express Scripts (ESRX), and Medco Health Solutions (MHS)—all ranked slightly below the mail-order average.
- Two franchise chains of independent pharmacies—Health Mart (McKesson) and Medicine Shoppe (Cardinal Health)—ranked above the chain segment average.
Gosh, how did the combined efforts of J.D. Power and WilsonRx fail to detect the supposed "widespread problems" with mail order?
A quick look at the respective methodologies makes the answer crystal clear. Click the table to enlarge it.
The NCPA survey suffers from what statisticians call “bias”—the tendency for a survey’s findings to be unrepresentative of the whole population. Sources of bias include:
- Non-response bias: Only consumers who shop at independent pharmacies were sampled.
- Invalid sample size: “Just over 400” surveys collected from more than 23,000 NCPA member pharmacies
- Non-sampling error: Did the pharmacy owners encourage only disgruntled consumers to fill out the survey? Who decided which surveys got returned for analysis?
Look, I understand that some pharmacists don’t like mail-order because it hurts their business. But it’s insulting to the discipline of statistics to place NCPA’s survey side-by-side with the other two surveys.
Before you accuse me of bias, keep in mind that I am ranting about statistics, not about mail-order pharmacies. If you are still confused, please read Chapter 4 of What is a Survey?, a straightforward, non-technical booklet designed to improve survey literacy.
Of course, that's just my opinion. I could be wrong.