MANAGED CARE March 1998. ©1998 Stezzi Communications
SPEAKING THEIR LANGUAGE
There are many reasons to develop health plan report cards. Just don't expect most beneficiaries to make use of them.
Over the past few years, considerable effort has been devoted to developing health plan report cards. Policy makers believe that if consumers are given detailed information about the quality of health care plans available to them, they will choose an "above average" plan and reject a "below average" plan. That won't happen, or at least won't happen as often as policy makers naively believe. It won't happen because the concept of HMO report cards for consumers actually fails to take into account the way they think and behave.
It won't happen because people don't always make decisions rationally. Rational researchers apparently believe that consumers make decisions the same way that they do — on the basis of objective information. In fact, most people do not rely on objective information to decide what to buy.
How consumers choose
A 1996 survey done for the federal Agency for Health Care Policy and Research found that most people relied on personal recommendations from doctors, family members and friends to make health care decisions. Family and friends were seen as good sources of information about health plans because they shared common concerns, while employers were seen as biased because of their interest in saving money.
While more than 80 percent who saw a report card thought it would be useful to someone trying to make decisions about health plans, only about one third have actually used the information to make decisions.
In its present form, the report card doesn't seem to have much effect on how consumers choose health plans, doctors or hospitals. The relationship between a physician and a patient is intimate, in some ways more intimate than any other relationship in a person's life. It shouldn't be surprising that consumers rely on family and friends for their opinions about physicians. Conversations with family and friends can address important issues that aren't included in an objective statistical analysis. Most people don't want to look at 15 pages of graphs that are overwhelming, contradictory and don't meet their needs.
Why is this the case? Research in human development shows that people can be concrete or abstract thinkers. Concrete thinkers tend to be concerned with immediate events in their lives; abstract thinkers are oriented toward the future. A concrete thinker might not be very interested in preventive medicine, since the problem isn't happening right now. An abstract thinker will look into the future, consider potential health care problems and take preventive action.
Concrete thinkers will rely on subjective personal experiences, while abstract thinkers will rely on objective, research-based information.
The "average" American has about 12 years of formal education and little experience reading and interpreting complex statistical reports. Most people do not understand what an HMO report card means when it says that "Differences are statistically significant" because they haven't taken an undergraduate course in statistics, and a person can go all the way through college without taking a course in statistics. Many well-educated people don't understand that term. And what of those who didn't go to college?
While statistical significance means everything to researchers, it means nothing to the average consumer.
How do researchers think?
People don't think like researchers. A typical report card for consumers will ask a question such as: "All things considered, how satisfied are you with your current HMO?" Their choices of dissatisfied, neither satisfied nor dissatisfied, somewhat satisfied and very or completely satisfied. Each plan is given a bar graph showing the percentage of responses for each choice, so that readers can compare plans with each other and with a state HMO average. Plan A may show that 55 percent of respondents were very or completely satisfied, while Plan B shows 49 percent, with the state average being 53 percent.
Do those differences mean anything? At the bottom of the page is the obligatory research disclaimer that "Small percentage differences may represent measurement (sampling) error rather than actual differences in HMO performance." How small is a small percentage difference?
While researchers understand sampling error, the general public certainly does not. So readers are faced with this apparent contradiction: On one hand, the report shows percentage differences between plans, but on the other hand, the report says that small differences may be due to sampling error — not differences in HMO performance. Is a conclusion even possible? What's a reader to do? And how readable are the report cards?
Card #1: You and Your Health Plan (1995) Minnesota Health Data Institute, St. Paul. This card scores high on cognitive terms, and is optimistic in its endorsement of groups and concepts. It emphasizes abstract ideas, a sense of difference from the norm, numerical terms and a present concern for social tasks and activities, which refers to ease of such actions as following instructions or calculating values. The card de-emphasizes change and does not reflect a concern for matters affecting people's daily lives, showing a low value for "human interest."
These characteristics are not surprising, given the need for consumers to analyze more than 50 charts, understand abstract concepts and determine which plan is the best.
Card #2: Comparing the Quality of Maryland HMOs: A Guide for Consumers (1997) Maryland Health Care Access and Cost Commission, Baltimore. Written at a more readable level than Card #1, this one emphasizes cognitive terms, present concern for social and task activities, numerical terms (23 graphs), a sense of difference from the norm and general statements instead of overstated, precise language.
Card #3: New Jersey HMOs: Performance Report (1997) New Jersey Department of Health and Senior Services, Trenton. This one is similar to the Maryland card in layout and presentation (24 graphs). It emphasizes satisfaction and a present concern for social and task activities. While this card generally endorses groups and concepts, it was written with a somewhat authoritarian perspective. It de-emphasizes action and shows a tendency for general, universal statements.
Too much data
The amount of statistical information can be mind-numbing. A single question (four alternatives, 15 health plans plus a state average) will have 64 numbers. If the report card has only 10 questions, there will be 640 numbers to consider. Since HMO report cards usually don't tell people how to read and interpret the statistics, readers are left to find their own way through page after page of graphs.
What are they to do if their existing plan is above average on five questions, average on three and below average on two? How does that compare to another plan that is above average on three, average on four and below average on three? What strategies should a reader use to compare 15 health care plans? Maryland included a worksheet with its report card on 15 HMOs. The worksheet requires readers to determine which HMOs:
1) are available where they live;
2) offer the benefits they want (by reviewing benefit information from employers and HMOs, although it doesn't say what to do if some plans offer some of the benefits desired);
3) are affordable based on cost information from one's employer or HMO;
4) include the preferred doctor, hospital or other provider by reviewing the HMOs' physician directories and telephoning the customer service departments, and
5) scored well on quality in terms of
a) customer service,
b) providing patient care,
c) helping to keep people healthy,
d) caring for the sick, and
e) accreditation.
Getting answers simple enough to put an "X" in a box will be difficult. "This has to be the most complicated product or service in our economy," Alain Enthoven, Stanford University professor and managed care advocate, has said. "A health insurance contract is extremely complex. Anybody who says they understand their health insurance doesn't understand the problem."
For HMO report cards that include a worksheet, both concrete (knowledge) and abstract (evaluation) thinking are needed. The usefulness of the card will be based largely on the thinking skills of the reader, since the card does not offer any strategies for evaluating the worksheet other than looking at it and making a decision.
For report cards that do not include a worksheet, the same kinds of thinking are required, but will include only knowledge and comprehension. The rest is up to the reader to figure out.
Too much time, effort
For many people, the time and effort involved in filling out the worksheet will probably result in it being thrown away. At least the IRS estimates how long it takes to fill out tax returns.
Are consumers willing to spend many hours trying to decide which HMO is best — this year? How often will the report cards come out? HMOs can change, and an organization with an average customer service rating in 1997 could be above average in 1999. The past does not always predict the future. Signing up with a plan that had a good record at the time of the survey does not guarantee that the plan will have a good record one or two years later. If thousands of consumers shift to an above average plan, that plan may not be able to absorb the rapid influx of patients. An above average plan can become average or worse fairly quickly.
If a reader conscientiously fills in the selection form — then what? There will probably be a grid of checkmarks, but little or no instruction on how to interpret the grid. No points were assigned for any of the categories, so the reader is left to make an intuitive decision as to which health plan is really best.
No wonder patients make decisions based on discussion with doctors, family and friends.