Think about kindergarten. Think about Little Johnny and the day he toppled a desk, heaved an alphabet block out the window and then threw a tantrum when his teacher told him to stop. In those days, Little Johnny was not described as just an excitable boy. He was a bad boy -- Johnny Rotten.
Society has changed since you were in kindergarten. "Bad" and, to a much lesser extent, "good" are considered labels to be avoided; they are generalizations that confuse, rather than enlighten. Little Johnny may be socially dysfunctional or emotionally maladjusted or educationally challenged. He is not bad.
Health plans that miss the mark in some way -- and there are nearly as many ways to define that as there are health plans -- are out there. But just try to find anyone who's willing to say a "bad" health plan exists.
A spokeswoman with a large health care consulting company won't go near the subject. "This is not good for us to comment on," she says. "We only publish the ratings of the good ones. You can understand the implications of us even mentioning that there are bad ones." And health plans that have received adverse publicity or have been denied accreditation by the National Committee for Quality Assurance (to name a couple of factors that can influence public perception of plan quality) are not lining up to ponder, publicly, just how they miss the mark. "We don't participate in industry stories -- particularly now," says a spokeswoman for Oxford Health Plans.
Most health care experts say trying to determine which HMOs miss the mark depends a lot on the evaluator's perspective.
Good and bad, after all, are Platonic terms. Medical science is an Aristotelian enterprise and the empirical evidence needed to designate good or bad HMOs does not exist, experts say.
Kathleen C. Mudd, NCQA's vice president for accreditation, says her organization simply rates the plans according to its standards and leaves the labeling to others.
"If you've seen one health plan, you've seen one health plan," says Mudd. "It's very difficult to make generalizations."
In many industries, when a failing company is taken over, new owners are often eager to point to the managerial shortcomings of their predecessors. CareNetwork, a Milwaukee HMO denied NCQA accreditation last year, struggled before it was purchased by Humana in 1994, says William Carr, executive director for Humana's Wisconsin market. Humana is due for another NCQA review next year. In the meantime, Carr says, Humana has been pouring resources into quality improvement activities. Ask Carr if CareNetwork, prior to 1994, was a bad HMO and he fires back, "Not at all. We don't acquire bad HMOs."
CareNetwork lacked the resources to do all the things a plan needs to do to get accredited, he says. "NCQA accreditation isn't necessarily an indication of the care that goes on in a plan. It's one of several measures that can be used by buyers. CareNetwork was immature in terms of measuring everything that it needed to measure."
Health plans that are denied NCQA accreditation may not be assumed to be bad, but being shut out can certainly make its employees feel bad, says one nurse for an Oregon health plan. Neither the nurse nor the health plan, which was slapped with a denial in 1996, wanted to be identified. The plan recently went through NCQA's review process again and the nurse says everyone involved is anxiously awaiting the results.
"There was a huge effort here to get that accreditation," he says, adding that he believes that the plan has provided high-quality care all along. "We had not understood what the NCQA wanted health plans to do for accreditation. The denial had to do with our documentation procedures."
More than half the nation's HMOs, covering 75 percent of all managed care enrollees, participate in NCQA's accreditation program, which was launched in 1991. Alan Hillman, M.D., M.B.A., director of the Center for Health Policy at the University of Pennsylvania's Leonard Davis Institute of Health Economics, says NCQA and the Joint Commission on Accreditation of Healthcare Organizations operate the two most respected rating systems.
But ranking plans has become a national sport, with many consulting companies participating. Rankings by the private sector should be viewed with skepticism, says Hillman: "Consultants have a vested interest in ranking HMOs their employers pay them to rank." Honest consultants, of course, give their clients the bad as well as the good news. Still, there is always the temptation to tell the customer what he wants to hear. The most objective views, according to Hillman, are offered by NCQA and JCAHO. "They have been set up in a way that's completely independent," Hillman says.
In a series of articles appearing in May in the New England Journal of Medicine, Robert Kuttner noted his concern that there might be "a conflict of interest in the way NCQA is financed. Half of its budget comes from accreditation fees, which may create an incentive against setting standards too high. Critics of NCQA and [its Health Plan Employer Data and Information Set] say accreditation and quality should be lodged in a body not beholden to industry."
Kuttner, who is co-editor of American Prospect, ominously titled his article "Must Good Health Plans Go Bad?" In it, he questions whether NCQA can evaluate plans in a way that is relevant to the average subscriber. "A plan can win accreditation by NCQA and still not be an HMO one would choose for oneself and one's family," he writes.
The article, initially at least, promises to cut to the chase. For one thing, Kuttner uses the words "good" and "bad" -- something of a departure for anyone who fancies himself a health care analyst. But he doesn't name HMOs that are worthy, so to speak, of being labeled "bad."
Anecdotally, at least, the reverse of Kuttner's contention that some bad plans get NCQA accreditation is true -- some very good plans are denied. The January 1997 issue of this magazine featured a story about the Albany, N.Y.-based Capital District Physicians' Health Plan under the title, "The Most Satisfying HMO: What It's Doing Right." Ironically, the plan had been denied NCQA accreditation three years earlier, which CDPHP officials say was due to inadequate documentation, not the quality of care.
CDPHP received full NCQA accreditation in April 1997, points out Dan Colacino, the HMO's vice president of marketing. "As far as our members were concerned, we were the same plan the day before approval that we were the day after. The trouble with NCQA accreditation is that it's a pass/fail system. If you're just below, you're a plan that fails, even though you do a lot of things well. If you're just above, you're a plan that passes, even though you need improvement in areas."
David Shulkin, M.D., chief medical officer and chief quality officer for the University of Pennsylvania Health System, thinks it's impossible to "label a health plan good or bad" using existing evaluative systems. "The NCQA accreditation system is the brand name out there but it evaluates a health plan from a very limited perspective." So do all other rating systems, private or public, he says. "In the absence of any information, these systems are helpful, but I think they have a long way to go until they are useful evaluative tools," says Shulkin.
More to the point, who cares? Not patients, Shulkin contends. "People pay very little attention to these types of rating systems. What really matters most is their personal experiences with health plans. There are no such things as great health plans or really bad ones. Health plans, even the poorer quality ones, do some things really well."
Shulkin says that of the two extremes, finding good plans is a less quixotic endeavor than finding bad ones -- and in that sense, systems that are attempting to do that have taken the correct approach.
NCQA's Mudd agrees to some extent. "If we say a plan is doing well, it's doing well overall," says Mudd. "It's more ambiguous on the other end of the scale."
Finding a good plan may be a matter of finding a big plan, many in the industry contend. But that doesn't mean small plans are bad. It means they just have to work harder.
"We have to agree that if you own the marketplace, you're doing something right," says Walter Lane, M.D., president of Tampa Bay Doctors, an independent practice association in Florida. "It's regrettable that we've come to that, but that is our system. The plan that does the best job overall is probably going to be the plan that has the largest number of enrollees. The largest plan in the community is meeting more of the population's perceived needs than the one that's coming in last."
Newer plans might also be at a disadvantage, something the NCQA tries to rectify with an accreditation process specifically geared to plans 30 or fewer months old. NCQA's Accreditation for New Health Plans program, launched last October, exempts fledgling plans from the full-accreditation requirement of having to demonstrate clinical programs that are relevant to their memberships. Four plans are scheduled to go through the process this year; five next year.
Survival of the fittest rules the marketplace. But should it rule medicine? Is there an inherent danger in equating big health plans with good health plans? United Health Plan, a 100,000-member HMO in Inglewood, Calif., was denied NCQA accreditation in 1994. A couple of years later, new management made improvements and a name change. United Health Plan is no more; in its place is UHP Healthcare. Rodney Armstead, M.D., executive vice president, is one of the new managers trying to turn things around.
"We are a much improved organization," says Armstead. "You can see it in our corporate culture and in what the board is requiring." Does that mean United Health Plan was a bad HMO? "Absolutely not," he replies. "The fact that a plan goes through a process and misses a mark doesn't mean it's not providing quality care. A lot of plans can't go through the accreditation process because they can't afford to." Armstead also suggests that accreditation processes do not always take into account the different cultures served by different HMOs. For instance, 70 percent of UHP's members are enrolled in Medicaid. The other 30 percent are divided between Medicare recipients and commercial small businesses in ethnic neighborhoods. "Small plans that take care of more vulnerable populations can get left behind if the NCQA continues to make accreditation more rigorous, absent any recognition of those variables," says Armstead, adding that UHP received a three-year network accreditation from JCAHO last year.
Peter Kongstvedt, M.D., national practice leader for managed care strategy with Ernst & Young, the consulting firm, says logic dictates that "fifty percent of the plans are going to be below the halfway mark."
CDPHP's Colacino draws on his many years of experience in saying that there is a wide range of performance among health plans. "I was in the Blues system for 17 years and there were plans in that system that just hadn't progressed much from what they were doing in the 1960s."
NCQA officials acknowledge a range of performance, too, but will not compare plans solely within a market. "We expect the plans to comply with our standards no matter where they're located," says Mudd.
So, if there's such a wide range of performance, why is it so difficult to state definitively, "This is a bad health plan"?
Because, says Kongstvedt, "There is a staggering lack of consistency in defining what the data and processes are. No one health plan does everything extremely well, though a lot of health plans do some things extremely well."
Kongstvedt is involved in a massive benchmarking effort at Ernst & Young in which the company is evaluating 11 health plans covering approximately 11 million lives.
How does Kongstvedt define a health plan that misses the mark? "It depends on what process you're looking at. For Wall Street, a bad HMO is one that generates inconsistent earnings. For physicians, a bad HMO is one that pays them poorly and harasses them a lot. For the HMO executive, a bad HMO is an HMO that has high enrollee turnover. For patients, a bad HMO is one that costs a lot of money and doesn't give them good access."
Still, Kongstvedt thinks the term bad shouldn't be used to describe HMOs. He uses "bad physician" as an analogy. "It is usually a very, very small percentage of doctors who account for a disproportionate amount of trouble. You're probably not going to find a 'bad' health plan. For the vast majority of HMOs, their members are satisfied that they're doing a good job."
Jeffrey Kovaleski, chief operating officer for Florida Affiliated Physicians, an IPA, agrees. "A lot of members define good or bad based on whether their personal physician is part of the plan," he says.
Barbara McMahon, a benefits consultant at Hewitt Associates, played a role in compiling the consulting company's recently released employer-sponsored evaluation of 2,000 health plans, the Hewitt Health Value Initiative. Its plan performance index examined such aspects as clinical quality, administrative effectiveness and organizational stability. Plans in 141 different markets were compared on a market-by-market basis (national plans were considered separate operations in different regions), so a plan in, say, Massachusetts -- a state with a long history of managed care -- isn't compared to a plan in Alabama, where the industry is just beginning to make inroads.
"I think what I've seen is the level of quality is improving consistently," says McMahon. "The majority of employers are pushing for this. If you're in a market where seven of the ten plans score well on NCQA's HEDIS evaluation, for instance, there's going to be considerable pressure on the other three to improve." And employers won't be the only ones applying that pressure. "If I were a physician, I would want to align myself with the best HMO. Physicians have the right to ask the health plans that they contract with for HEDIS information."
McMahon thinks NCQA has generally established itself as the 800-pound gorilla when it comes to singling out the better health plans. "Most major employers consider a plan's NCQA accreditation status to be an important benchmark."
Those same sentiments are echoed by Emma Hoo, a senior project manager at Pacific Business Group on Health, a San Francisco consortium that purchases coverage for 21 companies on the West Coast, covering about 400,000 lives.
"In California, most of the larger health plans have three-year NCQA accreditation," says Hoo. "I would say when any health plan performs poorly across the board, employers carefully look at that information. We try to let the numbers speak for themselves. It's not just the accreditation itself but the HEDIS information as well as members' overall satisfaction that we look at." PBGH presents a blue ribbon each year to the health plan it thinks best serves employees. When asked what plans exist at the other end of the spectrum, Hoo says the group tries to focus on continuous performance improvement.
Ditto for the corporate giant Xerox, where Patricia Nazemetz, director of total pay, says, "I'm sure there are some plans that are less good than others. We try to make our selections against the standards we have."
Xerox requires the 260 plans it does business with to obtain NCQA accreditation and to report HEDIS data (as does PBGH). What about plans that come up short of expectations? "We've come across plans that employees have complained about but, like a lot of things, there are multiple sides to the story. In the '80s, there were plans that collapsed and you found out afterward that they weren't good plans. I can't even remember the names of them any more."
HEDIS isn't the only way to gauge health plan quality, of course; its strong focus on procedural issues gives the user only a slice of the total quality pie. But Donald Nielsen, M.D., senior vice president for quality leadership at the American Hospital Association, says some interpretation of the numbers can tell a lot about how well a plan handles services for which it is responsible.
"Issues around immunization," Nielsen offers as an example. "A low score could signify problems with access." Similarly, the HEDIS measurement of follow-up after hospitalization for depression can be a warning about continuity of care, he says.
Beyond the quality scores in HEDIS, some of its narrative measures yield important information about a plan's financial stability. "Looking for those, which are frequently not included in the report card, is key," Nielsen says. And, yes, he adds, accreditation by NCQA or some other respected body matters. "It serves as the check as to whether a plan has adequate infrastructure."
Market forces will enhance the NCQA's stature, Mudd contends. "The market's going to get what the market wants," says Mudd. "We've had feedback indicating that purchasers, consumers and others prefer to take a chance on a plan that's not been through the accreditation process than on one that's been denied. We're seeing more business consortiums expressing support for the idea of health plan report cards."
Historically, the managed care industry has accused employers of being less concerned with health care quality than with cost. But there are many shades of gray, says Catherine Kunkle, vice president of the Washington-based National Business Coalition on Health. "Very often, the reason employers may not look at specific quality information is that it is unavailable, or the kind of quality information a plan provides is not useful for the demographics of the group the employer might represent." (See "Do employers care if a plan misses the mark?" on page 38D.)
Hillman, at the University of Pennsylvania, says an employer-driven health care system -- especially in times of high employment -- will demand higher standards. During the last decade in industry, he points out, when talks between unions and management have stalled, it has often been because of benefits. "Businesses are not going to sustain the best-quality product if they can't attract the best-quality work force."
What's good or bad about health plans, says Hillman, starts at the top. "I would say that health plans that are missing the mark have an executive management that looks to short-term, rather than long-term, growth. Managed care organizations that have done well have put aside capital and have set their visions on the long term. Health plan executives should follow the advice from Management 101: Hire people who are smarter than you who will disagree with you."
Physicians should be the first to spot a poorly performing HMO -- hopefully, before they sign on, says Hillman. "If an HMO uses a cascade of financial incentives that are extraordinarily cumbersome without installing any counterbalancing quality assurances, that's a very bad marker," says Hillman. "If it hasn't given any bonuses in the last few years, then that's a bad sign. If it doesn't have a robust, rules-based set of protocols, then that's a bad sign."
Hillman, like Kongstvedt, says that the distance between the better health plans and the rest of the pack is shrinking. Several years ago, the differences could be likened to the differences between students who score 100 and 50 on a test. Not any more. "That's because the so-called bad HMOs are being forced out of the market," he says. And plans can't do an end-around by putting a lot of money into advertising while ignoring overall performance. "That won't help them fake out people who are trained to look for the right measures," says Hillman.
What about plans that do not participate in the NCQA's accreditation process? NCQA says nothing should be read into that. Hillman sees it differently: "That's a bad sign, period."
Kovaleski, at Florida Affiliated Physicians, says, "Any health plan that's doing a good job is balancing the interests of all its customers." Like Hillman, Kovaleski thinks that health plans miss the mark when they focus on short-term gains. "It's difficult to think long term but I think that's what makes a successful HMO."
Hillman says health care executives constantly try to balance clinical considerations with the bottom line. "The organization that's exemplary focuses on quality improvement in a fashion that pervades the entire organization."
More easily said than done, says Robert A. Reid, M.D., president of the California Medical Association. "It's a problem for plans to try to hit the right mix of careful cost-consciousness, yet provide excellent care. Frankly, I believe it can be done, but the learning curve is steep."
Very steep. In 1996, CMA launched California Advantage, a physician-run HMO, with attributes it thought would please consumers, employers and doctors. But pleasing the crowd with one hand while trying to control medical costs with the other is a difficult act, and taken with what Reid calls the "cold, hard facts" of the competitive California marketplace, Cal Advantage ran into trouble right away. Premiums, set low to gain market share, proved too low. And the market share never came: Cal Advantage attracted only 6,000 subscribers -- far fewer than the 40,000 it said it needed to break even. The HMO never capitated its physicians -- one way a health plan controls its risk. "And," Reid adds, "being a doctor-owned company, going for a certain niche market, it tended to attract sicker people. The medical loss ratio its first year of operation was close to 125 percent." Cal Advantage filed for bankruptcy protection June 17.
Last year, the Ann Arbor, Mich.-based Medstat Group surveyed 9,000 physicians as part of its Quality Catalyst program. The health care consulting company asked doctors hundreds of questions aimed at determining whether the health plans they contract with enhance or diminish their ability to deliver high-quality care. On a scale of 0 to 100, 27 health plans received overall scores ranging from 50 to 87. The higher the score, the better the health plan -- in physicians' eyes, anyway -- in terms of procedural and administrative aspects and quality of care.
Is it safe to say that the plans that scored near 50 are missing the mark? Medstat's Dennis Becker, a senior vice president, only laughed when asked to name those plans clinging to the bottom of the scale.
Of course, plans and employers that buy Medstat reports will know where each falls.
One point Medstat officials stress is that how physicians feel about a health plan often mirrors how enrollees (another group surveyed by Medstat) feel about it. "In our six markets, we found convergence on the enrollees' and the physicians' positions." In addition, plans cannot avoid scrutiny by opting out -- one advantage Becker says Medstat's survey has over accreditation processes conducted by the NCQA or the JCAHO. "It's designed so that plans can't really choose not to participate," he says.
So what are some of the bad health plans?
There was a disheartening similarity in responses from experts. It was "I don't think you can label a health plan good or bad" or "It depends on what process you're looking at" or "It's very difficult to make broad generalizations" or "How do you define a thing like that?"
How indeed? How does Bill Gates define software that won't sell? How did Henry Ford recognize an automobile that wouldn't run? On such questions, industries rise or fall.
Whether you're a practicing physician or a managed care executive, note the little things that could mean a health plan isn't running as smoothly as it should. Experts say any of the following can be a clue:
Robert A. Reid, M.D., president, California Medical Association
Kathy Cartwright, consultant and former HMO/PPO president
Walter Lane, M.D., president, Tampa Bay Doctors IPA
Alan Hillman, M.D., director, Center for Health Policy, Leonard Davis Institute on Health Economics, University of Pennsylvania
The charge from the managed care industry has long been that employers, for all their talk about quality, are more interested in cost. While large employers were a major driver behind the health-plan premium meltdown of the mid-'90s, evidence suggests that they demand quality, too. You can't blame them for trying to drive a bargain at the same time.
Proof of employers' desire for high-quality health care is most evident among the largest employers. In many markets, large-employer purchasing cooperatives use their clout to wring better care out of health plans. The San Francisco-based Pacific Business Group on Health, which represents 34 private- and public-sector purchasers -- each with 2,000 or more benefits-eligible employees -- requires health plans that do business with it to demonstrate multiple commitments to quality. And this coalition uses a carrot-and-stick approach to get what it wants.
"We put two percent of the premium at risk for performance," says Pat Powers, PBGH president. "That means we set targets for improvements in areas of satisfaction, customer service, quality, data and partnering." If a health plan meets a target, it keeps the portion of the premium at risk for that measure. If it fails to do so, the money is returned to the employer.
In Texas, the state's largest buyers' group -- the Houston Healthcare Purchasing Organization, representing 85 large employers and about 5,000 smaller companies -- has its own yardsticks for quality. President Ralph Smith says NCQA's HEDIS measurements, for all their pluses, lack something because "HEDIS measures, for the most part, satisfaction with a plan -- and what you see in the scores, a lot of times, is that there are not huge variations in a market." Instead, HHPO measures clinical performance by hospital and physician, "Then we make the assumption that any differentiation among plans would be the result of some plan intervention rather than the clinical quality of medicine."
The conventional wisdom is that smaller companies are less interested than larger employers in quality. To put it more bluntly, many industry watchers believe price is far and away the number-one concern of smaller employers. Smith doubts that's the only concern -- it's one reason his group reaches out to small businesses -- and argues that most of them lack the resources to make sense of health care quality data.
Would smaller employers take a proactive interest in quality if they had a better handle on it?
Some inferences can be made from a study released in mid-June by the Kaiser Family Foundation. Owners of 800 businesses employing fewer than 100 people were asked whether they favor some of the major provisions of health-insurance reform legislation in Washington. Nearly 9 in 10 said, for instance, that they would support laws that force health plans to provide more information about how they operate and to allow members to appeal health plans' decisions. When asked if they would still support such measures if it meant higher premiums, about 7 in 10 said yes.
Catherine Kunkle, vice president of the National Business Coalition on Health, says there is a move in Washington to create purchasing pools that would do the negotiating and contracting for groups of smaller employers, while interpreting complex quality data for them before putting it into their hands. Says Kunkle, "That's a way to help make the information useful to the employer or the employee."
Employers of fewer than 100 workers say they strongly support five major aspects of the various health- insurance reform initiatives floating around Washington -- even if it means higher premiums.
More than 4 of every 5 small-business owners who provide health coverage for their workers offer employees no choice of plans. Expense is a major reason.
SOURCE: NATIONAL SURVEY OF SMALL-BUSINESS EXECUTIVES ON HEALTH CARE, KAISER FAMILY FOUNDATION, MENLO PARK, CALIF., JUNE 1998
You're the quality and accreditation manager for a mid-size health plan in a market with two major competitors. This morning, the bad news hit your desk: Three months after you handed HEDIS information over to the National Committee for Quality Assurance, you have learned that your competition has run circles around you on several key measures of quality of care. You know you're not that bad and are huddling with the public relations department to figure out how to explain it in terms of information system deficiencies and rushed NCQA deadlines. But you also know your competitors are going to rub it in your face.
Aside from the embarrassment factor, what are the actual implications of receiving a bad rating?
Kathy Cartwright, president of Health Care Delivery Systems, a Fresno, Texas-based physician consultancy, says you should be mindful of your providers' reactions.
"Prestigious provider groups will be more sensitive to the quality perception of an HMO than other groups will, particularly in a competitive market," she says. "Many times, HMOs use well-known university specialists as a marketing tool, and those physicians would be mindful."
Robert A. Reid, M.D., president of the California Medical Association, agrees.
"I think physicians feel that if a plan is rated badly, and if they are associated with that plan, it reflects on them," he says. "It is of great concern to doctors when they find out that a plan that covers a majority of their patients does not score well. It has the potential to undermine the patient's trust in the physician as an agent of the plan."
Not only are quality ratings important, but so are tools that measure consumer satisfaction. Donald Nielsen, M.D., the American Hospital Association's senior vice president for quality leadership, says that patient satisfaction surveys can signal problems that providers and health plan executives alike should note: "Issues of access, payment of claims" and the implications of consumer satisfaction surveys. "I think serious attention would have to be paid to the issues behind that dissatisfaction."
The Medstat Group, an Ann Arbor, Mich.-based consulting organization, conducts an annual evaluation of health plans, with rankings based largely on physician input. Dennis Becker, senior vice president, says the rankings "reflect how well plans create environments to help physicians practice."
Of course, provider behavior -- such as following respected clinical guidelines or demonstrating a commitment to HEDIS yardsticks -- plays a significant role in a health plan's overall quality. Nielsen says that opens opportunities for collaboration of hospitals, physicians and health plans.
But if a hospital or other provider contracts with a health plan that is the only one in an area, that presents more of a dilemma -- particularly if the health plan is on shaky financial ground. "The hospital or other provider should look to regulatory authorities to monitor that plan, so that all of a sudden, there isn't a collapse that raises issues of providers getting paid and patients receiving ongoing care," says Nielsen.
Still, if your health plan scores low on quality scales, should you brace for a mass provider defection? Cartwright doesn't think so, particularly if your provider relations department is on the ball.
"For the most part, physician groups that have bought into managed care probably would not bail out of an HMO that has bad HEDIS scores. Even if they have ten, fifteen percent of the market, they will be reticent to drop a contract that pays them on time, reimburses them adequately and follows up on what it promises."
"Case in point," she continues. "I went to see my doctor the other day, and he knows I'm a consultant. So he goes on and on about HMOs: 'Such-and-such an HMO doesn't pay all that well, but they always pay me on time, and if I call them up, they always return the call. And I've got my own provider rep. So I stay with them.'"