MANAGED CARE April 2010. ©MediMedia USA
When many randomized trials are badly done or irrelevant, what are formulary managers and committees to do?
Sean Karbowicz, PharmD, has some sobering news for anyone who designs clinical trials: Most of them really aren’t very useful for the managed care organizations that make coverage decisions. Too often, he contends, the studies contain a flaw — a lack of blinding, too small a sample, high drop-out rates, or weak endpoints. As a result, he says, winning formulary status can be increasingly difficult in an age when meaningful comparisons are more important every day.
“We see this as a growing problem,” says Karbowicz, who is a manager of clinical pharmacy services at RegenceRx, the not-for-profit arm of the Regence Group, which is based in Portland, Ore., and embraces several health care plans* in the Pacific Northwest.
“We keep track and, on average, over the past five years or so, we see just 1 out of 10 randomized clinical trials that are of fair quality, which means that 9 out of 10 are of poor quality. And studies of poor quality pose the potential for more opportunity for bias.”
Source: RegenceRx 2009
So what constitutes a reliable and useful trial? Lack of bias, confounders, and statistical flukes, plus the presence of data transparency, intent-to-treat analysis, and positive proof of cause and effect.
“The first thing we do is look for evidence of safety and efficacy, and then we’ll look at cost,” says Karbowicz. “When we know a drug is safer than an existing drug, we want to know if the cost is in line with reported improvements. But it is rare that we get to that point because we don’t have evidence of clinical advantages.”
Value judgment
He offered fibromyalgia medications as an example. Between 2007 and 2009, RegenceRx reviewed more than two dozen trials for several widely used drugs — Cymbalta, Lyrica, Savella, Neurontin, and various tricyclic antidepressants — and found that none of them looked useful in gauging which medication worked best. “We’re not saying they don’t work; all of them probably have some effect on fibromyalgia pain. But do we know which one is better? No,” says Karbowicz. “It is hard to make a sound value judgment. This is the problem.”
And so RegenceRx is trying to make the case that better information is needed from clinical trials if a pharmaceutical company wants coverage of its medications. The rationale is seemingly simple: Obtaining FDA approval is really only half of the game. The bar is now higher, and satisfying third-party payers is becoming nearly as important as satisfying regulators.
“This has become a two-step process,” says Marilyn Dix Smith, executive director of the International Society for Pharmacoeconomics and Outcomes Research, a not-for-profit organization.
“The pharmaceutical industry has to obtain FDA approval by showing safety and efficacy, but they now have an additional hurdle — to justify reimbursement for new drugs versus existing therapy. A new set of questions is being asked for this hurdle.”
Toward that end, RegenceRx conducted a study that covered about 1,000 clinical trials a year between early 2006 and early 2009. And the results were sobering. In its view, the percentage of reliable trials ranged from a stark 0 percent to no higher than 15 percent over the three years. During the first half of 2009, the most recent period covered by the review, no more than 8 percent of the studies were found to be reliable. In a related look at product value, RegenceRx examined drugs approved by the FDA between July 2005 and June 2009 and compared them with existing medications. RegenceRx found that just 15 percent offered improved effectiveness, only 1 percent offered added safety, 4 percent yielded cost savings, and 2 percent offered improvements in patient adherence.
| Product value categories | |
|---|---|
| Category | Description |
| Effectiveness |
|
| Safety | Safety or side effect advantage over other options |
| Improved adherence potential | Potential for improvement in medication adherence |
| Cost | Significant cost advantage over other options |
| Convenience | Simpler dosing, packaging, route of administration |
| No additional value | No clinical advantage over other options |
| Adapted from: Changing Patterns of Pharmaceutical Innovation. NIHCM Foundation. May 2002. www.nihcm.org | |
By contrast, 47 percent offered an advantage in convenience — and 56 percent yielded no additional value. In reaching its conclusions, RegenceRx used methodology that was developed by the National Institute for Health Care Management Foundation.
Based on methods adapted from: Changing Patterns of Pharmaceutical Innovation. NIHCM Foundation. May 2002. www.nihcm.org [1]
Source: RegenceRX 2009
There is more. RegenceRx also examined the dossiers that drug makers provided to managed care payers — health plans and pharmacy benefit managers — and found that these were also lacking. Of those submitted between July 2008 and June 2009, 36 percent were missing significant clinical trial data and 45 percent failed to provide pharmacoeconomic models. Only 19 percent offered both sets of information, according to RegenceRx’s review of FDA files.
Between July 2005 and June 2006, 27 percent of the dossiers were missing significant clinical trial data and a similar proportion were missing pharmacoeconomic models. In all, 46 percent offered both sets of information, substantially higher than what was found in 2009, three years later.
Karbowicz says that the dossiers (Regence pioneered the dossier system over 10 years ago, and the Academy of Managed Care Pharmacy quickly adopted something similar) “are providing helpful information and the manufacturers are getting better at providing the information we need. Part of the reason that the dossiers were missing [information] in 2009 was because we broadened the scope of drugs we looked at that year. Previously, we limited our formulary reviews to drugs covered under the prescription benefit, and manufacturers have been providing more and more evidence we need.
“We built an integrated formulary of medications that are covered under the prescription benefit — they usually have dossiers — and then in 2009 started looking at medications covered under the medical benefit; drugs that are administered in doctors’ offices or infusion clinics and that usually don’t have dossiers. The manufacturers probably haven’t been asked for them before and didn’t have them.
“You have to look to see why studies are performed in the first place and most are funded by the manufacturers to gain market approval,” says Karbowicz. “The FDA requirement is for safety and effectiveness, which is fine, but only up to a point. Rarely do we see valuable, reliable head-to-head studies. Unfortunately, yes, these are much more expensive and take longer. But they serve a purpose. Sometimes we see noninferiority studies, which are intended to show a drug is no worse than a competitor. But in those cases, the bar is actually lower, even if they prove their point, because they are still not saying their drug is better.”
A backdrop to this discussion, of course, is comparative effectiveness, which has emerged as something of a flash point in the national debate over health care reform. As purchasers ranging from federal and state agencies to insurers and employers seek to lower costs, prescription drugs have remained under the microscope. Congress, for instance, recently passed legislation that will provide more than $1 billion to support comparative effectiveness studies.
The issue was underscored by a recent study in the Journal of the American Medical Association that analyzed comparative effectiveness studies that were published in several leading medical journals and found that only 32 percent compared the effectiveness of existing drugs. Moreover, only 19 percent examined safety and just 2 percent eyed cost effectiveness.
And of the 212 randomized trials analyzed, 46 percent used an active comparator (an active therapy) while the rest used an inactive control, such as a placebo. Active-comparator trials were less likely (44 percent) than trials with inactive controls (66 percent) to report positive results.
The report, “Characteristics of Published Comparative Effectiveness Studies of Medications,” is in the March 10 issue of JAMA.
“Cost-effectiveness analyses are critical for promoting efficient and effective health care,” concluded the authors, Michael Hochman of the Keck School of Medicine at the University of Southern California and Danny McCormick of the department of medicine at Cambridge Health Alliance and of Harvard Medical School. “In addition, our findings highlight the need for regulatory agencies, like the FDA, to require active-comparator trials for medication approval whenever feasible.”
A spokesman for Pharmaceutical Research and Manufacturers of America declined to comment about the topic, saying clinical trial design and outcomes is a “company by company” issue. Nonetheless, some drug makers are waking up to the realities and are beginning to incorporate the heightened demands of third-party payers into their thinking.
Appropriate payment levels
At a pharmaceutical industry conference last fall, Peter Wirth, executive vice president for legal and corporate development at Genzyme, noted that the Cambridge, Mass., biotech tries to gauge appropriate payment levels. “We’re more aware of the need to demonstrate that there is a real benefit to the drugs we’re trying to develop,” he said. And at the same gathering, an executive from Eli Lilly mentioned that payment milestones — not just clinical and regulatory milestones — are now being considered when looking at structuring development and commercialization deals with smaller companies.
Meanwhile, a spokesman for the Association of Clinical Research Organizations (ACRO) maintains that RegenceRx has an understandable viewpoint, but one that doesn’t fully reflect the central mission behind clinical trial design.
“The perspective of a payer is much different from the perspective of a pharma company or regulator,” says ACRO’s John Lewis. “The primary goal of clinical trials is to first judge if a medication is safe and effective. Therefore, I question the RegenceRx conclusion that the trials are flawed, as that only reflects the perspective of the payer, not that of the study sponsors.
“We’ve seen, in the past few years, a few more examples of sponsors building some measures of comparative effectiveness, but that is not the goal of most clinical trials. The trials simply aren’t designed to say what is the best drug to use. That gets into another debate. What the payer wants from a clinical trial is different from what a sponsor wants.”
An executive at one of the large clinical research organizations says he is sympathetic to payers and the emphasis being placed on determining value, but he also maintains that the separation between the kind of information sought by regulators and by insurers will be difficult to overcome. Manufacturers may increasingly understand the need to provide a certain kind of information, but the cost of large-scale trials remains prohibitive.
“I understand that, from a payer’s perspective, these trials really aren’t meeting their needs,” says Chuck Stevens, vice president and general manager for reimbursement and market access at Parexel International, which is based in Waltham, Mass. “If the manufacturer wants to advance the ball to be in a better position from the marketplace perspective, it needs to understand what is happening from a clinical perspective in disease states and the treatment options available to patients.... They do need to provide safety and efficacy and value.
“But it is challenging for payers to say the trials are flawed. I don’t know that they are flawed. They may not like the trial design process, but the process isn’t going to necessarily offer a nice tidy package that meets everyone’s needs in today’s marketplace. It is simply not going to happen. The challenge from the manufacturer’s side is, of course, that regulatory approval doesn’t automatically equate to getting the payer on board. But that requires additional trials, often with large numbers of patients. Somebody has to shoulder the cost.”
As more biomarkers are discovered and utilized in gauging effectiveness among subsets of patients, the debate may shift. That is because the advent of personalized medicine suggests a more fine-tuned approach to clinical trial design and a more substantive set of outcomes that will appeal to payers. But for now, Karbowicz believes manufacturers have little choice but to design trials more rigorously. And this goes for first-in-class therapies just as much as for me-too meds.
“What we’re really talking about is marketability,” he continues. “If the companies want to support the prices they are charging for their products and prove their marketing claims are accurate, we need more data to determine the accuracy of those claims. And unfortunately, after FDA approval, those are often very sparse. And we will continually monitor and revisit drugs and clinical evidence that support their use.”
*The members of the Regence Group are Regence BlueShield of Idaho, Regence BlueCross BlueShield of Oregon, Regence BlueCross BlueShield of Utah, and Regence BlueShield (Washington).