A Comparison of Diabetes Patients' Self-Reported Health Status With Hemoglobin A1c Test Results In 11 California Health Plans

Jeffrey Wasserman, Ph.D.

The Medstat Group, Santa Barbara, Calif

Gifford Boyce-Smith, M.D.

Blue Shield of California, San Francisco;

David S. P. Hopkins, Ph.D.

Pacific Business Group on Health, San Francisco;

Vernon Schabert, Ph.D.

The Medstat Group, Santa Barbara, Calif

Mayer B. Davidson, M.D.

King-Drew Medical Center, Los Angeles;

Ronald J. Ozminkowski, Ph.D.

The Medstat Group, Santa Barbara, Calif

Ann Albright, Ph.D., R.D.

University of California, San Francisco

Sean Kennedy, B.S.

The Medstat Group, Santa Barbara, Calif

Abstract

Purpose: To examine the relationship between hemoglobin A_1c (HbA_1c) test rates and values and various self-reported measures of health status within a sample of diabetes patients drawn from 11 California health plans, with a focus on improving diabetes care in this patient population.

Design: The analysis relies on data obtained from medical records of a sample population of 4,747 diabetes patients and a patient survey mailed to a large subsample of patients included in the medical-records analysis.

Methods: Descriptive methods were used to compare the medical records and survey-data results.

Principal Findings: There were substantive differences noted between diabetes patients' self-reported health status, their level of satisfaction with the care they received, and the actual care they received. There was a large discrepancy between diabetes patients' perceptions of the care they received for their diabetes, which was overwhelmingly positive, and the HbA_1c test-frequency rates observed across the 11 health plans studied, which were low.

Conclusions: Patients' self-reports of health status, satisfaction with care, and extent of control over diabetes — a chronic condition that may have few perceptible symptoms — are associated with significant methodological limitations. Our examination of the relationship between perceived levels of self-management of diabetes and test status indicated that for patients who had at least one HbA_1c test, some education during that process may have resulted in behavioral change. Patients who received no tests, however, may remain unaware of their glycemic control and the long-term consequences associated with even mild hyperglycemia. A clear need thus exists to educate diabetes patients about their health status. Health plan and provider group investments in educational efforts aimed at increasing testing rates are likely to lead to improved glycemic control and a reduction in the incidence of diabetes-related complications and related expenditures.

INTRODUCTION

Diabetes mellitus and its complications have a profound effect on the health of our population as well as the U.S. economy. Diabetic retinopathy is the leading cause of blindness in the nation in people between 20 and 74 years of age.¹ Diabetic nephropathy is the leading cause of patients' undergoing dialysis for end-stage renal disease.² Diabetic peripheral neuropathy is the underlying cause of nontraumatic lower-extremity amputations in diabetes patients.³ More than half of lower-extremity amputations occur in people with diabetes,⁴ who constitute 7.8 percent of adults over 20 years of age in the United States, according to the Third National Health and Nutrition Examination Survey (NHANES III) conducted in 1988-1994.⁵ The prevalence of coronary artery disease is twofold higher in men with diabetes and fourfold higher in women with diabetes, compared to appropriate nondiabetic controls.⁶ Strokes are two to three times more common in people with diabetes than in those without the disease.⁷ Peripheral vascular disease is also much more common in diabetes patients compared to nondiabetic individuals.⁸

These complications may be avoidable. Although control of glycemia has not been shown to have a marked effect on the macrovascular complications of diabetes,⁹ there is irrefutable evidence that near euglycemia will delay, blunt, and possibly prevent the microvascular and neuropathic complications of both type 1 and type 2 diabetes.^10-12 Unfortunately, however, these beneficial effects are seldom achieved. Studies in both type 1 and type 2 diabetes patients show that the development and progression of retinopathy and nephropathy markedly increased with HbA_1c levels >8 percent, increased moderately at HbA_1c levels between 7 percent and 8 percent, and increased slightly or not at all with HbA_1c levels ¾7 percent.^{10,11, 13-15}

The average glycated hemoglobin level in 4,449 patients who were followed in HMO settings was 9.51 percent, and it was 9.59 percent in 3,140 patients seen in fee-for-service settings.¹⁶ Other studies also have revealed similar poor control when evaluated by fasting glucose concentrations^17-19 or glycated hemoglobin levels.^20-23

One of the barriers to achieving near euglycemia is that patients tend to be asymptomatic until one of the neuropathic, retinopathic, or macrovascular complications manifests. Patients therefore may lack ongoing motivation to do the work needed to attain tight glycemic control. Physicians may communicate the severity of the patient's condition to each individual, recommending specific treatments in every case, yet they and other health care professionals face a significant challenge in motivating their asymptomatic patients to achieve near euglycemia. Moreover, continually growing constraints on physicians' time affect their ability to adequately educate patients on the importance of maintaining near euglycemia. All this suggests a need for diabetes patients to have access to diabetes educators.

In this study, the relationship between HbA_1c testing rates and HbA_1c test values and various self-reported measures of health status were examined for a sample of diabetes patients drawn from 11 California health plans. The study was undertaken on behalf of the California Collaborative Healthcare Reporting Initiative. CCHRI, which comprises representatives from many of California's leading health care purchasers, health plans, and provider groups, is dedicated to measuring and improving the quality of health care delivered to Californians.

The HbA_1c test is a critical indicator of diabetes care. This test, which ideally should be performed at 3-month intervals, provides a measure of plasma glucose control for the previous 3 months. For asymptomatic diabetes patients, HbA_1c levels are the most reliable clinical indicators of health status and glycemic control. Nevertheless, studies have shown that many diabetes patients fail to receive this test at least once yearly.²¹ Good control of blood glucose levels has been shown to prevent many complications that are associated with diabetes.^11–18

Previous studies of patient-reported outcomes have tended to focus on conditions that have perceptible symptoms and complications. Patients with depression,²⁴ arthritis,²⁵ and ulcers,²⁶ as well as patients recovering from surgery,^27,28 have accurately reported their health status relative to objective clinical information. Evaluations of conditions like diabetes,²⁹ with few detectable symptoms in the early stages, focus on the internal consistency of patient-reported responses rather than correlations with objective clinical anchors. Patients with hypertension, for instance, another condition without symptoms in its early stages, showed no significant differences in self-reported health status when compared to the self-reports of patients with no chronic conditions.³⁰ It is unlikely that patient reports can be a reliable source for detecting asymptomatic conditions. If patients could report adverse health status for asymptomatic conditions, then these conditions would probably not be significantly underdiagnosed in the general population.

Methods

The analysis relied on data obtained from two sources: a sample of diabetes patients' medical records and a patient survey that was mailed to a large subsample of the patients included in the medical-records analysis.

Medical-records data

To measure HbA_1c testing-frequency rates and values, we analyzed medical records from 11 of the health plans that agreed to participate in the California Collaborative To Improve Diabetes Management Project, or the Diabetes CQI Project. The project was conducted under the aegis of the CCHRI. All participating plans offered commercial plans, and four of the 11 also covered Medicare beneficiaries.

Specifically, we collected information on the frequency of HbA_1c tests performed on the patient population during calendar year 1997. We also analyzed data on the results of the HbA_1c tests. During the course of our analysis, we abstracted clinical information from a total of 4,747 medical records obtained from 11 health plans, or approximately 431 records per plan. Diabetes patients were identified using the 1998 specifications for the measure "Eye Exams for People with Diabetes," part of the Health Plan Employer Data and Information Set (HEDIS 3.0/1998). These specifications rely on both pharmacy and encounter data: A patient was identified as having diabetes if pharmacy data indicated that the individual was on insulin or oral hypoglycemics, or if encounter data indicated that the patient had at least two outpatient visits associated with a diagnosis of diabetes or one hospitalization associated with a diagnosis of diabetes. We used a systematic sampling approach, as prescribed by the HEDIS 3.0/1998 specifications, to select records for study.

The medical records were abstracted by nurses and medical-records professionals, all of whom attended a training session and received written instructions elucidating the numerator and denominator criteria. A team of clinicians developed a standardized data-abstraction form to collect the requisite data. Abstractors used an electronic version of this form to facilitate data collection and entry. To ensure data quality, record-review supervisors "overread" at least 5 percent of the records assigned to each abstractor, and provided timely feedback on errors and other data-quality issues.

Patient-survey data

As part of the Diabetes CQI Project, we mailed a survey instrument to 3,296 persons with diabetes who were enrolled in the 11 participating health plans. [The sample size was based on power calculations conducted to determine the number of individuals required to measure the effects of a planned intervention to improve outcomes of care for diabetes patients.] The sampling frames used to select patients for the survey were the patient samples that the plans provided for the medical-records analysis. The purpose of the survey was to create a knowledge base of information about persons with diabetes and the professional care they receive, from the patients' perspectives. The details of the patient-survey process were reported by Freeman, Sullivan & Co. (1999).³¹ The instrument used to collect the survey data was, in many respects, similar to that used in the Diabetes Patient Outcomes Research Team (PORT) project.³²

Analysis

For the most part, descriptive methods were used to compare the medical records and survey data results on the 11 participating health plans. We calculated means and standard deviations for all continuous variables, and compared differences in means using a standard t-test. In addition, we generated frequency distributions for all categorical variables and calculated chi-square statistics.

We also used multivariate statistical techniques to develop a set of weights that were applied to the survey results. We determined that such weights were appropriate to increase the survey's usefulness, as described below.

We achieved a 43 percent response rate to the mail survey. The response rate varied substantially by plan and plan type. In general, the response rate was higher from beneficiaries of the Medicare plans (53 percent) than from those covered by the commercial plans (39 percent). We conducted a logistic regression analysis to estimate the effects of the various patient factors on the likelihood of responding to the mail survey. Specifically, the logistic regression analysis was used to estimate response probabilities, which in theory may range from 0 percent to 100 percent. The range of survey response probabilities was found to vary from 22 percent to 76 percent. As recommended by Kalton and Kasprzyk,³³ the inverse of these probabilities (1/prob) was used to construct a weighting variable that was applied to all subsequent analyses reported in this paper. The weights were higher for respondents who had lower response probabilities, compared to the weights generated for those with higher response probabilities (e.g., 1/0.22 >1/0.76). By applying these response weights, the results of the statistical analyses reported can be more readily generalized to the underlying population of people with diabetes in the 11 participating plans than would be the case if analyses that were not weighted had been conducted.

It should be noted that some of the analyses that were conducted relied on both the medical records and survey data. That is, responses obtained from various survey questions were compared to test results found in the medical records.

RESULTS

In the subsections below, we report the results of our medical-record reviews and patient survey, respectively. We then report the results that were obtained after we merged the medical records and survey data.

Data from medical records

Table 1 shows HbA_1c test-related data for each plan participating in the collaborative. The test rate measures the proportion of a plan's diabetic patients that received at least one test in 1997. The overall test rate was 49.2 percent, ranging from a low of 38.5 percent for Plan D to a high of 58.1 percent for Plan G. None of the plans' test rates were statistically different from the overall mean rate of 49.2 at the 0.05 level.

TABLE 1 HbA1c Test Data by Plan — 1997
Plan	HbA1c Test Rates	Average Number of HbA1c Tests	Change in Average HbA1c Test Values	Proportion of Final HbA1c Values >9.5%
A	47.6	1.9	-0.6*	12.2
B	53.6	2.1	-0.6*	23.3
C	51.8	2.0	-0.8*	27.2
D	38.5	1.8	-0.3	17.5
E	56.7	1.9	-0.4*	24.1
F	46.8	1.8	-0.8*	23.9
G	58.1	1.8	-0.7*	25.1
H	41.7	2.0	-0.7*	35.6
I	49.1	1.8	-0.7*	30.8
J	49.3	1.9	-0.9*	26.7
K	48.2	1.8	-0.7*	19.2
* P<0.05

Diabetes patients who had been tested received, on average, two tests in 1997. Moreover, we observed remarkably little variation across plans in this regard, as the mean number of tests ranged from a low of 1.8 for Plans D, F, and, G to a high of 2.1 for Plan B.

In addition to examining the frequency of HbA_1c testing, we also calculated average HbA_1c test values by plan. Specifically, for patients with two or more tests in 1997, we calculated the average values of the initial and final tests and the differences between the two. Interestingly, the mean values improved for all plans, and all the differences were statistically significant (P<0.05), with the exception of Plan D.

In a further effort to generate insight into how blood glucose control varies across plans, we calculated the percentage of each plan's patients that had HbA_1c test values beyond the Diabetes Quality Improvement Project (DQIP)³⁴ threshold of 9.5 percent. [Technically, the relevant DQIP rate also includes people who were not tested.] The results indicate that approximately a quarter of the patients included in the sample had final (for 1997) HbA_1c values that exceeded 9.5 percent. The plan-specific rates ranged from a low of 12.2 for Plan A to a high of 35.6 for Plan H.

Survey data

Survey response rates differed by age, health plan, plan type, and treatment patterns. About 52 percent of the survey respondents were male and 77 percent were enrolled in commercial plans.

A number of the questions posed in the survey related to the respondents' perceptions of their health status along with their level of satisfaction with various aspects of the care they received. For example, 73 percent of the respondents reported that they were in good to excellent health. Moreover, just over 84 percent of the respondents indicated that their health status was at least as good as it was a year ago. Overall, respondents reported that they were quite satisfied with the care they received for their diabetes. Specifically, more than 83 percent rated the quality of care between good and excellent.

Three survey questions related to diabetes management. The first asked respondents to comment on how well controlled their disease was. Response categories ranged from "very well controlled" to "not controlled at all." Overall, 44.4 percent of the commercial patients and 60.4 percent of the Medicare patients stated that their diabetes was either "well controlled" or "very well controlled."

Combined data

Using both the survey and medical-records data, we examined the degree of correspondence between patient self-reports of health status behavior and actual tests administered to patients. As shown in Table 2, in both samples, respondents who had an HbA_1c test were less likely to say their disease was either well controlled or very well controlled, compared to those who did not have an HbA_1c test in the previous year. [The relationship between HbA_1c status and perceptions of control over diabetes was statistically significant (P-value of the chi-square <0.05) for those in commercial plans but not for those in Medicare plans. ] This result might seem counterintuitive at first glance, since the purpose of the HbA_1c test is to provide definitive information on blood sugar levels and thereby enhance the ability to control diabetes. Only 51.1 percent of the sample received an HbA_1c test, however, and these tests may be prescribed more often for those with a history of poor blood-glucose control. Alternatively, the results from the HbA_1c test probably came back before the survey was completed, and poor results from that test may have prompted many test takers to report their disease as poorly controlled.

TABLE 2 Relationship Between Perceived Control Over Diabetes and HbA1c Test Status
	Commercial (weighted n = 2,182)		Medicare (weighted n = 640)
Response	No HbA1c Test	Had HbA1c Test	No HbA1c Test	Had HbA1c Test
Very well controlled	12.7	8.3	18.9	14.3
Well controlled	35.2	32.8	42.2	39.5
So-so	42.1	48.6	32.0	34.3
Not well controlled	8.1	9.6	5.6	9.2
Not controlled at all	1.7	0.4	0.6	2.7
No answer	0.3	0.3	0.7	0.0
Total	100%	100%	100%	100%

A second survey question asked respondents to describe how well they kept tabs on their disease. As indicated in Table 3, about 54.3 percent of the commercial patients and 68.8 percent of the Medicare patients said they were "always on top" of their disease or "on top of it most of the time." Among the commercial patients, those who had had an HbA_1c test sometime during the past year were significantly less likely than those who had not had an HbA_1c test to say they were "always on top of their disease" or to say they were "on top of it most of the time" (50.1 percent for those with an HbA_1c test, vs. 59.8 percent for those without an HbA_1c test.) Again, it may be that the HbA_1c test was more often prescribed for patients with a poor history of diabetes self-management or that patients aware of high HbA_1c levels were less likely to conclude that they were "on top of their disease." Within the Medicare sample, no relationship was found between HbA_1c status and perceptions of self-management. The Medicare sample was much smaller, however, which may explain why no statistically significant results were observed.

TABLE 3 Relationship Between Perceived Levels of Patient Self-Management of Diabetes and HbA1c Test Status
	Commercial (weighted n = 2182)		Medicare (weighted n = 640)
Response	No HbA1c Test	Had HbA1c Test	No HbA1c Test	Had HbA1c Test
Always on top of the disease	17.9	13.2	30.3	17.7
On top of the disease most of the time	41.0	36.9	38.5	51.3
Sometimes on top of the disease	25.9	26.6	18.7	17.5
Slip up more than patient should	11.8	18.0	8.8	10.8
Slip up way too much	3.0	4.2	1.5	0.00
No Answer	0.6	1.0	2.2	2.8
Total	100%	100%	100%	100%

DISCUSSION

The results reported in the preceding section present substantial variations between diabetes patients' self-reported health status, their level of satisfaction with the care they received, and the actual care they received for their diabetes. Specifically, we observed a large discrepancy between the patients' perceptions of the care they received for their diabetes, which was overwhelmingly positive, and the frequency of HbA_1c testing observed across the 11 health plans, which was disappointingly low.

The low testing-frequency rates that were observed are in line with the results reported by Peters et al.,²¹ which, using 1993 data from a major California-based HMO, found that only 44 percent of the medical records reviewed on diabetes patients had a documented glycated hemoglobin test. Unfortunately, as our results indicate, little progress has been made in the interim by California HMOs. Thus, a considerable opportunity for improvement exists.

Our results, however, are encouraging in the sense that those patients who received at least one test actually averaged almost two tests per year. Furthermore, patients who received two or more tests actually showed a statistically significant improvement in their HbA_1c test values. The occurrence of multiple tests per year is encouraging because it suggests that physicians who order HbA_1c tests, and patients who receive them, understand that routine testing is an important part of their diabetes treatment plan. The significant decrease in test values among patients with multiple tests suggests that the initial test results alerted both patients and physicians of the need to take steps to improve blood glucose levels through education, diet, exercise, and, when necessary, medication.

Moreover, in a set of results not reported here, for tested patients we found a high correlation between HbA_1c test values and patient reports of their levels of diabetes glucose control and self-management. These results may indicate that patients who received HbA_1c tests were aware enough of their results to accurately assess their health status and its relationship to their diabetes management efforts. It is reasonable to assume that the process by which physicians communicated HbA_1c results to their patients was also an opportunity for the physician to educate patients on their health status and any need for improved diabetes management practices. These findings, combined with the observation that twice-tested patients improved their glycemic control, confirm the value of routine HbA_1c testing as an education- and health-improvement vehicle for diabetes patients.

One troubling finding is that in spite of the low overall testing rates and percentage of patients with exceedingly high HbA_1c levels, most patients reported being in good health and on top of their diabetes management. The most ironic part of that finding is that patients with no HbA_1c test in the prior year perceived themselves to be in better health and control than did patients who had received at least one test. Because most patients in this sample had no tangible symptoms, patients without an awareness of their glycemic control may have assumed that "no news is good news." Specifically, untested patients may have assumed that they were in good health and good control in the absence of physical discomfort or objective clinical evidence. Patients who had received HbA_1c test results, however, may have been more likely to report poorer health status and control because they had spoken with their physicians about their HbA_1c results and obtained a better understanding of what it means to be in good control.

In light of these findings, a clear need exists for educating diabetes patients about their health status. For those patients who received at least one HbA_1c test, some educational process may have occurred during the physician-patient interaction that resulted in behavioral change. Patients who received no tests, however, may remain unaware of their degree of glycemic control and the long-term consequences associated with even mild hyperglycemia. Health plan and provider group investments in physician and patient educational efforts aimed at increasing test rates are likely to lead to improved glycemic control and have a high pay-off in terms of reducing the incidence of diabetes-related complications and related expenditures.

Based on the observations gained from this study, increased HbA_1c testing rates also may have educational value, encouraging physicians and patients to improve glycemic control and secondary prevention of complications.

Finally, our results suggest that self-reports of health status, satisfaction with care, and extent of control over a patient's disease may not be sufficient indicators of quality of care for chronic conditions with few perceptible symptoms. The reliability of patients' self-reports are most suspect when patients have varying levels of awareness about their disease and clinical outcomes. Additional analysis of the differences in self-reported health status between clinically informed and uninformed patients may further elucidate the implications of using patient reports for asymptomatic conditions. Many chronic illnesses, such as hypertension and hyperlipidemia, have few perceptible symptoms until the patient's condition has advanced to the point of developing vascular complications (e.g., heart attack, stroke). Given this fact and the findings presented here, patients with chronic asymptomatic conditions who report good health status may need more targeted outreach and education from their physicians and health plans.

ACKNOWLEDGEMENTS

The authors would like to express their sincere appreciation to Bayer Corporation, Bristol-Myers Squibb, the Diabetes Control Program of the California Department of Health Services, Hoechst Marion Roussel, Inc., LifeScan, Parke-Davis, and Pfizer for their generous financial support.

REFERENCES

1. Klein R, Klein BEK. Vision disorders in diabetes. In: National Diabetes Data Group, eds. Diabetes in America. 2nd ed. Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;293–338.
2. Nelson RG, Knowler WC, Pettit DJ, Bennett PH. Kidney diseases in diabetes. In: National Diabetes Data Group, eds. Diabetes in America. 2nd ed. Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;349–400.
3. Habershaw G. Foot lesions in patients with diabetes: cause, prevention, and treatment. In: Kahn CR, Weir GC, eds. Joslin's Diabetes Mellitus. 13th ed. Philadelphia: Lea & Febiger, 1994:962–969.
4. Reiber GE, Boyko EJ, Smith DG. Lower extremity foot ulcers and amputations in diabetes. In: Diabetes in America. 2nd ed. Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;409–428.
5. Harris MI, Flegal KM, Cowie CC, et al. Prevalence of diabetes, impaired fasting glucose, and impaired glucose tolerance in U.S. adults: The Third National Health and Nutrition Examination Survey, 1988–1994. Diabetes Care. 1998;21:518–524.
6. Wingard DL, Barrett-Connor E. Heart disease and diabetes. In: National Diabetes Data Group, eds. Diabetes in America. 2nd ed. Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;429–448.
7. Kuller LH. Stroke and diabetes. In: National Diabetes Data Group, eds. Diabetes in America. 2nd ed., Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;449–456.
8. Palumbo PJ, Melton LJ III. Peripheral vascular disease and diabetes. In: National Diabetes Data Group, eds. Diabetes in America. 2nd ed. Bethesda, Md: National Institutes of Health, NIH Pub. No. 95-1468. 1995;401–408.
9. Wild SH, Dunn CJ, McKeigue PM, Comte S. Glycemic control and cardiovascular disease in type 2 diabetes: a review. Diabetes Metab Res Rev. 1999;15:197–204.
10. The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. New Engl J Med. 1993;329:977–986.
11. Ohkubo Y, Kishikawa H, Araki E, et al. Intensive insulin therapy prevents the progression of diabetic microvascular complications in Japanese patients with non-insulin-dependent diabetes mellitus: a randomized prospective 6-year study. Diabetes Res Clin Pract.1995;28:103–117.
12. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphyonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352:837–853.
13. The Diabetes Control and Complications Trial Research Group. The relationship of glycemic exposure (HbA_1c) to the risk of development and progression of retinopathy in the Diabetes Control and Complications Trial. Diabetes. 1995;44:968–983.
14. Krolewski AS, Laffel LMB, Krolewski M, Quinn M, Warram JH. Glycosylated hemoglobin and the risk of micro-albuminuria in patients with insulin-dependent diabetes mellitus. New Engl J Med. 1995;332: 1251–1255.
15. Tanaka Y, Atsumi Y, Matsuoka K, Onuma T, Tohjima T, Kawamori R. Role of glycemic control and blood pressure in the development and progression of nephropathy in elderly Japanese NIDDM patients. Diabetes Care. 1998;21:116–120.
16. Davidson MB. Diabetes care in health maintenance organization and fee-for-service settings. Dis Manage Health Outcomes. 1997;2:189–197.
17. Wei M, Gaskill SP, Haffner SM, Stern MP. Effects of diabetes and level of glycemia on all-cause and cardiovascular mortality. The San Antonio Heart Study. Diabetes Care. 1998;21:1167–1172.
18. Berlowitz DR, Ash A, Friedman R, Hickey E, Kader B, Moskowitz MA. Problems in assessing diabetes control in an ambulatory setting. Am J Med Qual. 1998;13:89–93.
19. Smith NL, Heckert SR, Bittner VA, et al. Antidiabetic treatment trends in a cohort of elderly people with diabetes: the Cardiovascular Health Study, 1989-1997. Diabetes Care. 1999;22:736–742.
20. Smith SA, Murphy ME, Huschka TR, et al. Impact of a diabetes electronic management system on the care of patients seen in a subspecialty diabetes clinic. Diabetes Care. 1998;21:972–976.
21. Peters AL, Legorreta AP, Ossorio RC, Davidson MB. Quality of outpatient care provided to diabetic patients. A health maintenance organization experience. Diabetes Care. 1996;19:601–606.
22. Aubert RE, Herman WH, Waters J, et al. Nurse case management to improve glycemic control in diabetic patients in a health maintenance organization. A randomized, controlled trial. Ann Intern Med. 1998;129:605–612.
23. Benjamin EM, Schneider MS, Hinchey KT. Implementing practice guidelines for diabetes care using problem-based learning. A prospective controlled trial using firm systems. Diabetes Care. 1999;22:1672–1678.
24. Berwick DM, Murphy JM, Goldman PA, Ware JE Jr, Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Med Care. 1991;29(2):169–176.
25. Keller SD, Ware JE Jr, Hatoum HT, Kong SX. The SF-36 Arthritis-Specific Health Index (ASHI): II. Tests of validity in four clinical trials. Med Care. 1999;37(suppl 5), MS51–MS60.
26. Rampal P, Martin C, Marquis P, Ware JE Jr, Bonfils S. A quality of life study in five hundred and eighty one duodenal ulcer patients. Maintenance versus intermittent treatment with nizatidine. Scand J Gastroenterol Suppl. 1994;206:44–51.
27. Cleary PD, Greenfield S, McNeil BJ. Assessing quality of life after surgery. Control Clin Trials. 1991;12(suppl 4):198S–203S.
28. Kantz ME, Harris WJ, Levitsky K, Ware JE Jr, Davies AR. Methods for assessing condition-specific and generic functional status outcomes after total knee replacement. Med Care. 1992;30(suppl 5): MS240–MS252.
29. Sullivan LM, Dukes KA, Harris L, Dittus RS, Greenfield S, Kaplan SH. A comparison of various methods of collecting self-reported health outcomes data among low-income and minority patients. Med Care. 1995;33(suppl 4):AS183–AS194.
30. Stewart AL, Greenfield S, Hays RD, et al. Functional status and well-being of patients with chronic conditions. Results from the Medical Outcomes Study. JAMA. 1989;262:907–913.
31. Freeman, Sullivan & Company. Survey operations report: diabetes patient survey. Prepared for the California Collaborative to Improve Diabetes Management. San Francisco: Freeman, Sullivan & Co., 1999.
32. Greenfield S, Kaplan SH, Silliman RA, et al. The uses of outcomes research for medical effectiveness, quality of care, and reimbursement in type II diabetes. Diabetes Care. 1994;1(suppl):32–39.
33. Kalton G, Kasprzyk D. The treatment of missing survey data. Survey Methodology. 1986;12:1–16.
34. American Diabetes Association, Diabetes Quality Improvement Project, Initial Measurement Set (Final), Alexandria, Va. American Diabetes Association, August 14, 1998.

Corresponding Author:
Jeffrey Wasserman, Ph.D.
The Medstat Group
5425 Hollister Avenue, Suite 140
Santa Barbara, Calif. 93111-2348
Email: jeffrey@rand.org [1]