MANAGED CARE July 2001. ©MediMedia USA
Abstract
Purpose: Describe patient satisfaction and patient-reported outcomes after voluntary use of a telephone-based nurse triage service.
Methods: A random sample of symptomatic callers who contacted the triage service in 1999 was identified. A computer-assisted telephone survey was conducted, resulting in a response rate of 58.9 percent and a sample size of 35,374.
Summary: Overall satisfaction with the service was 90.4 percent and did not vary greatly when stratified by demographic and health-status characteristics. Of all callers who reported following the triage recommendation to use self-care instructions while monitoring the condition for change (n=12,037), 11.5 percent scheduled an office visit and 1.5 percent used hospital emergency-room (ER) services for further care.
Conclusions: Overall satisfaction with telephone-based nurse triage services was high and did not vary substantially by caller characteristics.
Key terms: nurse triage; program evaluation; satisfaction; outcomes
INTRODUCTION
In recent years, consumer use of internet and telephone-based services to obtain health information and advice has increased dramatically. As the use of such services continues to rise rapidly, it becomes increasingly important to improve understanding of these services. This study examines satisfaction with a telephone-based nurse triage service and describes self-reported patient outcomes.
Over 75-million people in the United States, as well as many people in other countries, have access to telephone triage services.1-7 Health plans and employer groups provide these services to improve access to medical information and advice. Triage nurses typically use clinical tools to assess callers' acute medical complaints and to make recommendations as to the level of care that is needed and how soon treatment should be sought. Use is voluntary, and the majority of calls occur during evenings and weekends.
Although high levels of satisfaction with nurse triage services have been reported previously, only one study was of a United States population.8-11 Due to limited sample sizes, no studies to date have stratified satisfaction by demographic and health characteristics that may have an effect on patient-caller satisfaction.8,9,11 Moreover, information describing caller outcomes (e.g., agreement and compliance with the nurse-recommended course of care, post-call utilization of medical services, and medical-complaint resolution) is limited.
Methods
The data in this study include a random sample of nurse triage callers from 40 health plans located throughout the United States. The health plans contracted with one provider of telephone-based nurse triage services and agreed to participate in the outcomes survey. Health plan members were able to contact the service 24 hours a day, 7 days a week at no cost through toll-free numbers. The primary reason to contact the triage service is to have a registered nurse assess medical symptoms; these callers are classified as symptomatic.
The triage service's clinical-decision architecture uses more than 570 symptom-based algorithms to evaluate callers' medical complaints and suggest appropriate management. Other callers contact the nurse triage service to obtain provider referrals, general health information, or information concerning specific health plan coverage and services.
Participation in the study was limited to symptomatic callers. Biweekly, data on a random sample of symptomatic callers were generated and sent to an independent research company throughout 1999. Specially trained representatives contacted callers within two to four weeks of having used the nurse triage service. Interviewers conducted a computer-assisted telephone survey that was monitored to ensure quality. In addition, health plan staff were able to monitor survey administration.
The research firm attempted to contact triage service users a minimum of five times during a two-week period, including Saturdays, and at various times of the day. The survey methodology resulted in an overall reach rate of 58.9 percent. Representatives attempted to call triage service users at the phone number from which they contacted the nurse triage service. Since callers may have been at another location when they contacted the nurse triage line, attempts to reach some callers were unsuccessful. Thirty percent of callers were not contacted during any of the five attempts made, and seven percent declined to participate when contacted. Data for 35,871 symptomatic callers were obtained.
Those surveyed were asked to respond to a series of questions concerning satisfaction with the nurse triage service, agreement and compliance with the algorithm recommended course of care, and the level of medical service sought after the call. Health plan administrators participated in the development of the survey instrument. If a person participated in the survey but did not answer the first two questions concerning overall satisfaction and satisfaction with the nurse's medical knowledge, the individual's data were excluded from the analysis (n=497). Otherwise, differences in the reported sample size for each question indicate that some people were nonrespondents for a particular question.
Interviewers spoke directly with the health plan member who contacted the triage service. In most instances (n=21,651), people called about their own health. A person could have called, however, about a medical complaint of their child (n=5,714), parent (n=193), spouse (n=1,003), grandparent (n=26), or a person with another type of relationship to the caller (n=400).
Survey responses pertaining to satisfaction were recorded using a scale of 1 to 5, where 1 represents "very dissatisfied" and 5 represents "very satisfied." Only responses of 4 ("satisfied") and 5 ("very satisfied") were considered to express caller satisfaction. An alpha code was assigned to responses given for agreement and compliance with the algorithm recommendation and resolution of medical condition.
Caller satisfaction is reported by a number of patient characteristics. Information concerning age, gender, and the caller/patient relationship are routinely collected during the nurse-caller interaction. Data concerning caller race/ethnicity and educational attainment was obtained during survey administration. Due to the sensitivity of the questions, several health plans requested that this demographic data only be gathered for four weeks of the yearlong survey. Therefore, caller demographic data on race/ethnicity and education level were voluntarily collected for roughly 8,000 callers during 1999. Information concerning type of plan coverage was available for 17,409 callers, due to variations in the coding of different types of plan coverage in the triage database.
Satisfaction was also evaluated by caller-redirection status. Triage callers are provided with a recommended course of care that includes a referral to the ER or urgent care center; to schedule a physician appointment; to speak with the on-call medical provider; or to use self-care instructions while monitoring the patient's status for any changes.
During triage assessments, callers are asked what type of care they would have sought if the triage service had not been available to them. Depending on symptom severity, the algorithm may recommend a more acute, similar, or less-acute level of care than the patient had intended to seek. If the algorithm recommendation differs from the caller's intended approach to obtaining medical care, this is called redirection. We used these data to describe satisfaction by type of caller redirection.
Logistic-regression analysis was used to determine the statistical significance of variations in satisfaction by caller characteristics. Three models were estimated, due to data limitations for some characteristics. The first model, the base model, included age, gender, patient/caller relationship, triage recommendation, and redirection as independent characteristics. The second model included the base model with the addition of race/ethnicity and education, and the third model included the base model with type of coverage.
Results
The satisfaction survey sample of 35,374 callers was similar with regard to age, gender, and patient-caller relationship to the overall population of callers in 1999. The top-ten algorithms used to triage symptomatic callers in the sample included: upper respiratory illness, abdominal pain, trauma to an extremity, insect bite, vomiting, diarrhea, skin-wound problems, skin rash, sore throat, and cough.
Overall satisfaction with the nurse triage service was high. More than 90 percent of callers reported being "satisfied" or "very satisfied" with the care received. Nearly three-fourths of all respondents reported being "very satisfied." (See Table 1.)
| Table 1. Caller satisfaction with the nurse triage service | |||||
|---|---|---|---|---|---|
| Survey Item | Sample (n) | Frequency | Percent | ||
| Q1. | Overall satisfaction with care received from nurse (responses 4 and 5) | 35,374 | 31,962 | 90.4 | |
| By actual response | |||||
| 5 = "very satisfied" | 25,616 | 72.5 | |||
| 4 = "satisfied" | 6,346 | 18.0 | |||
| 3 = "indifferent" | 2,134 | 6.0 | |||
| 2 = "dissatisfied" | 670 | 1.9 | |||
| 1 = "very dissatisfied" | 581 | 1.6 | |||
| Q2. | Satisfaction with nurse's medical knowledge | 35,374 | 32,268 | 91.3 | |
| Q3. | Satisfaction with nurse's understanding of condition | 35,116 | 32,276 | 91.9 | |
| Q4. | Comfort level speaking to nurse | 35,165 | 32,897 | 93.6 | |
| Overall satisfaction (Q1) stratified by patient characteristics | |||||
| Patient Age (n=32,130) | |||||
| 0–4 | 5,512 | 5,003 | 90.8x | ||
| 5–17 | 4,558 | 4,163 | 91.3x | ||
| 18–44 | 12,417 | 11,126 | 89.6 | ||
| 45–64 | 6,244 | 5,622 | 90.4 | ||
| 65 + | 3,399 | 3,161 | 93.0 | ||
| Patient Gender (n=31,625)x | |||||
| Male | 10,777 | 9,669 | 89.7x | ||
| Female | 20,848 | 18,907 | 90.4 | ||
| Caller Ethnicity (n=7,976) | |||||
| Caucasian | 6,005 | 5,449 | 90.7 | ||
| African American | 1,321 | 1,198 | 90.7 | ||
| Hispanic | 268 | 244 | 91.0 | ||
| Asian | 122 | 104 | 85.3 | ||
| Other | 260 | 243 | 93.5 | ||
| Caller Educational Attainment (n=8,085) | |||||
| Less than high school, high school graduate, or GED | 3,008 | 2,774 | 92.2 | ||
| Some college | 2,284 | 2,074 | 90.8 | ||
| College graduate | 2,793 | 2,479 | 88.8 | ||
| Caller relationship to patient (n=28,987) | |||||
| Self | 21,651 | 19,552 | 90.3 | ||
| Parent | 5,714 | 5,160 | 90.3 | ||
| Spouse | 1,003 | 903 | 90.0 | ||
| Child | 193 | 176 | 91.2x | ||
| Other | 426 | 385 | 90.4 | ||
| Type of health insurance coverage (n=17,409) | |||||
| Point of Service | 1,521 | 1,386 | 91.1 | ||
| Preferred Provider Organization | 1,861 | 1,705 | 91.6 | ||
| Health Maintenance Organization | 10,761 | 9,724 | 90.4 | ||
| Medicare | 739 | 680 | 92.0 | ||
| Medicaid | 1,216 | 1,115 | 91.7y | ||
| Other | 1,311 | 1,183 | 90.2 | ||
| Algorithm group (n=6,732) | |||||
| Adult cardiovascular | 990 | 912 | 92.1 | ||
| Adult musculoskeletal | 1,693 | 1,508 | 89.1 | ||
| Adult gastrointestinal | 1,836 | 1,662 | 90.5 | ||
| Pediatric upper-respiratory illness | 1,988 | 1,783 | 89.7 | ||
| Seniors upper-respiratory illness | 225 | 208 | 92.5 | ||
| Triage recommendation (n=35,270) | |||||
| Go to the hospital emergency room | 654 | 600 | 91.7 | ||
| Use urgent care services | 4,895 | 4,430 | 90.5y | ||
| Make an appointment to see your medical provider | 5,091 | 4,535 | 89.1y | ||
| Call your medical provider | 11,472 | 10,273 | 89.6 | ||
| Use self-care recommendations at home | 13,158 | 12,055 | 91.6x | ||
| Caller redirection (n=27,810) | |||||
| Directed to a less acute level of care | 14,322 | 12,904 | 90.1x | ||
| Directed to the same level of care | 7,815 | 7,143 | 91.4 | ||
| Directed to a more acute level of care | 5,673 | 5,162 | 91.0 | ||
| y,xStatistically significant difference at the 0.05 and 0.01 levels, respectively. The reference group for age was 18-44 years; gender, males; caller relationship to patient, self; triage recommendation, call your provider; redirection, same level of care; race, Caucasian; education, some college and plan type, HMO. The statistical findings must be interpreted with caution, as some differences in satisfaction levels are statistically significant due in part to the large sample size for some characteristics; larger differences were not significant due in part to smaller sample sizes. The significance of algorithm type was not tested since most algorithms could not be grouped into meaningful categories. | |||||
When satisfaction levels were stratified by age, gender, caller relationship to patient, and race/ethnicity, satisfaction generally remained at or above 90 percent. Satisfaction for Asian callers was somewhat lower, however, at 85.3 percent. Although there were variations in satisfaction by caller educational level and by type of insurance coverage, satisfaction levels were 90 percent or above for all groups. (The statistical significance of these differences is presented in Table 1.)
Only minor differences in satisfaction were observed when satisfaction was stratified by type of medical complaint as assessed by the type of algorithm used during the triage call. At least 90 percent of callers triaged using the upper respiratory infection and abdominal pain algorithms were satisfied with the service. Similarly, callers triaged by the chest pain (92.1 percent) and musculoskeletal pain (89.1 percent) algorithms were satisfied with the nurse triage service. There was little variation in satisfaction by type of triage recommendation or by caller redirection. Approximately 90 percent of callers reported satisfaction with the service despite being redirected to a less-acute setting than they would have otherwise used.
Three indicators of triage quality were assessed. When callers were asked whether they followed the care-algorithm recommendation, 90 percent of callers (29,685 of 32,995 respondents) reported complying with the nurses' recommendations. Second, caller agreement with the triage recommendation was evaluated. Callers who reported they followed the recommendation were asked to review their triage call experience and rate (on the 1–5 scale) the appropriateness of the algorithm recommendation. Nearly all these respondents (95.2 percent) reported that the care-algorithm recommendation was appropriate.
The third indicator of triage quality related to use of medical services following a triage call. The investigators examined the follow-up care obtained by callers who reported following triage recommendations for less-acute types of medical care. Of the 3,754 persons provided the recommendation to make an appointment with their providers, only 41 people (1.0 percent) reported use of the ER, and three (0.07 percent) reported use of urgent care services. Six percent scheduled follow-up office visits with a physician.
Of the 12,037 persons who were provided with self-care recommendations, 11.5 percent (n=1,391) went on to schedule a visit with their provider, 1.5 percent (n=182) used the ER, and 0.2 percent (n=19) used urgent care services.
Roughly 10 percent of callers reported that they did not follow the algorithm recommendation. Reasons given for noncompliance are provided in Table 2. Twenty-eight percent of those who did not comply reported that they felt the patient was not as sick as the nurse had believed, while 16.2 percent felt the patient was sicker than the nurse had believed. Another 31.7 percent did not want to call their provider, could not contact their physician, or did not have time to follow the recommendation.
| Table 2. Reasons for noncompliance with algorithm recommendation | ||
|---|---|---|
| Number | Percent | |
| Felt patient was not as sick as nurse believed | 938 | 28.3 |
| Felt patient was sicker than nurse believed | 535 | 16.2 |
| Did not want to call provider | 528 | 16.0 |
| Wanted second opinion | 274 | 8.3 |
| Did not have time to follow recommendation | 275 | 8.3 |
| Could not contact physician | 244 | 7.4 |
| Did not want to go to the ER | 205 | 6.2 |
| Felt nurse did not have enough information to make a decision | 128 | 3.9 |
| Financial reasons | 95 | 2.9 |
| Does not have a regular provider to see/call | 88 | 2.7 |
| Total | 3,310 | 100.0 |
Callers who did not comply were asked how they resolved their medical conditions. Their responses closely matched their reasons for noncompliance, with the majority seeking care at a less-acute level than that recommended. Of those who were given the recommendation to see their provider, call their provider, or take care of their condition at home, 14.9 percent reported they sought services at an ER or urgent care center.
Discussion
Only one previous study of satisfaction with nurse triage systems in the United States had been conducted, and this had a relatively small sample size of 422 pediatric patients.11 In this paper, we have described the satisfaction of 35,374 callers of all ages who used a nurse service during 1999. The results of this study confirm previous findings concerning high levels of satisfaction with such services in the United States as well as other countries.8-11
Due to the large sample size, the investigators were able to examine satisfaction by demographic variables such as patient age and gender, caller educational attainment, ethnicity, patient-caller relationship, type of insurance plan, triage recommendation, and type of medical complaint. Although there was some minor variation, satisfaction rates remained generally at or above 90 percent when stratified by caller characteristics. Findings indicate that callers were satisfied even when directed to a lower level of medical care than they would have otherwise used.
The investigators assessed three indicators of patient-reported quality of care in this study: caller adherence to the triage recommendation, caller agreement that the recommendation was appropriate, and the use of medical services following the call.
Ninety percent of callers reported following the triage-recommended course of care and 95 percent still perceived the algorithm recommendation to have been appropriate when asked two to four weeks after the triage call. An analysis of the use of medical services after the call indicated that only an extremely limited number of callers sought additional medical care at a later date. Nevertheless, to continue to improve the quality of the triage service, there is a need to further understand the reasons callers/patients do not follow triage recommendations.
One limitation of this study is the response rate. Although a response rate of nearly 60 percent is reasonable for a telephone-based survey, respondents could have viewed the service more favorably than nonrespondents.
In addition, the self-reported compliance data could include an upward bias. The compliance question was worded: "I understand the triage nurse recommended you [triage recommendation]. Did you decide to [triage recommendation]?" Even given this wording, it is likely that reported compliance with the nurses' recommendation might be biased upwards.
Medical claims and encounter data were used to assess compliance rates for recommendations to use an ER, physician office services, and self-care advice for the same triage service.12 According to these data, which have a downward bias, approximately 66 percent of patients adhered to the recommendation. It may be that actual compliance lies between the two levels, given the biases of the two different types of data.
This study addresses satisfaction and self-reported outcomes for one nurse triage service. The quality of these services varies due to the clinical rigor of the decision-support tools that range from loosely structured, paper-based guidelines to peer-reviewed, computerized algorithms, as well as nursing training, triage quality monitoring activities, and the frequency of algorithm updates.2,13-15
The results indicate that telephone-based triage services may be a useful method of providing health care information and advice, especially during evenings and weekends. Additional research is needed, however, to further assess patient compliance with telephone-based nurse advice and to clinically evaluate triage quality.
REFERENCES
- Emerging Market Trends. Strategic Briefings in Telemedicine. Jacksonsonville, Ore: Feedback Research Services; 1999;2(6): 7–10
- Lattimer V, George S, Thompson F, Thomas E, Mullee M, Turnbull J, et al. Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial. BMJ. 1999;317:1054–1059.
- McCarthy R. It takes more than a phone call to manage demand. Business and Health. 1997;5:36–41.
- Sher C, Mimouni M, Weitzen T, Waisman Y. Pediatric telephone advice: a new medical service in Israel. Isr J Med Sci. 1994;30:651–666.
- Marklund B, Bengtsson C. Medical advice by telephone at Swedish health centres: who calls and what are the problems? Fam Practice. 1989;6(1):42–46.
- Christensen M, Olesen F. Out of hours service in Denmark: evaluation five years after reform. BMJ. 1998;613:1502–1504.
- Evans RJ, McCabe M, Allen H, Rainer T, and Richmond PW. Telephone advice in the accident and emergency department: a survey of current practice. Arch Emer Med. 1993;10:216–219.
- Gallagher M, Huddart T, Henderson B. Telephone triage of acute illness by a practice nurse in general practice: outcomes of care. Br J Gen Pract. 1999;48(429):1141–1145.
- Jones K, Gilbert P, Little J, Wilkinson K. Nurse triage for house call requests in Tyneside general practice: patients' views and effect on doctor workload. Br J Gen Pract. 1999;48:1303–1306.
- Dale J, Crouch R, Patel S, Williams S. Patients telephoning A&E for advice: a comparison of expectations and outcomes. J Accid Emerg Med. 1997;14(1):21–23.
- Poole SR, Schmitt BD, Carruth T, Person-Smith A, Slusarski M. After-hours telephone coverage: The application of an area-wide telephone triage and advice system for pediatric practices. Pediatrics. 1993;92(5): 670–679.
- O'Connell JM, Towles W, Yin M, Malakar CL. Patient adherence to telephone-based nurse triage recommendations. Unpublished manuscript, 2001.
- Salk ED, Schriger DL, Hubbell KA, Schwartz BL. Effect of visual cues, vital signs, and protocols on triage: a prospective randomized crossover trial. Ann Emerg Med. 1999;32(2):655–664.
- Sullivan G. Advice or diagnosis? A legal perspective. Business and Health. 1997;5:40–42.
- Wolcott B. Putting clinical pathways to work: using algorithms. In: Howe R, editors. Clinical Pathways for Ambulatory Care Management. Gaithersburg, MD: Aspen Publishers, Inc.; 1996: p. 1–7.
Author correspondence:
Joan M. O'Connell, Ph.D., M.H.S.
Director, Health Information
Anthem Blue Cross Blue Shield
700 Broadway
Denver, CO 80273
Phone: (303) 831-3254
E-mail: joan.o'connell@anthem.com