Objective  To determine the patient, system, and physician factors that may explain why physicians may not follow guidelines.

Methods  We used chart review and physician surveys to measure adherence with an actively implemented guideline to reduce hospitalizations for patients coming to the emergency department with community-acquired pneumonia. Logistic regression analyses were used to identify factors associated with guideline nonadherence.

Results  Overall nonadherence with the guideline was 43.6%, with 71 of 163 low-risk patients with pneumonia being hospitalized despite the recommendation for outpatient therapy. In univariate analyses, nonadherence to the guideline was more likely for patients who were aged 65 years or older, were male, were employed, and had multilobar disease or other comorbid conditions (P<.05). Active involvement of a primary care physician in the admission decision also increased nonadherence (odds ratio, 4.9; 95% confide0nce interval, 2.2-11.0). Physicians with more pneumonia experience were more likely not to follow the guideline (P<.001). In multivariate models, the odds of nonadherence were 2 to 3 times greater when patients were 65 years or older, were male, or had multilobar disease, or the primary care physician was involved in the triage decision (P<.05). Physicians' reasons for admission were the presence of active comorbidities (55%), the primary care physician's wish for hospitalization (41%), the presence of worse pneumonia than the guideline indicated (36%), patient preference (17%), and inadequate home support (16%).

Conclusions  Nonadherence to a pneumonia guideline was associated with a variety of patient, system, and physician factors. Guideline implementation strategies should take into account the heterogeneous forces that can influence physician decision making.

Despite the fact that physician adherence with guidelines is critical to their ultimate success, there have been few published studies of the factors that influence such behavior. Several classes of factors have been suggested to explain nonadherence with guidelines, including applicability to individual patients,^{6, 10} scientific validity,^11-13 usability in real time,^{7, 13} inefficiencies of the health care system,^{10, 14} level of local participation and endorsement by opinion leaders,^12-13,15 patient preferences,⁶ and physicians' knowledge, attitudes, and beliefs.^{13, 16} Guideline misclassification errors have also been shown to be important potential sources of apparent nonadherence.¹⁴ Efforts to improve the effectiveness of guidelines will depend critically on the ability to identify and modify the factors that influence nonadherence.

We prospectively studied adherence with an emergency department practice guideline that sought to identify low-risk patients with community-acquired pneumonia who could be treated as outpatients.¹⁷ The purpose of this study was to (1) identify patient, system, and physician factors that influenced guideline nonadherence; and (2) compare physicians' self-reported reasons for nonadherence with objectively determined reasons for nonadherence.

PATIENTS AND METHODS

This study was done as part of an intervention trial evaluating the impact of a practice guideline to safely decrease hospitalizations for patients coming to the emergency department (ED) of Massachusetts General Hospital, Boston, with community-acquired pneumonia. Previous studies suggest that many patients who are hospitalized for pneumonia are at very low risk of adverse events and could be safely treated as outpatients.¹⁸ On the basis of this literature, we developed a guideline to decrease unnecessary hospitalizations for low-risk patients with pneumonia.

The guideline identified low-risk adult patients (aged 18-84 years) through 3 steps. First, patients had to have community-acquired pneumonia supported by history, symptoms, and chest radiographic findings. Second, patients who were considered inappropriate candidates for outpatient therapy were excluded (criteria included oxygen saturation less than 90% on room air, long-term dependence on supplemental oxygen therapy, inability to take oral medications, human immunodeficiency virus disease, immunosuppression, recent hospitalization, nursing home residence, injection drug use, neuromuscular disease, pregnancy, psychiatric disease, substance abuse, homelessness, or lack of a telephone). Third, a validated prediction rule was used to identify low-risk cases according to the Pneumonia Severity Index (PSI).¹⁸ Eligible patients with a PSI score of 90 or less (30-day mortality risk, <2.8%) were considered truly "low risk" and were the target group for this intervention. The PSI is a pneumonia severity measure based on patient age, sex, selected comorbid conditions, and vital signs and laboratory values at initial examination. The guideline instructed physicians to use their clinical judgment to decide if the recommendation for outpatient treatment was appropriate for their patient.

PATIENT SELECTION

From April 1, 1996, through February 28, 1997, 826 consecutive patients coming to the ED with pneumonia were screened. Of these, 576 were excluded because they were not candidates for outpatient therapy according to the guideline. The most common reasons for exclusion were hypoxemia (n = 220), age greater than 84 years (n = 123), inability to take oral medications (n = 119), and recent hospitalization (n = 98). Of the remaining 250 eligible patients, 166 were identified as low risk by the PSI score (90). Three patients were misclassified as low risk (a 1.8% error rate) because of missed exclusion criteria (methotrexate use, surreptitious intravenous drug use, and long-term nocturnal oxygen therapy). These 3 cases were excluded from our analyses of nonadherence. As we previously reported, the guideline increased the proportion of patients initially treated as outpatients from 41.5% in the control period to 56.6% in the guideline intervention period (36.4% relative increase; 95% confidence interval [CI], 8%-72%; P = .01).¹⁷ Patients treated during the guideline period had similar rates of symptom resolution and functional recovery compared with controls. There were no deaths in the 30-day follow-up period (95% CI for mortality rate, 0%-2.2%).

STUDY SITE CHARACTERISTICS

Massachusetts General Hospital is a large, urban teaching hospital with approximately 63 000 ED visits and 800 pneumonia admissions each year. The ED is staffed with 22 attending physicians, as well as residents in internal medicine, emergency medicine, and surgery.

GUIDELINE EDUCATION AND IMPLEMENTATION

The guideline was approved by a multidisciplinary group of opinion leaders. Awareness of the guideline was supported by face-to-face educational sessions for ED attending physicians and house officers. In addition, the guideline was actively implemented in the ED by a dedicated study nurse who screened all patients with pneumonia, determined their risk status, and alerted the ED physicians if patients were low risk according to the guideline. To further facilitate outpatient therapy, patients could be sent home with a course of oral clarithromycin (Biaxin, 500 mg twice daily for 10 days), receive home nurse visits 24 and 48 hours after discharge (if deemed appropriate by the physician), and have a follow-up outpatient physician visit arranged for them. The antibiotic was provided at no cost to the patient. Visiting nurse services were ordered for 27% of outpatients. The study nurse was available weekdays from 8 AM to 6 PM. At night and on weekends, guideline implementation was the responsibility of the ED physicians. All decisions about appropriateness for outpatient therapy and choice of antibiotic agent were made by the ED physicians in consultation with the patient's primary care physician (if available). The study was approved by the Massachusetts General Hospital Subcommittee on Human Studies.

BASELINE DATA

Information on sociodemographic characteristics, vital signs, physical findings, pneumonia severity, comorbidities, laboratory values, radiographic findings, symptoms, functional status, and triage decision (outpatient vs inpatient care) was collected from medical records. System factors, such as when the patient came to the ED (time of day, day of the week) and whether the patient had a primary care provider, were also noted.

PHYSICIAN SURVEY DATA

Before implementing the guideline, we asked all attending physicians (n = 22) and house officers (n = 138) who staff the ED to complete a 4-page baseline questionnaire. This included questions about demographics, training, experience treating pneumonia, practice style (estimated percentage of their patients with pneumonia they admit to the hospital), and attitudes about pneumonia care. We used previously validated questionnaires to assess general attitudes about practice guidelines¹⁶ and risk avoidance.¹⁹ Once the decision to admit or discharge an eligible patient was made, a 3-page encounter survey was given to the ED physician(s) caring for the patient. This survey assessed the participants in the triage decision and the level of agreement among them. If the patient was admitted, we asked physicians to identify and rate the importance of a variety of factors that may influence the hospitalization decision based on a previously published survey and an informal focus group.²⁰ We received completed encounter survey data for 139 (85.3%) of 163 cases.

STATISTICAL ANALYSES

The triage decision was the primary unit of analysis. Nonadherence was defined as cases in which the patients were admitted despite the guideline recommendation for outpatient therapy. We used ², t, and Fisher exact tests to identify patient and system variables that were associated with nonadherence. Because many physicians examined more than 1 study patient (median, 5 patients per ED physician), we used general estimating equation (GEE) models to account for the clustering structure of data to examine the effect of physician-level variables on nonadherence.²¹ We considered the attending physician to be the primary decision maker. In 27 (16.6%) of 163 cases, we did not have completed survey data from the attending physician, so we used the house officer's responses.

We used multivariate logistic regression with GEE models to control simultaneously for the influence of patient, system, and physician factors on nonadherence. Our multivariate models were built in 2 steps to help deal with collinearity between related variables. First, we developed a multivariate GEE model for each of the 4 classes of variables (sociodemographic, clinical, system, and physician factors), entering all variables within a class that were significant at the P<.10 level in the univariate analysis.²² All variables significant at the P<.10 level from the separate sociodemographic, clinical, system, and physician models were entered into a final multivariate regression equation (age 65 years, sex, Medicaid, PSI 71, multilobar disease, 1 comorbidity, active involvement of a primary care physician, physician's pneumonia experience, and agreement with the statement "Most patients would prefer to be treated as outpatients"). Alternate stepwise and backward elimination regression models using all univariate predictors significant at the P.10 level produced similar results. Because of small cluster size, we report odds ratios (ORs) derived from model-based GEE estimates. Two-sided P values of .05 or less were considered statistically significant. All analyses were performed with SAS 6.12 statistical software (SAS Institute Inc, Cary, NC).

RESULTS

PATIENT PREDICTORS OF NONADHERENCE

Sociodemographic factors associated with admission despite the guideline recommendation included age, sex, insurance status, and employment (Table 1). Patients 65 years or older had nearly 4 times greater odds of nonadherence (OR, 3.9; 95% CI, 2.0-7.4). Patients who were employed or had Medicaid insurance were less likely to be hospitalized. There were no significant effects of race, living arrangements, education level, or marital status (data not shown). Specific clinical features were also important. Although all patients were low risk according to the guideline, the odds of nonadherence were 2.6 times greater among patients with worse pneumonia (PSI, 71-90) compared with the lowest-risk cases (PSI, 70) (95% CI, 1.3-5.1). Low-risk patients with multilobar pneumonia were more than 4 times more likely to have been admitted (OR, 4.2; 95% CI, 1.6-11). The presence of another coexisting medical problem more than doubled the odds of nonadherence (OR, 2.3; 95% CI, 1.2-4.5). Among these, concomitant chronic obstructive pulmonary disease, congestive heart failure, coronary artery disease, and hypertension each increased the odds of hospitalization 3- to 6-fold.

View this table: [in this window] [in a new window] Table 1. Predictors of Nonadherence With Pneumonia Triage Guideline*

SYSTEM PREDICTORS OF NONADHERENCE

Having a primary care physician more than doubled the chance of admission (OR, 2.4; 95% CI, 1.3-4.5). The odds of nonadherence went up by a factor of 5 when the primary care physician was actively involved in the hospitalization decision (OR, 4.9; 95% CI, 2.2-11.0). There was a trend toward fewer unnecessary hospitalizations among patients who came to the ED at night or on weekends. However, decision making was not influenced by disagreement among physicians, disagreement between physicians and patients, or poor timing of the guideline implementation (in only 7 cases did the physicians say the information was given too late).

PHYSICIAN PREDICTORS OF NONADHERENCE

We received baseline surveys from 140 of 160 ED physicians (87.5% response rate). Eighteen were attending physicians (81.8% of the 22 attending physicians) and 122 were house officers (88.4% of the 138 house staff). There were no significant differences between responders and nonresponders with regard to sex, years in practice, and training. We had completed survey data from the attending physician for 136 (83.4%) of 163 cases. We used GEE techniques (which adjust for clustering of multiple patients by physician) to assess physician-level effects. Physicians who reported conservative attitudes about pneumonia, as measured by agreement with statements such as "all patients greater than 65 years old should be admitted" and "most patients with pneumonia have a greater than 15% chance of death within 30 days" were more likely to avoid adhering to the guideline, although the differences were not statistically significant (Table 2). Paradoxically, physicians who agreed that "most patients would prefer treatment as outpatients" were more likely to admit patients. On average, physicians believed that patients with a 30-day mortality risk (±SD) of 7.2% ± 5.1% or more should be hospitalized. However, self-reported mortality risk thresholds for hospitalization were not associated with admission decisions (P = .82). Nor did we find an association between nonadherence and our measures of general attitudes about practice guidelines, risk avoidance, or familiarity with the guideline (data not shown). Knowledge about the guideline seemed adequate, as none of the attending physicians said they were "not at all familiar" with the protocol. Our results were the same whether we considered the ED attending physician or the house officer as the primary decision maker.

View this table: [in this window] [in a new window] Table 2. Associations Between Guideline Nonadherence and Physician's Attitudes About Pneumonia Care*

Among the other physician variables we examined, only experience treating pneumonia was associated with nonadherence. Emergency department attending physicians who reported treating more cases of pneumonia in the past 12 months were more likely to diverge from the guideline recommendations (mean, 59.2 cases by physicians who did not follow the guideline recommendation vs 46.6 cases for physicians who did; P<.001). There were no significant associations with physician sex, years in practice, emergency medicine training, time spent in direct patient care, or estimated proportion of pneumonia cases admitted.

MULTIVARIATE PREDICTORS OF NONADHERENCE

To adjust simultaneously for significant patient, system, and physician factors that influenced decision making, we performed multivariate logistic regression with GEE techniques. The independent multivariate predictors of nonadherence are displayed in Table 3. Patients who were 65 years or older, had multilobar disease, or were male had 2 to 4 times greater odds of admission despite the guideline recommendation. There was a trend toward patients with other comorbid conditions having twice the likelihood of admission (OR, 1.9; 95% CI, 0.9-4.2). Finally, nonadherence was 3 times more likely when the primary care physician was actively involved in the triage decision (OR, 3.0; 95% CI, 1.2-7.1).

View this table: [in this window] [in a new window] Table 3. Multivariate Predictors of Nonadherence With the Guideline*

PHYSICIAN REASONS FOR ADMISSION

The ED physicians interacted actively with the guideline (spoke with the study nurse or completed on their own the risk algorithm) in 53 of the 71 cases in which the patient was admitted. In these 53 instances, we also asked the ED physician to complete a 1-page survey regarding the reasons for admission. We received completed questionnaires for 47 (89%) of 53 cases. The ED physicians' self-reported reasons for admission are shown in Table 4. More than 1 reason per patient could be given. More than half of all patients (55%) were admitted because of other active comorbidities, such as congestive heart failure, angina, and chronic obstructive pulmonary disease. Deference to the primary care physician's (or covering provider's) desire for admission was the second most common reason, occurring 41% of the time. In 36% of cases, physicians said the patient's pneumonia was worse than the PSI score indicated. Patient age, fever, abnormal blood pressure, and multilobar disease were some of the specific factors indicated by physicians to put patients at higher risk than the guideline specified. Less common reasons for admission included patient preferences, inadequate home care support, doubts about patient reliability, and failure of oral therapy. There were no significant differences in the stated reasons for hospitalization regardless of whether the respondent was an attending physician or house officer.

View this table: [in this window] [in a new window] Table 4. Physicians' Reasons for Admitting Patients Despite the Guideline Recommendation for Outpatient Care (n = 47)*

COMMENT

Although all patients identified by the guideline were low risk according to a validated prediction rule, those who were elderly, had worse pneumonia (multilobar disease or higher PSI scores), or had other comorbid conditions had 2 to 4 times greater odds of being admitted. We have several possible explanations for these results. First, half of the patients admitted despite the guideline recommendation were hospitalized for management of an active comorbid condition. Most of these cases likely represent clinically appropriate overrides of the guideline. Second, the risk of dying of pneumonia is an important factor in the hospitalization decision, but not the only one. Older and sicker patients may be admitted for administration of parenteral antibiotics, close observation for clinical deterioration, or general nursing.²⁰ Although our offer of enhanced visiting nurse services was intended to provide some of this extra "caring" and observation, it may not have been comprehensive enough to be a readily acceptable alternative to hospitalization. Whereas low-risk patients with relatively higher PSI scores (71-90) were more likely to be hospitalized, the guideline achieved the greatest relative reduction in the chance admission among these higher-risk patients.¹⁷ Third, clinicians may overemphasize the prognostic importance of certain clinical variables (such as older age) on the basis of traditional teaching or anecdotal experience. Nearly all the factors that physicians said were inadequately accounted for by the guideline (such as age, vital signs, and comorbidities) were, in fact, specifically part of the PSI-based guideline algorithm. Thus, despite our educational efforts, physicians may not have fully accepted the validity of the guideline for certain patients, especially those older than 65 years.

THE PRESENCE and active involvement of the patient's primary care physician were the main system factors that influenced decision making, even after controlling for the fact that patients with regular physicians tended to be older and sicker. In a poststudy focus group, the ED attending physicians reported that patients were often sent to the ED "to be admitted" after seeing or speaking with their primary care physician. Therefore, involvement of a primary care physician in the triage decision may be a marker of heightened patient and physician expectation of hospitalization. We had expected less adherence with the guideline at night and on weekends when the implementation nurse was not present. Remarkably, this was not the case. This suggests that we produced a sustainable change in local practice style and that 24-hour-a-day, 7-day-a-week active implementation (which is extremely costly) may not be necessary to produce marked behavioral change.

We found mixed evidence to support the hypothesis that physician characteristics, experience, and attitudes are important influences on clinical decision making.^{19, 25-28} There was no association between nonadherence and generic physician variables such as sex, specialty training, years in practice, or general attitudes about guidelines or risk tolerance. However, we did find some evidence that disease-specific attitudes and experience were correlated with behavior. Physicians with conservative attitudes about pneumonia care tended to be less likely to follow the guidelines, while those with less experience treating pneumonia were more likely to adhere to them. In addition, residents, who had less experience treating pneumonia than attending physicians, were more likely to report that the guideline was helpful to their decision making.²⁴ Guidelines may be most influential when clinicians are less experienced and/or lack a dominant practice style. Thus, long-held beliefs may be difficult to modify even with rigorously implemented, evidence-based guidelines.

This is the first study, to our knowledge, to assess the independent effects of a broad variety of patient, system, and physician factors on guideline adherence. Ellrodt and colleagues¹⁴ found that misclassification errors and system inefficiencies were the main reasons for nonadherence with a guideline to reduce hospital length of stay in patients with chest pain. Active screening of appropriate patients for our pneumonia guideline by the ED study nurse probably explains our very low misclassification rate. Previous studies of the influence of risk attitudes on clinical decision making have conflicting results.^{19, 25, 28-29} The most comparable studies found that risk avoidance was significantly associated with rates of admitting ED patients with acute chest pain¹⁹ and resource use in health maintenance organizations.²⁵ Since we used the identical risk attitude instrument, our divergent findings may reflect different considerations in the management of pneumonia compared with chest pain (or overall care of patients in health maintenance organizations). This was part of our rationale for assessing disease-specific attitudes and beliefs.

Our findings have broader policy implications. Adherence to guidelines will vary considerably depending on the nature of the guideline, specific clinical problem, patient group targeted, and mode of implementation. In certain circumstances, nonadherence may be the most appropriate course of action. At other times, nonadherence may reflect differences in physician knowledge, attitudes, and beliefs, as well as system inefficiencies, skewed incentives, organizational culture, or deference to primary care physician or patient preferences. To overcome these potential barriers, guidelines should be evidence-based, clinically sensible, usable in real time, and flexible enough to allow for individual clinical judgment. In addition, they should be aligned with other practice incentives. When guidelines are intended for patients likely to have multiple medical problems, there may be a limit on the maximum achievable adherence. Finally, because patient and system factors can critically influence clinical care, raw rates of guideline adherence are unlikely to be valid measures for profiling physician performance in the absence of adequate risk adjustment or consideration of nonclinical issues.

Several limitations are worth noting. First, our sample size was modest, limiting our ability to detect weak associations. Second, the small number of attending physicians limited our ability to detect physician-level influences, because the GEE models that took into account the clustering of patients by physician produced wide CIs. Similarly, the small number of study patients per physician (median, 5) may have diminished our ability to establish stable, physician-specific estimates of guideline nonadherence. Third, although our regression analyses appeared to corroborate the physician-reported reasons for nonadherence, we did not rate the medical "appropriateness" of admissions in any systematic way. Fourth, there were no financial incentives for the ED physicians to follow the guidelines. Traditional preferences for hospitalizing patients are likely to change markedly as capitated reimbursement becomes more common and economic incentives favor outpatient treatment. Finally, although psychosocial factors such as inadequate social support were infrequently cited as reasons for nonadherence, patients with psychiatric illness, substance abuse, or homelessness were explicitly excluded from our study.

In conclusion, nonadherence to a pneumonia practice guideline was associated with a variety of patient, system, and physician factors. To increase the likelihood of success, guideline implementation strategies should take into account the heterogeneous forces that can influence medical decision making. In the case of our pneumonia guideline, providing physicians with additional information about the safety of treating older patients in the ambulatory setting, as well as intermediate treatment options, such as a subacute care stay or substantially enhanced outpatient treatment (home nursing and/or parenteral antibiotics), may help to further reduce the use of acute care hospital services for older or sicker but still low-risk patients.

This research was supported in part by National Research Services Award grant PE11001-08 from the National Institutes of Health, Bethesda, Md (Dr Halm), and Abbott Laboratories Inc, Abbott Park, Ill. Our funding sources had no role in the acquisition, analysis, or interpretation of data, or in the writing of this report.

An earlier version of this work was presented at the Society of General Internal Medicine meeting, Chicago, Ill, April 25, 1998.

We are indebted to Diann Burnham, RN, for implementation of the guideline. We also wish to thank all of the participating emergency department attending physicians, house officers, and nurses for their invaluable cooperation.

Reprints: Ethan A. Halm, MD, MPH, Department of Health Policy, Box 1077, Mount Sinai Medical Center, One Gustave L. Levy Place, New York, NY 10029 (e-mail: ethan.halm{at}mountsinai.org).