 |
|
Improving Preventive Care by Prompting Physicians
E. Andrew Balas, MD, PhD;
Scott Weingarten, MD;
Candace T. Garb, BS;
David Blumenthal, MD;
Suzanne Austin Boren, MHA;
Gordon D. Brown, PhD
Arch Intern Med. 2000;160:301-308.
ABSTRACT
| |
Objectives To assess the impact of prompting physicians on health maintenance, answer questions regarding the mode of delivery, and identify opportunities and limitations of this information intervention.
Methods Systematic electronic and manual searches (January 1, 1966, to December 31, 1996) were conducted to identify clinical trial reports on prompting clinicians. Three eligibility criteria were applied: (1) randomized controlled clinical trial, (2) clinician prompt, alert, or reminder in the study group and no similar intervention in the control group, and (3) measurement of the intervention effect on the frequency of preventive care procedures. Data were abstracted by independent reviewers using a standardized abstraction form, and quality of methodology was scored. A series of meta-analyses on triggering clinical actions was performed using the random-effects method. The statistical analyses included 33 eligible studies, which involved 1547 clinicians and 54,693 patients.
Results Overall, prompting can significantly increase preventive care performance by 13.1% (95% confidence interval [CI], 10.5%-15.6%). However, the effect ranges from 5.8% (95% CI, 1.5%-10.1%) for Papanicolaou smear to 18.3% (95% CI, 11.6%-25.1%) for influenza vaccination. The effect is not cumulative, and the length of intervention period did not show correlation with effect size (R = -0.015, P = .47). Academic affiliation, ratio of residents, and technique of delivery did not have a significant impact on the clinical effect of prompting.
Conclusions Dependable performance improvement in preventive care can be accomplished through prompting physicians. Vigorous application of this simple and effective information intervention could save thousands of lives annually. Health care organizations could effectively use prompts, alerts, or reminders to provide information to clinicians when patient care decisions are made.
INTRODUCTION
WITH THE growing emphasis on health maintenance and preventive care, physicians are increasingly expected to perform tasks that are unrelated to the acute problem of the patient. The most common reason that women give for not undergoing screening for breast cancer and cervical cancer is that it was not offered or recommended by their physicians, especially by the youngest group of internists and family practitioners.1 Various quality scorecard systems encourage plans and clinicians to use every episode of patient care to promote health maintenance.2 Based on the latest recommendations of the American Academy of Pediatrics, a visit to a physician for a child's broken arm becomes an opportunity to update immunization.3
Controversies exist regarding whether physician prompting affects clinical outcomes. Some consider prompts to be very effective in bringing physicians' attention to a necessary clinical action and to avoid medical errors due to information overload.4-5 Others note that prompts do not always produce the expected change, and some analyses indicated nil to moderate effect on clinical practices.6-7
Unfortunately, the significant benefits of prompting preventive care have never been persuasively communicated to clinicians and health care plans. Researchers also continue placing people in control groups of prompting trials, raising serious ethical questions. Conventional, nonquantitative reviews have failed to highlight that thousands of lives could be saved by prompting.
Conventional approaches to research synthesis, such as reviews or odds ratio calculations, cannot specify the change that results from prompting preventive care. Furthermore, most health services research studies have been conducted in an academic environment, raising questions about the generalizability of results.8 This study highlights opportunities to elevate preventive care, use diverse and inexpensive technologies for prompting, and improve health plan scorecard performance. The systematic review of studies is supplemented with a meta-analysis of the effect size, critical information for designing quality improvement programs. The objectives of this study were (1) to quantify the impact of clinician prompting on the provision of preventive care and (2) to identify the effect of various covariates (reimbursement type, clinician characteristics, clinician specialty, and computerization).
METHODS
Studies meeting the following criteria were eligible for inclusion in this series of meta-analyses: (1) randomized controlled clinical trial, (2) physician prompt in the study group and no similar intervention in the control group, and (3) measurement of the effect on the number of preventive care activities. Studies qualified for automatic exclusion by failing to (1) randomly assign subjects to intervention and control groups and (2) test the randomization by comparing the intervention and control groups at baseline. Preventive care actions can be administered by clinicians with a variety of specialties. Articles that involved clinicians with certain clinical specialties or that focused on unique preventive care actions (eg, alcohol abuse counseling) were not excluded.
COLLECTION OF REPORTS
Many articles eligible for this series of meta-analyses were already in place as part of the Columbia Registry. The Columbia Registry is a systematic collection of medical management trials at the University of Missouri, School of Medicine.9 Extensive searches were done to collect all additional relevant controlled clinical trial reports: bibliographic database retrievals (eg, MEDLINE, CINAHL, HEALTH), manual searches (monographs, unindexed publications, reference lists), and informal contacts. By mapping the eligibility criteria, the search strategy for the collection of trials on prompting included a combination of medical subject heading (MeSH) terms and textwords (TWs) for each eligibility criterion. The words are MeSH terms unless noted as TW. The criteria were (1) random (truncated TW), group (truncated TW), random allocation (MeSH and TW), randomized controlled trial (publication type), or clinical trial (publication type); (2) checklist (TW), encounter forms (TW), tags (TW), triggers (TW), reminder systems, alert, (TW), reminder (TW), leaflets (TW), stickers (TW), messages (TW), or tailored messages (TW); and (3) preventive health services, immunization, vaccination, smoking, smoking cessation, mass screening, mammography, prenatal care, hypertension, blood pressure, diabetes mellitus, alcoholism, substance-related disorders, vaginal smears, hypercholesterolemia, glaucoma, or occult blood.
SCORING AND ABSTRACTION
Two research associates (C.T.G. and S.A.B.) checked eligibility and abstracted information from the reports using standardized and reproducible methods. The methodologic quality of each of the eligible trial reports was evaluated using a validated clinical trial scoring system tailored for health services research trials.8 The scoring system awards a score on an arbitrarily selected scale from 1 to 100.8 Aspects of the studies scored included sample definition, testing randomization, intervention, effect variable definition, blinding, numeric table of effect variables, ratio of withdrawals, and analysis of effect variables.8 In scoring, technical aspects related to proper sampling are weighted heavily (10 points each).8 Since more than half of the items are study characteristics, as opposed to reporting characteristics, the minimum required score was set at 50 in this study. Subsequently, raw data were derived from all eligible articles (eg, site, patient sample, clinician sample, intervention, and effects). These covariates were required for a study to be included. Authors were also contacted if additional information was needed.
STATISTICAL SYNTHESIS
The health maintenance rate was defined as the ratio of the number of preventive care actions to the number of eligible physician-patient encounters (opportunities). The number of eligible physician-patient encounters was used as the denominator, because the latest clinical recommendations indicate that virtually every visit regardless of reason is an opportunity to provide preventive care. Therefore, a missed opportunity is any physician-patient encounter that does not address preventive care. In addition, the higher the number of physician-patient encounters, the more opportunities for improvement exist. The clinical effect of prompts was estimated by the difference between the health maintenance rate in the intervention and control groups. The time during which health maintenance is observed is incorporated in the denominator. Point and interval (0.95) estimates of the prompting effect were calculated with models based on random-effects assumptions.10 These assumptions were raised by the diversity of clinical settings and groups of subjects analyzed. Homogeneity is not an assumption of the random-effects model used in this analysis.
To estimate the overall prompting effect, we used the (trimmed) modified DerSimonian-Laird estimator.11 The estimate was calculated by a weighted average of the individual rate differences. For each study, the weight was the reciprocal of the sum of an unbiased estimate of the within-study and the between-study variance so that studies having a large variance (and therefore less reliable) received a small weight (eg, in case of studies with few observations). In addition, we estimated the variance and the confidence interval (CI) of the estimated overall rate difference.
Furthermore, a random-effects regression model was applied to identify covariates with possible influence on prompting effect on health maintenance activities.12 Covariate analysis attempts to identify reasons for variation in the rates. Such factors are sources of heterogeneity in the efficacy of prompts among the studies. In this study, 9 variables were assumed to be associated with the effect of physician prompts based on preliminary literature review: trial quality, site, reimbursement method, characteristics, clinical specialty, size, delivery mode, targeted clinical action, and duration. Single-covariate and multiple-covariate regression models were used. The regression coefficients were estimated by the least-squares method. In addition, SEs and P values were calculated to determine the significance of the effects (covariates). The proportional reduction in variance among studies between conditional and unconditional models measured the degree to which a covariate accounted for the overall variance of the study.
The estimated reduction in deaths was calculated as the product of the rate difference, annual number of deaths, and mortality reduction rate for each of the targeted procedures. The rate difference was calculated by subtracting the health maintenance rate in the control group from the health maintenance rate in the intervention group. The annual number of deaths (55,300 from colon and rectum cancer,13 40,000 from influenza,14 46,000 from breast cancer,13 4800 from cervical cancer,13 40,000 from pneumococcal infections,15 and 12 from tetanus16) and the mortality reduction rates (33% for colon and rectum cancer,17 54% for influenza,14 30% for breast cancer,18 99% for cervical cancer,18 0% for pneumococcal infections,19 and 100% for tetanus20) for each targeted procedure were obtained from the literature. The mortality reduction of some of these activities may vary among the individuals in a population. Patient population factors, including baseline rates of preventive care, can be a large determinant of actual mortality gain.
Taking into account publication bias (that studies with negative results are less likely to be published), sensitivity analysis was performed to assess the potential influence of unidentified negative trials in overturning the results of the study. Calculated tolerance levels were compared with the corresponding threshold tolerance levels. Calculated tolerance is the number of additional but unpublished negative studies that could reverse the conclusions of this study. Calculated tolerance is defined with the formula 20k - n, where k is the number of positive studies in a category and n is the number of studies in a category.21 A study was positive if there was a significant difference for the intervention group compared with the control group at follow-up. Threshold tolerance is the number of unpublished negative studies that could reasonably exist. The Rosenthal formula of 5n + 10 has been widely applied to calculate threshold tolerance, where n is the number of studies in a category.21 When the calculated tolerance level exceeds the threshold tolerance, it is unlikely that unpublished negative studies can overturn the results.
RESULTS
Literature searches identified 101 pertinent clinical studies. During the filtering process, 68 studies were excluded for various reasons: same data as from another eligible study (3 studies), planned studies (2 studies), not a clinician reminder intervention but a patient reminder intervention (54 studies), other irrelevant reminder (7 studies), and no control group (2 studies). This left 33 eligible reports on randomized controlled clinical trials. No studies were excluded for only failing to test randomization by baseline comparison of intervention and control groups. All studies meeting the inclusion criteria and not qualifying for exclusion exceeded the quality scoring threshold. Using the replicable scoring technique, the average ± SD quality score was 69.7 ± 7.9. The characteristics of all eligible studies are listed in Table 1. All studies were parallel group trials, with the exception of the 2 studies that had a crossover design. In the pool of studies, 17 randomized patients directly, and 16 studies randomized through physicians. In health services research, patients are often randomly assigned to groups through their clinicians, because it is often the clinician who is directly targeted by the intervention.
|
|
|
|
Table 1. Characteristics of the 33 Studies*
|
|
|
Overall, data on 1547 clinicians and 54,693 patients were included in this meta-analysis. In the group of eligible studies, 3 trials were conducted in private offices recruited by the researchers, and 30 were conducted in university-affiliated clinics (19 studies) or public clinics (11 studies). Specialty of physicians included internal medicine (15 studies), family practice (general practice) (20 studies), and obstetrics (1 study). In all but one50 of the 33 studies, the patients were adults. The number of patients in the studies ranged from 57 patients33 to 7397 patients,44 with an average of 1657. The average ratio of patients to clinicians was 35.3.
INTERVENTION TECHNIQUES
The prompts were always delivered before a scheduled encounter. A few of the studies specified exactly when the prompt was delivered: once per year during the patient's month of birth,31 following randomization,45-47,53 1 month in advance of due date for targeted action,24 the night before the scheduled visit,52 the morning of the scheduled visit,44 before the visit at the reception desk,26 or during routine intake procedures.30
The prompts contained a variety of information for the clinicians. The most frequently included aspects indicated were the following: patient name,4, 52-53 patient diagnosis,4, 30, 43 notice of deviation from standard (overdue procedures or tests)22, 24, 31-32,34, 36-44,51-53, criteria for procedures or tests4, 24, 27-29,32, 34-35,44, 47, 52, recommendation for treatment or action,4, 30, 51 indication of when upcoming procedures or tests are due,22, 43, 48, 50 information on previous procedures or tests,24, 31, 44, 51, 53 and space for clinician to indicate that the procedure or test was offered or why it was not offered.24, 35, 44 Five of the studies provided generic prompts that were not patient specific.27-29,47, 53 The generic prompts were a checklist of guidelines applicable to any patient. Frequently, there was no room to record data of performed actions on the generic prompts. Some studies indicated that the prompts were repeated until the procedures or tests were completed.39-40
In some studies, the patients were reminded in addition to the clinicians. Several studies indicate that offering prompts to clinicians and patients amplifies the effect of the intervention.23, 31, 39-41,44, 48-50 Patients were reminded by telephone,23, 39-40,48 letter,33, 39-40,45-46,48 postcard,24, 50, 53 or a copy of their health maintenance report.31
One study, in addition to randomizing clinicians to groups who always received prompts and never received prompts, had a group of clinicians who sometimes received prompts.26 These clinicians were less likely than the always reminded clinicians and more likely than the never reminded clinicians to vaccinate their patients.
For most trials included in this study, the targeted actions were considered complete if the clinician made a note in the medical record. In addition to a medical record audit, one study also interviewed physicians following the encounters.40 Two studies indicated that a random selection of medical records was audited to obtain the rates of compliance with the recommended procedures.27, 41 In other studies, patients reported the completion of the targeted action through postencounter telephone interviews.4, 30
Only 7 of the 33 studies reviewed provided costs of the reminder intervention. None of the articles included the start-up costs of establishing the computerized reminder systems. Instead, the cost analyses focused on the operating expenses of the reminder systems. Studies showed that the computer-based reminder system cost $0.78 per patient per year to operate26; the cost to maintain the patient's medical record and produce reminders was estimated at $2 per patient visit38; the cost per cervical cancer screening gained was $11.7539; the cost per blood pressure screening gained was $1.7040; the computerized system cost $0.02 per record review42; with the use of the physician reminder system, improvement in preventive services was achieved for less than $5 per extra procedure completed48; and the cost per additional vaccination recorded was $0.43.49
IMPACT ON QUALITY SCORECARD PERFORMANCE
Prompts to clinicians resulted in a significant increase in the performance of all 16 preventive care procedures, including cancer screening (fecal occult blood, mammography, Papanicolaou smear), immunization (influenza vaccination, pneumococcal vaccination [Pneumovax], tetanus vaccination), diabetes management, hemoglobin management, blood pressure management and follow-up, cardiac care, cholesterol management, smoking cessation, glaucoma screening, alcohol abuse counseling, prenatal care, and tuberculosis testing. Most of the studies addressed the clinical areas of cancer screening and prevention (20 studies), immunization (14 studies), and diabetes management (4 studies). Eleven of the studies addressed 2 of these clinical areas, whereas 6 of the studies focused on other unlisted areas. The rate difference and calculated tolerance (ie, the number of additional but unpublished negative studies that could reverse the conclusions of this study) for selected cancer screening and immunization procedures are presented in Table 2. Only 6 of the clinical procedures are presented in Table 2, because the remaining clinical procedures were tested in fewer than 6 studies. All calculated tolerance levels exceeded the corresponding threshold tolerance, and it is therefore unlikely that unpublished negative studies would overturn these results. Physician prompting for the 6 procedures presented in Table 2 (fecal occult blood test, influenza vaccination, mammogram, Papanicolaou smear, pneumococcal vaccination, and tetanus vaccination) could expect to save 8333 lives annually.
|
|
|
|
Table 2. Effect of Prompting on Selected Procedures*
|
|
|
OTHER EFFECTS OF PROMPTING
The procedure-specific analysis of covariates indicated that the number of clinicians subjected to prompting, the ratio of residents among those receiving prompts, or the academic affiliation of the participating clinics did not make a difference in the effect of prompting. There are only 4 significant impacts based on a P<.05 level. With Bonferroni adjustment for multiple testing (P<.001), only 1 impact would be considered statistically significant, ie, larger patient-clinician ratio predisposes to larger improvement in tetanus vaccination as a result of prompting (0.0017 ± 0.0004, P <.001). A similar effect was also observed for pneumococcal vaccination (0.0073 ± 0.0025, P = .004). The results of the multiple covariate analysis indicate that prompting has significantly less effect on Papanicolaou smear rates in capitated outpatient care and when prompting is computer generated (-0.097 ± 0.043 [P = .02] and -0.078 ± 0.039 [P = .05], respectively). The few significant regression coefficients indicate the resistance of prompting to various covariates.
Presentation of the prompt appeared to have no significant effect on clinical responses (Table 3). Twenty-six studies using prompts attached to the front of the patient's medical record made a 14.0% (95% CI, 11.1%-16.9%) rate difference in the overall performance of prompts, whereas 7 studies using prompts placed elsewhere, such as tagged progress notes or computer monitor display, achieved a rate difference of 12.1% (95% CI, 5.4%-18.9%).
|
|
|
|
Table 3. Comparison of Various Prompting Techniques
|
|
|
Generation of prompts also did not show a significant difference in the clinical effect of prompting (Table 3). Generation of the prompting message through computerized records occurred in 25 studies. Although computerization appeared to adversely change the impact of prompting on Papanicolaou smear rates, this confounding effect could not be confirmed by the multivariate analysis (-0.091 ± 0.051, P = .07). Studies using computer-generated prompts achieved a rate difference of 13.6% (95% CI, 10.9%-16.3%) compared with 10.1% (95% CI, 1.3%-18.9%) for studies using non–computer-generated prompts. However, computerized record keeping and prompt generation was associated with larger studies. The average study size for the computerized studies was 2003 ± 2241 patients compared with noncomputerized studies, which averaged 577 ± 677 patients (P = .009).
Finally, the length of intervention period did not show a significant influence on the overall difference made by prompting. The studies ranged in duration from 5 to 161 weeks, with a total of 1596 study weeks. The average study length was 83 weeks. There was no association between the length of study and the health maintenance rate (R = -0.015, P = .47).
RATE DIFFERENCE
The rate differences and 95% CIs for the analyzed studies are presented in Figure 1. The resulting cumulative health maintenance rate difference was 13.1% (95% CI, 10.5%-15.6%). Only 7 of the studies, when analyzed individually, did not demonstrate significant improvement in the performance of preventive care items.29, 32-33,39, 43, 46-47
|
|
|
|
Figure 1. The rate differences, with 95% confidence intervals, for the analyzed studies.
|
|
|
CUMULATIVE RATE DIFFERENCE
The cumulative rate differences and 95% CIs for the analyzed studies are presented in Figure 2. The studies were cumulated chronologically based on publication year. Since additional studies were included in the health maintenance rate difference calculations, the CI narrowed. Since the publication of the first randomized prompting study by McDonald37 in 1976, cumulative effect has always indicated significant improvement in preventive care.
|
|
|
|
Figure 2. The cumulative rate differences, with 95% confidence intervals, for the analyzed studies.
|
|
|
COMMENT
The results of this study document that prompting physicians can lead to a significant improvement in health maintenance. The observed and much needed increase in performance of preventive care efforts could reap substantial reductions in total mortality. The many prompting tools offer a wide selection of options that are equally effective and easily applicable in most health care organizations (eg, checklists attached to the patient chart, tagged notes, computer-generated encounter forms, prompting stickers, patient-carried prompting cards). The fact that the beneficial effect of prompting does not seem to endure highlights the need for a complex intervention package to improve health maintenance activities that incorporates but is not limited to prompts. Larger incremental changes in clinical practices probably require combination with other interventions (eg, education, feedback, patient involvement).54
The objective of this article was to analyze the human response to a trigger on the part of clinicians to prompting messages. The measure of the clinical procedure is a quantification of this response. There is no practical link between the odds ratio and the rate of change perceived by clinicians and patients to prompting. By using the rate difference, improvement in HEDIS (The Health Plan Employer Data and Information Set) scorecard performance can be forecast. For example, a managed care organization that has an inferior 34% diabetic eye examination rate could become an above average performer by the application of prompting (national average, 38.4%).55
The rationale of prompting is to promote well-established and effective clinical interventions. For example, it has been documented that uncomfortable and inconvenient preventive care procedures are performed at a much lower rate.56 Yet, even Papanicolaou smear rates that are known to be more resistant to change can still be influenced. Prompts also have been used successfully to remind clinicians to discuss sigmoidoscopy with their patients. Rather than assume that sigmoidoscopy is too uncomfortable for all patients to tolerate, clinicians should inform the patient of the potential benefits and risks and enable the patient to make an informed decision.57 In one study using prompts, screening sigmoidoscopy was offered to 58% of patients.58 Significantly more patients in the intervention group completed the sigmoidoscopy (29% vs 2%) compared with the control group. The risks associated with the physician prompting intervention are low, whereas the potential benefits are significant in both statistical and practical terms. If a particular decision is supported by one prompt, then it is unlikely to be overwhelming even when several decisions are supported by several prompts. Local factors vary substantially among different health care settings and may determine the exact amount of improvement.
Physicians need reliable information about costs to implement reminder systems. Statements regarding cost without substantiating data are made habitually in reports of clinical trials.59 Most of the reported costs seem reasonable but still represent a significant operating expense for the practice. The cost of a system to support preventive care reminders could be higher for a practice without computers or a database.
The debate about priorities in cancer research highlights the need to compare the promises of prompting preventive care and the treatment of cancer. For example, a recent analysis estimated the number of patients who would have to be treated to save one life by mammography or by adding a combination product of cyclophosphamide, methotrexate, and fluorouracil to tamoxifen in the treatment of breast cancer (600 and 25, respectively).60 The annual number of mammographies is about 7.9 million and there are 24,000 newly detected cases of breast cancer in patients between the ages of 50 and 69 years.18 Therefore, an 11.5% improvement in the rate of breast cancer screening achieved through prompting could save 50% more lives than adding cyclophosphamide, methotrexate, and fluorouracil to tamoxifen. This calculation does not compare the human and financial costs of prevention and chemotherapy. Medical research has traditionally emphasized the discovery of new technologies, tests, devices, and medications while many scientific advances have not achieved their potential for improving patient care. Our study indicates that the benefits of quality improvement and better implementation may be comparable to the benefits of some technological advances.
Meta-analysis is informative when the same intervention has been tested repeatedly, and therefore the effect can be estimated quantitatively. To commit resources to a quality improvement intervention, clinicians and provider organizations need to know the magnitude of the expected benefits. In our study, quantitative analysis was necessary because of the apparent lack of awareness of the benefits of reminding, numerous repeated tests of the same intervention, and the ambiguous recommendations coming from nonquantitative reviews. Based on the significant effect of prompting, future experimentation and placement of randomly selected patients to control groups should raise ethical questions.
One potential problem with defining the health maintenance rate as the ratio of the number of preventive care actions to the number of visits is that it can be changed simply by altering the denominator. Thus, in situations in which the patient is seen often, the apparent health maintenance rate would go down, yet it would not necessarily be appropriate to engage in preventive actions at all visits.
Numerous factors have been identified as obstacles to preventive care, but the interest of patients lies in those that can be changed to improve quality. Lurie et al61 found that patients who had free care received more preventive services than patients who shared the cost. Fragmentation of care, patients seeing several physicians of different specialties or at different locations, also presents a barrier to preventive care.62 Among patients surveyed by Bindman et al,63 the most important factor associated with receipt of preventive care was having a regular source of care. Often patients and physicians have an expectation that the agenda will be limited to the immediate concern, with preventive care reserved for the checkup, or periodically scheduled physical examination, which may not take place in a timely fashion, if at all.62
The American College of Physicians recommends that "the external oversight of care be restructured and that the primary locus of quality assurance be returned to the medical profession and to health institutions."64 Timely possession of pertinent information could empower physicians to direct medical resources and improve the quality of patient care. Such reallocation of information is more likely to lead to much needed changes that can actually prevent disease and benefit patients.
AUTHOR INFORMATION
Accepted for publication October 14, 1999.
This research was supported by a grant from the Center for Health Management Research of the National Science Foundation and University of Washington, Seattle, and by grant LM05545 from the National Library of Medicine, Bethesda, Md.
We are indebted to the following colleagues for statistical advice and constructive assistance: Jane C. Johnson, MA, John E. Hewett, PhD, Kui-Chun Su, MLIS, and Isabelle Grimmenstein.
Reprints: E. Andrew Balas, MD, PhD, School of Medicine, University of Missouri, 313 Clark Hall, Columbia, MO 65211 (e-mail: ABalas{at}health.missouri.edu).
From the School of Medicine, University of Missouri, Columbia (Drs Balas and Brown and Mss Garb and Boren); Health Services Research, Cedars-Sinai Medical Center, Los Angeles, Calif (Dr Weingarten); and Health Policy Research and Development Unit, Massachusetts General Hospital, Boston (Dr Blumenthal).
REFERENCES
| |
1. Lurie N, Slater J, McGovern P, Ekstrum J, Quam L, Margolis K. Preventive care for women: does the sex of the physician matter? N Engl J Med. 1993;329:478-482.
FREE FULL TEXT
2. Iglehart JK. The National Committee for Quality Assurance. N Engl J Med. 1996;335:995-999.
FREE FULL TEXT
3. American Academy of Pediatrics, Committee on Practice and Ambulatory Medicine. Implementation of the immunization policy (S94-26). Pediatrics. 1995;96:360-361.
FREE FULL TEXT
4. Buchsbaum DG, Buchanan RG, Lawton MJ, Elswick RK, Schnoll SH. A program of screening and prompting improves short-term physician counseling of dependent and nondependent harmful drinkers. Arch Intern Med. 1993;153:1573-1577.
FREE FULL TEXT
5. Austin SM, Balas EA, Mitchell JA, Ewigman BG. Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials. J Am Med Inform Assoc. 1994;1(suppl):121-124.
6. Lipton HL, Bero LA, Bird JA, McPhee SJ. The impact of clinical pharmacists' consultations on physicians' geriatric drug prescribing: a randomized controlled trial. Med Care. 1992;30:646-658.
FULL TEXT
|
ISI
| PUBMED
7. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423-1431.
ABSTRACT
8. Balas EA, Austin SM, Ewigman BG, Brown GD, Mitchell JA. Methods of randomized controlled clinical trials in health services research. Med Care. 1995;33:687-699.
ISI
| PUBMED
9. Balas EA, Stockham MG, Mitchell JA, Austin SM, West DA, Ewigman BG. The Columbia Registry of Information and Utilization Management Trials. J Am Med Inform Assoc. 1995;2:307-315.
FREE FULL TEXT
10. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177-188.
FULL TEXT
|
ISI
| PUBMED
11. Emerson JD. Simple robust procedures for combining risk differences in sets of 2 x 2 tables. Stat Med. 1996;15:1465-1488.
ISI
| PUBMED
12. Stram DO. Meta-analysis of published data using a linear mixed-effects model. Biometrics. 1996;52:536-544.
FULL TEXT
|
ISI
| PUBMED
13. Wingo PA, Tong T, Bolden S. Cancer statistics, 1995. CA Cancer J Clin. 1995;45:8-30.
FREE FULL TEXT
14. Nichol KL, Margolis KL, Wuorenma J, Von Sternberg T. The efficacy and cost effectiveness of vaccination against influenza among elderly persons living in the community. N Engl J Med. 1994;331:778-784.
FREE FULL TEXT
15. Increasing pneumococcal vaccination rates among patients of a national health-care alliance—United States, 1993. MMWR Morb Mortal Wkly Rep. 1995;44:741-745.
PUBMED
16. Prevots R, Sutter RW, Strebel PM, Cochi SL, Hadler S. Tetanus surveillance—United States, 1989-1990. Morb Mortal Wkly Rep CDC Surveill Summ. 1992;41:1-9.
17. Mandel JS, Bond JH, Church TR, et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. N Engl J Med. 1993;328:1365-1371.
FREE FULL TEXT
18. Update: national breast and cervical cancer early detection program—July 1991-September 1995. MMWR Morb Mortal Wkly Rep. 1996;45:484-487.
PUBMED
19. Fine MJ, Smith MA, Carson CA, et al. Efficacy of pneumococcal vaccination in adults. Arch Intern Med. 1994;154:2666-2677.
20. Izurieta HS, Sutter RW, Strebel PM, et al. Tetanus surveillance—United States, 1991-1994. Morb Mortal Wkly Rep CDC Surveill Summ. 1997;46:15-25.
21. Rosenthal R. The "File drawer problem" and tolerance for null results. Psychol Bull. 1979;86:638-641.
FULL TEXT
|
ISI
22. Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983;21:400-409.
FULL TEXT
|
ISI
| PUBMED
23. Becker DM, Gomez EB, Kaiser DL, Yoshihasi A, Hodge RH. Improving preventive care at a medical clinic: how can the patient help? Am J Prev Med. 1989;5:353-358.
ISI
| PUBMED
24. Burack RC, Gimotty PA, George J, Stengle W, Warbasse L, Moncrease A. Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994;32:609-624.
FULL TEXT
|
ISI
| PUBMED
25. Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM. Microcomputer-generated reminders: improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989;29:273-280.
ISI
| PUBMED
26. Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract. 1991;4:19-26.
27. Cheney C, Ramsdell JW. Effect of medical records' checklists on implementation of periodic health measures. Am J Med. 1987;83:129-136.
FULL TEXT
|
ISI
| PUBMED
28. Cohen DI, Littenberg B, Wetzel C, Neuhauser DVB. Improving physician compliance with preventive medicine guidelines. Med Care. 1982;20:1040-1045.
FULL TEXT
|
ISI
| PUBMED
29. Cowan JA, Heckerling PS, Parker JB. Effect of a fact sheet reminder on performance of the periodic health examination: a randomized controlled trial. Am J Prev Med. 1992;8:104-109.
ISI
| PUBMED
30. Cummings SR, Richard RJ, Duncan CL, et al. Training physicians about smoking cessation: a controlled trial in private practice. J Gen Intern Med. 1989;4:482-489.
ISI
| PUBMED
31. Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking. Arch Fam Med. 1994;3:581-588.
FREE FULL TEXT
32. Headrick LA, Speroff T, Pelecanos HI, Cebul RD. Efforts to improve compliance with the National Cholesterol Education Program guidelines: results of a randomized controlled trial. Arch Intern Med. 1992;152:2490-2496.
FREE FULL TEXT
33. Landis SE, Hulkower SD, Pierson S. Enhancing adherence with mammography through patient letters and physician prompts. N C Med J. 1992;53:575-578.
PUBMED
34. Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993;8:311-317.
ISI
| PUBMED
35. Lobach DF, Hammond WE. Development and evaluation of a computer-assisted management protocol (CAMP): improved compliance with care guidelines for diabetes mellitus. J Am Med Inform Assoc. 1994;1(suppl):787-791.
36. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. Ann Intern Med. 1984;100:130-138.
37. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectibility of man. N Engl J Med. 1976;295:1351-1355.
ABSTRACT
38. McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976;84:162-167.
39. McDowell I, Newell C, Rosser W. Computerized reminders to encourage cervical screening in family practice. J Fam Pract. 1989;28:420-424.
ISI
| PUBMED
40. McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989;27:297-305.
FULL TEXT
|
ISI
| PUBMED
41. McPhee SJ, Bird JA, Jenkins CNH, Fordham D. Promoting cancer screening. Arch Intern Med. 1989;149:1866-1872.
FREE FULL TEXT
42. Morgan M, Studney DR, Barnett GO, Winickoff RN. Computerized concurrent review of prenatal care. QRB Qual Rev Bull. 1978;4:33-36.
43. Nilasena DS, Lincoln MJ. A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines. J Am Med Inform Assoc. 1995;2(suppl):640-645.
44. Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services. J Fam Pract. 1991;32:82-90.
ISI
| PUBMED
45. Pierce M, Lundy S, Palanisamy A, Winning S, King J. Prospective randomised controlled trial of methods of call and recall for cervical cytology screening. BMJ. 1989;299:160-162.
46. Pritchard DA, Stratoy JAY, Hyndman J. Cervical screening in general practice. Aust J Public Health. 1995;19:167-172.
ISI
| PUBMED
47. Robie PW. Improving and sustaining outpatient cancer screening by medicine residents. South Med J. 1988;81:902-905.
FULL TEXT
|
ISI
| PUBMED
48. Rosser WW, McDowell I, Newell C. Use of reminders for preventive procedures in family medicine. CMAJ. 1991;145:807-814.
ABSTRACT
49. Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination. CMAJ. 1992;146:911-917.
ABSTRACT
50. Soljak MA, Handford S. Early results from the Northland immunisation register. N Z Med J. 1987;100:244-246.
ISI
| PUBMED
51. Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993;94:619-625.
FULL TEXT
|
ISI
| PUBMED
52. Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24:659-666.
FULL TEXT
|
ISI
| PUBMED
53. Turner RC, Waivers LE, O'Brien K. The effect of patient-carried reminder cards on the performance of health maintenance measures. Arch Intern Med. 1990;150:645-647.
FREE FULL TEXT
54. Greco PJ, Eisenberg JM. Changing physicians' practices. N Engl J Med. 1993;329:1271-1273.
FREE FULL TEXT
55. National Committee for Quality Assurance. Quality compass: diabetic retinal exam rate. Available at: http://www.ncqa.org/qc/diabetic.htm. Accessed July 7, 1998.
56. Pommernke FA, Weed DL. Physician compliance: improving skills in preventive medicine practices. Am Fam Physician. 1991;43:560-568.
ISI
| PUBMED
57. McCarthy BD, Moskowitz MA. Screening flexible sigmoidoscopy: patient attitudes and compliance. J Gen Intern Med. 1993;8:120-125.
ISI
| PUBMED
58. Bejes C, Marvel MK. Attempting the improbable: offering colorectal cancer screening to all appropriate patients. Fam Pract Res J. 1992;12:83-90.
PUBMED
59. Balas EA, Kretschmer RAC, Gnann W, et al. Interpreting cost analyses of clinical interventions. JAMA. 1998;279:54-57.
FREE FULL TEXT
60. Kattlove H, Liberati A, Keeler E, Brook RH. Benefits and costs of screening and treatment for early breast cancer: development of a basic benefit package. JAMA. 1995;273:142-148.
FREE FULL TEXT
61. Lurie N, Manning WG, Peterson C, Goldberg GA, Phelps CA, Lillard L. Preventive care: do we practice what we preach? Am J Public Health. 1987;77:801-804.
FREE FULL TEXT
62. Frame PS. Health maintenance in clinical practice: strategies and barriers. Am Fam Physician. 1992;45:1192-1200.
ISI
| PUBMED
63. Bindman AB, Grumbach K, Osmond D, Vranizan K, Stewart AL. Primary care and receipt of preventive services. J Gen Intern Med. 1996;11:269-276.
ISI
| PUBMED
64. American College of Physicians. The oversight of medical care: a proposal for reform. Ann Intern Med. 1994;120:423-431.
FREE FULL TEXT
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Performance in Practice: Clinical Tools to Improve the Care of Patients with Posttraumatic Stress Disorder
Duffy et al.
Focus 2009;7:186-203.
ABSTRACT
| FULL TEXT
Patient and Physician Reminders to Promote Colorectal Cancer Screening: A Randomized Controlled Trial
Sequist et al.
Arch Intern Med 2009;169:364-371.
ABSTRACT
| FULL TEXT
Making research relevant: if it is an evidence-based practice, where's the practice-based evidence?
Green
Fam Pract 2008;25:i20-i24.
ABSTRACT
| FULL TEXT
Self Determination Theory and Preventive Care Delivery: A Research Involving Outpatient Settings Network (RIOS Net) Study
Sussman et al.
J Am Board Fam Med 2008;21:282-292.
ABSTRACT
| FULL TEXT
Next Generation of Health Information Tools: Where Do We Go From Here?
Grant
Mayo Clin Proc. 2008;83:745-746.
FULL TEXT
Prompting Clinicians about Preventive Care Measures: A Systematic Review of Randomized Controlled Trials
Dexheimer et al.
J. Am. Med. Inform. Assoc. 2008;15:311-320.
ABSTRACT
| FULL TEXT
Performance in Practice: Sample Tools for the Care of Patients with Major Depressive Disorder
Fochtmann et al.
Focus 2008;6:22-35.
ABSTRACT
| FULL TEXT
How to Increase Colorectal Cancer Screening Rates in Practice
Sarfaty and Wender
CA Cancer J Clin 2007;57:354-366.
ABSTRACT
| FULL TEXT
Adoption of Order Entry with Decision Support for Chronic Care by Physician Organizations
Simon et al.
J. Am. Med. Inform. Assoc. 2007;14:432-439.
ABSTRACT
| FULL TEXT
A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network
Wright et al.
J. Am. Med. Inform. Assoc. 2007;14:489-496.
ABSTRACT
| FULL TEXT
Electronic Health Records in Four Community Physician Practices: Impact on Quality and Cost of Care
Welch et al.
J. Am. Med. Inform. Assoc. 2007;14:320-328.
ABSTRACT
| FULL TEXT
Failure to Adopt Beneficial Therapies Caused by Bias in Medical Evidence Evaluation
Aberegg et al.
Med Decis Making 2006;26:575-582.
ABSTRACT
Improved therapeutic monitoring with several interventions: a randomized trial.
Feldstein et al.
Arch Intern Med 2006;166:1848-1854.
ABSTRACT
| FULL TEXT
What Interventions Should Pharmacists Employ to Impact Health Practitioners' Prescribing Practices?
Grindrod et al.
The Annals of Pharmacotherapy 2006;40:1546-1557.
ABSTRACT
| FULL TEXT
The Art and Complexity of Primary Care Clinicians' Preventive Counseling Decisions: Obesity as a Case Study
Sussman et al.
Ann Fam Med 2006;4:327-333.
ABSTRACT
| FULL TEXT
ASHP Therapeutic Position Statement on the Treatment of Hypertension.
Zillich and Haines
Am J Health Syst Pharm 2006;63:1074-1080.
FULL TEXT
The impact of prescribing safety alerts for elderly persons in an electronic medical record: an interrupted time series evaluation.
Smith et al.
Arch Intern Med 2006;166:1098-1104.
ABSTRACT
| FULL TEXT
A Practice-Sponsored Web Site to Help Patients Pursue Healthy Behaviors: An ACORN Study
Woolf et al.
Ann Fam Med 2006;4:148-152.
ABSTRACT
| FULL TEXT
Translation of Research Into Practice: Why We Can't "Just Do It"
Green and Seifert
J Am Board Fam Med 2005;18:541-545.
ABSTRACT
| FULL TEXT
Breast and Cervical Cancer Screening: Clinicians' Views on Health Plan Guidelines and Implementation Efforts
Zapka et al.
J Natl Cancer Inst Monogr 2005;2005:46-54.
ABSTRACT
| FULL TEXT
Epidemiology of Uncontrolled Hypertension in the United States
Wang and Vasan
Circulation 2005;112:1651-1662.
FULL TEXT
Interventions to Regulate Ordering of Serum Magnesium Levels: Report of an Unintended Consequence of Decision Support
Rosenbloom et al.
J. Am. Med. Inform. Assoc. 2005;12:546-553.
ABSTRACT
| FULL TEXT
A trial of education, prompts, and opinion leaders to improve prescription of lipid modifying therapy by primary care physicians for patients with ischemic heart disease
Bloomfield et al.
Qual Saf Health Care 2005;14:258-263.
ABSTRACT
| FULL TEXT
Effect of CPOE User Interface Design on User-Initiated Access to Educational and Patient Information during Clinical Care
Rosenbloom et al.
J. Am. Med. Inform. Assoc. 2005;12:458-473.
ABSTRACT
| FULL TEXT
Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success
Kawamoto et al.
BMJ 2005;330:765.
ABSTRACT
| FULL TEXT
Health Care Provider-Directed Intervention to Increase Colorectal Cancer Screening Among Veterans: Results of a Randomized Controlled Trial
Ferreira et al.
JCO 2005;23:1548-1554.
ABSTRACT
| FULL TEXT
Opening the Black Box: Cognitive Strategies in Family Practice
Christensen et al.
Ann Fam Med 2005;3:144-150.
ABSTRACT
| FULL TEXT
Should dentistry be part of the National Health Information Infrastructure?
SCHLEYER
Journal of the American Dental Association 2004;135:1687-1695.
ABSTRACT
| FULL TEXT
Patient Education for Colon Cancer Screening: A Randomized Trial of a Video Mailed before a Physical Examination
Zapka et al.
ANN INTERN MED 2004;141:683-692.
ABSTRACT
| FULL TEXT
Clinical Informatics in Critical Care
Martich et al.
J Intensive Care Med 2004;19:154-163.
ABSTRACT
Variations in Diabetes Care and the Influence of Office Systems
Ellerbeck et al.
American Journal of Medical Quality 2004;19:12-18.
ABSTRACT
Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure
Chobanian et al.
Hypertension 2003;42:1206-1252.
ABSTRACT
| FULL TEXT
Impact of Population Management With Direct Physician Feedback on Care of Patients With Type 2 Diabetes
Grant et al.
Diabetes Care 2003;26:2275-2280.
ABSTRACT
| FULL TEXT
Providers' Reactions to an Automated Health Maintenance Reminder System Incorporated into the Patient's Electronic Medical Record
Schellhase et al.
J Am Board Fam Med 2003;16:312-317.
ABSTRACT
| FULL TEXT
Improving Guideline Adherence: A Randomized Trial Evaluating Strategies to Increase {beta}-Blocker Use in Heart Failure
Ansari et al.
Circulation 2003;107:2799-2804.
ABSTRACT
| FULL TEXT
The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: The JNC 7 Report
Chobanian et al.
JAMA 2003;289:2560-2571.
ABSTRACT
| FULL TEXT
Addressing Alcohol Problems in Primary Care: A Cluster Randomized, Controlled Trial of a Systems Intervention: The Screening and Intervention in Primary Care (SIP) Study
Saitz et al.
ANN INTERN MED 2003;138:372-382.
ABSTRACT
| FULL TEXT
Automating Complex Guidelines for Chronic Disease: Lessons Learned
Maviglia et al.
J. Am. Med. Inform. Assoc. 2003;10:154-165.
ABSTRACT
| FULL TEXT
A Research Agenda For Bridging The 'Quality Chasm'
Fernandopulle et al.
Health Aff (Millwood) 2003;22:178-190.
ABSTRACT
| FULL TEXT
External Incentives, Information Technology, and Organized Processes to Improve Health Care Quality for Patients With Chronic Diseases
Casalino et al.
JAMA 2003;289:434-441.
ABSTRACT
| FULL TEXT
A Proposal for Electronic Medical Records in U.S. Primary Care
Bates et al.
J. Am. Med. Inform. Assoc. 2003;10:1-10.
ABSTRACT
| FULL TEXT
From Physician to Consumer: The Effectiveness of Strategies to Manage Health Care Utilization
Flynn et al.
Med Care Res Rev 2002;59:455-481.
ABSTRACT
Task Force #5--the role of cardiovascular specialists as leaders in prevention: from training to champion
Merz et al.
J Am Coll Cardiol 2002;40:641-649.
FULL TEXT
Screening for Colorectal Cancer in Adults at Average Risk: A Summary of the Evidence for the U.S. Preventive Services Task Force
Pignone et al.
ANN INTERN MED 2002;137:132-141.
ABSTRACT
| FULL TEXT
{beta}-Blockers in Heart Failure: Clinical Applications
Farrell et al.
JAMA 2002;287:890-897.
ABSTRACT
| FULL TEXT
Videotape-Based Decision Aid for Colon Cancer Screening
Michael et al.
ANN INTERN MED 2001;135:634-635.
FULL TEXT
Prevention of Glucocorticoid-Induced Osteoporosis: Experience in a Managed Care Setting
Yood et al.
Arch Intern Med 2001;161:1322-1327.
ABSTRACT
| FULL TEXT
Prevention of Venous Thromboembolism
Geerts et al.
Chest 2001;119:132S-175S.
FULL TEXT
Videotape-Based Decision Aid for Colon Cancer Screening: A Randomized, Controlled Trial
Pignone et al.
ANN INTERN MED 2000;133:761-769.
ABSTRACT
| FULL TEXT
|
