This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in this journal  Social Bookmarking   What's this?

Is It Magic or Methods?

Mary E. Charlson, MD; Kathy L. Ales, MD; Richard Simon, PhD; C. Ronald MacKenzie, MD

Arch Intern Med. 1987;147(12):2155-2161.

Abstract
• When prognostic indexes have been tested in a second population, they have often performed less well. Since this is believed to be inevitable, methodologic differences that may explain the discrepancies have been overlooked. Data from a prospective study of 232 patients undergoing noncardiac surgery were used to examine the effect of methodologic differences in assembly of population, postoperative surveillance, and the criteria for cardiac complications on the performance of Goldman's cardiac risk index. Our prospective population was used to simulate the methods used in Goldman's study and in three other studies using the risk index to demonstrate the potential impact of differences in population, surveillance, and outcome criteria for cardiac complications. If Goldman's detection and outcome criteria were employed and only the eligibility criteria used for assembly of the populations differed, the overall complication rates would be between 5.2% and 6.9%; and the complication rates for the different Goldman classes were similar. When both different detection strategies and different outcome criteria were used, however, important discrepancies in cardiac complication rates emerged. For example, complication rates in class 2 varied from 2% to 23%. In conclusion, important discrepancies in performance of prognostic indexes may arise from differences in surveillance strategies and definitions of outcome. With sufficient attention to methodologic consistency, the performance of predictive indexes may not inevitably deteriorate in subsequent studies.

(Arch Intern Med 1987;147:2155-2161)

Author Affiliations
From the Clinical Epidemiology Unit, Department of Medicine, Cornell University Medical College, New York (Drs Charlson, Ales, and Mac-Kenzie); and the Biometric Research Branch, National Cancer Institute, Bethesda, Md (Dr Simon).

Footnotes
Accepted for publication Aug 12, 1987.

Reprint requests to 515 E 71st St, New York, NY 10021 (Dr Charlson).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

The San Francisco Syncope Rule did not accurately predict serious short-term outcome in patients with syncope
Carpenter
Evid. Based Med. 2009;14:25-25.
FULL TEXT  

Benefits and pitfalls of pooling datasets from comparable observational studies: combining US and Dutch nursing home studies
van der Steen et al.
Palliat Med 2008;22:750-759.
ABSTRACT  

When to Use Computed Tomography in Minor Pediatric Head Trauma
Bratton and Paul
AAP Grand Rounds 2008;20:14-15.
FULL TEXT  

Performance Characteristics of Depression Screening Instruments in Survivors of Acute Myocardial Infarction: Review of the Evidence
Thombs et al.
Psychosomatics 2007;48:185-194.
ABSTRACT | FULL TEXT  

Quantifying Risk and Benchmarking Performance in the Adult Intensive Care Unit
Higgins
J Intensive Care Med 2007;22:141-156.
ABSTRACT  

Linking Surveillance to Action: Incorporation of Real-time Regional Data into a Medical Decision Rule
Fine et al.
J. Am. Med. Inform. Assoc. 2007;14:206-211.
ABSTRACT | FULL TEXT  

Clinical Prediction Rule for Identifying Children With Cerebrospinal Fluid Pleocytosis at Very Low Risk of Bacterial Meningitis
Nigrovic et al.
JAMA 2007;297:52-60.
ABSTRACT | FULL TEXT  

Comparison of the Canadian CT Head Rule and the New Orleans Criteria in Patients With Minor Head Injury
Stiell et al.
JAMA 2005;294:1511-1518.
ABSTRACT | FULL TEXT  

Utility of the Chronic Disease Score and Charlson Comorbidity Index as Comorbidity Measures for Use in Epidemiologic Studies of Antibiotic-resistant Organisms
McGregor et al.
Am J Epidemiol 2005;161:483-493.
ABSTRACT | FULL TEXT  

Prospective Validation of a Prediction Model for Isolating Inpatients With Suspected Pulmonary Tuberculosis
Wisnivesky et al.
Arch Intern Med 2005;165:453-457.
ABSTRACT | FULL TEXT  

CUGE: A SCREENING INSTRUMENT FOR ALCOHOL ABUSE AND DEPENDENCE IN STUDENTS
Van Den Bruel et al.
Alcohol Alcohol 2004;39:439-444.
ABSTRACT | FULL TEXT  

Perioperative myocardial injury: individual and population implications
Howell and Sear
Br J Anaesth 2004;93:3-8.
ABSTRACT | FULL TEXT  

Chronic Kidney Disease, Mortality, and Treatment Strategies among Patients with Clinically Significant Coronary Artery Disease
Reddan et al.
J. Am. Soc. Nephrol. 2003;14:2373-2380.
ABSTRACT | FULL TEXT  

Adjustment for Comorbidity in Studies on Health Status in ESRD Patients: Which Comorbidity Index to Use?
van Manen et al.
J. Am. Soc. Nephrol. 2003;14:478-485.
ABSTRACT | FULL TEXT  

Identification of poor prognostic features among patients requiring mechanical ventilation after hematopoietic stem cell transplantation
Bach et al.
Blood 2001;98:3234-3240.
ABSTRACT | FULL TEXT  

Steady-State End-Tidal Alveolar Dead Space Fraction and D-Dimer : Bedside Tests To Exclude Pulmonary Embolism
Rodger et al.
Chest 2001;120:115-119.
ABSTRACT | FULL TEXT  

Risk stratification in unstable angina: the role of clinical prediction models
Katz
J Am Coll Cardiol 2000;36:1809-1811.
FULL TEXT  

Predictive accuracy study: comparing a statistical model to clinicians' estimates of outcomes after coronary bypass surgery
Ivanov et al.
Ann. Thorac. Surg. 2000;70:162-168.
ABSTRACT | FULL TEXT  

The Stroke Prognosis Instrument II (SPI-II) : A Clinical Prediction Instrument for Patients With Transient Ischemia and Nondisabling Ischemic Stroke
Kernan et al.
Stroke 2000;31:456-462.
ABSTRACT | FULL TEXT  

The Validity of Medical History, Classic Symptoms, and Chest Radiographs in Predicting Pulmonary Tuberculosis: Derivation of a Pulmonary Tuberculosis Prediction Model
Tattevin et al.
Chest 1999;115:1248-1253.
ABSTRACT | FULL TEXT  

Ready-Made, Recalibrated, or Remodeled? : Issues in the Use of Risk Indexes for Assessing Mortality After Coronary Artery Bypass Graft Surgery
Ivanov et al.
Circulation 1999;99:2098-2104.
ABSTRACT | FULL TEXT  

Assessing the Generalizability of Prognostic Information
Justice et al.
ANN INTERN MED 1999;130:515-524.
ABSTRACT | FULL TEXT  

Cardiac Surgical Mortality: Comparison Among Different Additive Risk-Scoring Models in a Multicenter Sample
Pons et al.
Arch Surg 1998;133:1053-1057.
ABSTRACT | FULL TEXT  

Development and Prospective Validation of a Clinical Index to Predict Survival in Ambulatory Patients Referred for Cardiac Transplant Evaluation
Aaronson et al.
Circulation 1997;95:2660-2667.
ABSTRACT | FULL TEXT  

Use of a Probabilistic Neural Network to Estimate the Risk of Mortality after Cardiac Surgery
Orr
Med Decis Making 1997;17:178-185.
ABSTRACT  

Prospective Validation of a Decision Rule for the Use of Radiography in Acute Knee Injuries
Stiell et al.
JAMA 1996;275:611-615.
ABSTRACT  

A Comparison of Four Severity-Adjusted Models to Predict Mortality After Coronary Artery Bypass Graft Surgery
Orr et al.
Arch Surg 1995;130:301-306.
ABSTRACT  

Decision Rules for the Use of Radiography in Acute Ankle Injuries: Refinement and Prospective Validation
Shell et al.
JAMA 1993;269:1127-1132.
ABSTRACT  

Validation of Probabilistic Predictions
Miller et al.
Med Decis Making 1993;13:49-57.
ABSTRACT  

Predicting Postoperative Pulmonary Complications: Is It a Real Problem?
Williams-Russo et al.
Arch Intern Med 1992;152:1209-1213.
ABSTRACT