 |
|
Evaluation of the Framingham Risk Score in the European Prospective Investigation of Cancer–Norfolk CohortDoes Adding Glycated Hemoglobin Improve the Prediction of Coronary Heart Disease Events?
Rebecca K. Simmons, PhD;
Stephen Sharp, MSc;
S. Matthijs Boekholdt, MD, PhD;
Lincoln A. Sargeant, PhD;
Kay-Tee Khaw, PhD;
Nicholas J. Wareham, PhD;
Simon J. Griffin, DM
Arch Intern Med. 2008;168(11):1209-1216.
ABSTRACT
| |
Background There is a continuous relationship between glycated hemoglobin (HbA1c) and coronary heart disease (CHD) risk, even below diagnostic thresholds for diabetes mellitus.
Methods To evaluate the Framingham risk score in a UK population-based prospective cohort (European Prospective Investigation of Cancer [EPIC]–Norfolk) and to assess whether adding HbA1c improves the prediction of CHD. Participants aged 40 to 79 years were recruited from UK general practices, attended a health check, and were followed up for CHD events and death. The Framingham risk score was computed for 10 295 individuals with data on age, total cholesterol, high-density lipoprotein cholesterol, systolic blood pressure, diabetes mellitus, and smoking status. We developed a Cox proportional hazards regression model with the original Framingham covariates and then added HbA1c to determine whether this improved the prediction of CHD. Model discrimination was compared by using area under the receiver operating characteristic curves (AUROCs), and the correctness of reclassification was determined by calculating the net reclassification improvement and the integrated discrimination improvement. The main outcome measures were CHD-related hospital admission and death.
Results A total of 430 men and 250 women developed CHD during 8.5 years of follow-up. The AUROC for the original Framingham risk score was 0.71. Using the Framingham variables with coefficients fitted from the EPIC-Norfolk data, the AUROC was 0.72 for men and 0.80 for women, compared with 0.73 and 0.80, respectively, in a score including HbA1c. This difference was significant for men only (P = .005). The net reclassification improvement was 3.4% (P = .06) in men and –2.2% (P = .27) in women.
Conclusions The Framingham risk score predicts CHD in this cohort. The addition of HbA1c made a small but statistically significant improvement to discrimination in men but not in women, without significant improvement in reclassification of risk category.
INTRODUCTION
The Framingham risk score is a prediction algorithm used by physicians to estimate coronary heart disease (CHD) risk in middle-aged people.1 Information on age, sex, diabetes mellitus, smoking, blood pressure, total or low-density lipoprotein cholesterol, and high-density lipoprotein (HDL) cholesterol is used to calculate the absolute risk of a CHD event during the next 10 years. The score was originally developed in a small population-based cohort in Framingham, Massachusetts. It has good predictive value in other US populations,2 but its performance varies considerably between different countries and populations.3-5 Consequently, there have been calls to refine the score.6 Identifying individuals at high risk can target evidence-based treatment, which has been shown in clinical trials to significantly reduce the risk of CHD events. Improved CHD risk prediction can also inform cost-effective prescribing policy.
One of the variables used to calculate the Framingham risk score is a dichotomous variable for the presence or absence of diabetes mellitus. This approach does not account for the increased risk associated with elevated levels of glucose below the diagnostic threshold for diabetes mellitus,7 which have been demonstrated to have a continuous relationship with cardiovascular disease risk,8-10 or for the increased risk associated with high blood glucose levels in diabetic patients. Glycated hemoglobin (HbA1c) has been shown to predict cardiovascular disease events and all-cause mortality independently of other cardiovascular risk factors11 in people with12 and without13-14 diabetes mellitus. Although not approved for the diagnosis of diabetes mellitus, HbA1c is an easily measured biochemical risk factor that correlates well with ambient glycemia during a 2- to 3-month period.15 It provides a practical means for assessing hyperglycemia and can be measured using a relatively inexpensive capillary finger prick test at any time of the day under fasting or nonfasting conditions. We hypothesized that the continuous risk factor HbA1c would improve risk prediction using the Framingham risk algorithms, which include diabetes mellitus as a dichotomous variable.
We tested this hypothesis using data from the European Prospective Investigation of Cancer–Norfolk (EPIC-Norfolk), an ongoing population-based prospective cohort study. We aimed to explore (1) whether the Framingham risk score predicts CHD events and CHD death and (2) whether adding HbA1c to the Framingham risk score improves the prediction of CHD.
METHODS
EPIC-Norfolk is a prospective cohort study in which men and women aged 40 to 79 years were recruited from general practices in the Norfolk region, England. Full details of the population are reported elsewhere.16 In brief, between March 1993 and February 1998, 25 639 individuals underwent a baseline health examination that included anthropometric and blood pressure measurements and completion of a general health questionnaire, with questions on personal and family history of disease, medications used, and lifestyle factors, including smoking habits. Participants were asked to confirm whether they were a current, former, or never smoker. They were also asked whether a physician had ever told them that they had any of the conditions contained in a list that included diabetes mellitus, heart attack, and stroke. In addition, baseline diabetes mellitus status was also ascertained by means of (1) self-report of diabetes medication use, (2) diabetes medication brought to the baseline health check, (3) the participant indicating modification of the diet in the past year because of diabetes mellitus, or (4) the participant indicating adherence to a diabetic diet. Nonfasting blood samples were collected, and starting in 1995, when funding became available, HbA1c levels were measured on fresh EDTA blood samples using high-performance liquid chromatography (Diamat Automated Glycated Hemoglobin Analyzer; Bio-Rad Laboratories Ltd, Hemel Hempstead, England).
The Norfolk area is slightly healthier than the general UK population, with a standardized mortality ratio of 93 (source: Office for National Statistics registration data, 2006). However, EPIC-Norfolk is similar to a nationally representative sample regarding anthropometric variables, blood pressure, and serum lipids.16
We report the results of follow-up to December 31, 2005, a mean of 8.5 years. All EPIC-Norfolk participants were flagged for death certification at the Office for National Statistics, and vital status was obtained for the entire cohort. Participants admitted to a hospital were identified by their National Health Service number. Hospitals were linked to the East Norfolk Health Authority database, which identifies all hospital contacts throughout England and Wales for Norfolk residents. Participants were identified as having a CHD event during follow-up if CHD was the underlying cause of a hospital admission or death. Coronary heart disease was defined using International Classification of Diseases, Ninth Revision, codes 410 to 414 or International Classification of Diseases, Tenth Revision, codes I22 to I25. These codes encompass the clinical spectrum of CHD (eg, unstable angina, stable angina, and myocardial infarction). Previous validation studies in this cohort indicate high specificity of such case ascertainment.17
Baseline characteristics are summarized separately in men and women using means and percentages. The number of CHD events, person-years of follow-up, and hazard ratios and associated 95% confidence intervals (CIs) were calculated for different subgroups defined by published baseline covariates.1 We calculated the Framingham risk score in the EPIC-Norfolk cohort using published Cox proportional hazards regression coefficients for age, total cholesterol, HDL cholesterol, systolic blood pressure, diabetes mellitus, and smoking status.1 We then calculated 3 novel risk scores by fitting Cox proportional hazards regression models to the EPIC-Norfolk data, each with the log hazard of CHD as the outcome, separately in men and women, and with covariates included as follows: model A—age, total cholesterol, HDL cholesterol, systolic blood pressure, smoking status, and diabetes mellitus; model B—age, total cholesterol, HDL cholesterol, systolic blood pressure, smoking status, and HbA1c; and model C—age, total cholesterol, HDL cholesterol, systolic blood pressure, smoking status, diabetes mellitus, and HbA1c
To compare the discrimination of these risk scores in the EPIC-Norfolk data, we plotted receiver operating characteristic (ROC) curves and compared the areas under the ROC curves (AUROCs) using a nonparametric algorithm.18 We also computed a Bayes Information Criterion statistic to assess the global fit of each model.19 We examined the proportion of men and women who would be reclassified into higher- or lower-risk categories between models A and B and between models A and C20 and calculated the values of the net reclassification improvement (NRI) and the integrated discrimination improvement (IDI) comparing models A and B and models A and C.21 We calculated a further model using continuous measures of total cholesterol, HDL cholesterol, systolic blood pressure, and HbA1c, along with dichotomous smoking status and diabetes mellitus variables. Sensitivity analyses were conducted to examine possible differences in baseline characteristics between participants with and without HbA1c data and whether these might affect the comparison of the different risk scores.
All the analyses were completed using a statistical software program (Stata version 9.0; Stata Corp, College Station, Texas). The EPIC-Norfolk study was approved by the Norfolk Local Research Ethics Committee, and participants gave written consent before the first health check.
RESULTS
We excluded individuals with self-reported CHD at baseline (n = 798) and those with missing values for 1 or more of the variables used to calculate the Framingham risk score (n = 528). Because HbA1c measurement started approximately halfway through the data collection period, only 43.6% of the original sample had HbA1c values. The data set included 4513 men and 5782 women who had complete data available, including HbA1c levels.
Baseline characteristics of the study population are given in Table 1. Men and women were a mean (SD) age of 58 (10) years. Social class distributions were similar for the 2 sexes. Women had slightly higher baseline levels of total and HDL cholesterol but lower levels of systolic blood pressure. Men had a higher self-reported prevalence of diabetes mellitus and were slightly more likely to be current smokers than women. The mean (SD) HbA1c values were 5.3% (0.9%) and 5.3% (0.8%) in men and women, respectively.
|
|
|
|
Table 1. Baseline Characteristics of 10295 Participants From the EPIC–Norfolk Cohort, United Kingdom, 1993 to 2005
|
|
|
Table 2 provides the number of CHD events according to different subgroups, with associated person-years of follow-up and hazard ratios and 95% CIs. A total of 430 men (9.5%) and 250 women (4.3%) developed CHD during 8.5 years of follow-up. Age was significantly related to risk of CHD events at follow-up in both sexes. Total cholesterol was positively associated with risk of CHD events in men and women, with those in the highest cholesterol group at baseline (>282 mg/dL [to convert to millimoles per liter, multiply by 0.0259]) having the highest risk of a CHD event. Similar associations were seen for blood pressure. The HDL cholesterol level was inversely associated with risk of CHD events in men, but the trend was less clear in women. The hazard ratio for a 1% increase in HbA1c was 1.4 (95% CI, 1.3-1.5) for men and 1.5 (95% CI, 1.4-1.6) for women. Current smoking was associated with a significant risk of a CHD event in women but not in men.
|
|
|
|
Table 2. Number of CHD Events in Each Covariate Subgroup for Men and Women in the EPIC–Norfolk Cohort, United Kingdom, 1993 to 2005
|
|
|
Table 3 gives the regression coefficients underlying the original Framingham risk score and those generated from models A, B, and C. The AUROC for the original Framingham risk score in the EPIC-Norfolk was 0.71 for both men and women. In a Cox proportional hazards regression model using the Framingham variables (model A), the AUROC was 0.72 for men and 0.80 for women compared with 0.73 and 0.80, respectively, for a score that included a continuous measure of HbA1c (model B). In model C, which included diabetes mellitus and HbA1c, the AUROC remained at 0.73 for men and 0.80 for women. Comparing the ROC curves from models A and B showed that adding HbA1c to the Framingham risk score significantly improved discrimination in men (P = .005) but did not improve discrimination in women (P = .24). In contrast, comparing models A and C showed that having diabetes mellitus and HbA1c in the same model improved discrimination in men (P = .002) and in women (P = .04). The Bayes Information Criterion value was similar for all models in men and women, indicating good global model fit.
|
|
|
|
Table 3. Regression Coefficients Underlying CHD Prediction Models Tested in the EPIC–Norfolk Cohort (1993-2005), Including the AUROC and the Bayes Information Criteria, for Men and Women
|
|
|
Reclassifications are summarized in Table 4 and Table 5. In 430 men who experienced events, model B improved classification in 32 but worsened it in 20, a net gain in reclassification of 2.8%. In the 4080 men who did not experience an event, model B reclassified 244 down and 220 up, a net gain in reclassification of 0.6%. The NRI for model B over model A was, therefore, estimated to be 3.4% (P = .06). There was a small increase in the IDI of 0.52% (P = .06) that can be interpreted as the increase in mean sensitivity given no changes in specificity. The NRI for model C over model A was 2.6% (P = .12), whereas the increase in the IDI was 0.58% (P = .02). In women, the NRI was actually negative comparing model B with model A and model C with model A, although the values were not statistically significant.
|
|
|
|
Table 4. CHD Risk Classification Comparing Model A With Model B and Model A With Model C for Men and Women in the EPIC–Norfolk Cohort, 1993 to 2005a
|
|
|
|
|
|
|
Table 5. Improvement Variables Comparing Model A With Model B and Model A With Model C for Men and Women in the EPIC–Norfolk Cohort, 1993 to 2005
|
|
|
The Figure shows the ROC curves for models A, B, and C separately for men and women. The shapes of all 6 ROC curves are similar. The continuous model showed identical AUROC values compared with models B and C for men and women.
|
|
|
|
Figure. Area under the receiver operating characteristic curves for models A (blue), B (red), and C (green) in male (A) and female (B) participants in the European Prospective Investigation of Cancer–Norfolk cohort (March 1993-December 2005).
|
|
|
Sensitivity analyses demonstrated that there were differences between participants with and without HbA1c data for certain baseline characteristics (eg, in men and women, those without HbA1c data were significantly older and had higher total cholesterol levels, HDL cholesterol levels, and systolic blood pressure [data not shown]). To examine whether this difference would affect the comparison of the calculated risk scores, we compared theAUROC for a score developed on all participants using all the variables in the original Framingham risk score with that of a score developed on the subset of participants we actually used to fit model B (ie, excluding participants with a missing value for HbA1c). TheAUROCs were similar in the 2 analyses, indicating that the comparison of the risk scores was not affected by the exclusion of participants with missing HbA1c values.
COMMENT
The Framingham risk score performs reasonably well at predicting CHD events in the EPIC-Norfolk cohort, with an AUROC of 0.71 for men and women. Including HbA1c as a continuous variable made a small (AUC, 1%) but statistically significant improvement to discrimination in men but did not significantly improve discrimination in women. Reclassification was improved slightly in men but, if anything, worsened slightly, albeit not significantly, in women after the inclusion of HbA1c. Including diabetes mellitus and HbA1c improved discrimination in men and women, although reclassification was not improved. Thus, the AUC, the NRI, and the IDI lead to broadly similar conclusions. These data reinforce the idea that CHD risk models are associated with misclassification20 and that the addition of a risk factor that did not make a large difference to the AUC can lead to some reclassification of men and women into different risk categories. These findings have implications for decision making in clinical practice and represent a timely addition to the current debate on cardiovascular risk algorithms.20, 22-23
The predictive value of the Framingham risk score in the EPIC-Norfolk cohort is lower than that reported in the original Framingham population.1 This is not surprising given changes in the nature and distribution of cardiovascular risk factors across time, within and between populations. A recent systematic review3 of 27 external validity studies found that the performance of the Framingham risk score varies considerably between different countries and ethnic groups. Predicted to observed ratios ranged from an underprediction of 0.43 in a higher-risk population to an overprediction of 2.87 in lower-risk populations.3 Even in the United Kingdom, regional differences in the risk of CHD mean that the accuracy of the Framingham risk score varies, with overestimation in areas of low incidence6 and underestimation in socially deprived areas, where the incidence of heart disease is high.24 Such findings have led to calls to refine the score to identify and target those at high risk and to target appropriate preventive action.6
The rationale for including HbA1c as a continuous variable had statistical and practical value in this analysis. Grouping the values of continuous variables into 2 or more categories can lead to (1) a reduction in statistical power as information is lost, (2) possible underestimation of the extent of variation in outcome between groups, and (3) concealment of any nonlinearity in the relation between variable and outcome.25 At the same time, caution must be exercised in searching for and adding new variables to a CHD prediction model. Colinearity or coexposure to high blood glucose levels and other cardiovascular risk factors, such as smoking, high blood pressure, and dyslipidemia, may have accounted for the lack of extra discrimination when HbA1c was added to the prediction algorithm in women and the small increase seen in men. Indeed, despite the inclusion of variables that have been shown to be consistently and independently associated with CHD, the models presented in this article showed only modest increases in predictive value. A risk factor must be very strongly associated with a disorder to be a worthwhile screening test, and the strength of the association required to give useful discrimination is often underestimated.26
Glycated hemoglobin has been shown to be an independent predictor of cardiovascular events and mortality from all causes in people with12 and without13-14 diabetes mellitus. However, randomized controlled trials27-28 evaluating the efficacy of glucose-lowering drugs have largely not shown a significant reduction in cardiovascular events, which remain the major cause of morbidity, mortality, and health service costs in diabetic patients. In the UK Prospective Diabetes Study,28 strict control of blood glucose levels significantly reduced microvascular complications by 25%, but the study had insufficient power to detect a difference in myocardial infarction. Prevention trials in people with impaired glucose tolerance demonstrate that lifestyle and pharmacologic intervention can reduce progression to diabetes mellitus,29-31 but it has not yet been shown that such interventions also diminish the risk of macrovascular complications. Indeed, in the recent DREAM (Diabetes REduction Assessment with ramipril and rosiglitazone Medication) trial,31 although rosiglitazone (8 mg/d) substantially decreased the risk of progression to diabetes mellitus across 3 years, it was not associated with the expected reduction in cardiovascular risk.
There is a need for closer integration of programs for the assessment of risk of CHD and diabetes mellitus in populations. In addition to enhancing CHD prediction, a simple finger prick test for HbA1c might improve risk stratification and the management of hyperglycemia and might inform further testing to identify individuals who would benefit from a diabetes mellitus prevention program. However, issues such as cost, standardization, and interpretation in regions with a high prevalence of hemoglobinopathies need consideration before such a program could be implemented.
Measurement errors in determining HbA1c concentrations and cardiovascular disease outcomes may have been present in this analysis. Blood samples were nonfasting and were collected at different times of the day. We did not have information on specific medications that may have affected glucose metabolism. Four-fifths of the CHD events were nonfatal and were identified by linking records with hospital admission data. Although we could ascertain all deaths in the EPIC-Norfolk, we could not identify all nonfatal cardiovascular events. However, previous validation studies in this cohort indicate high specificity of such case ascertainment.17 Hospital admission data probably underestimate nonfatal CHD events because not all of them result in hospital admission. Nevertheless, this method probably identifies the nonfatal events of most clinical importance (eg, those resulting in hospital admission). Diagnostic codes for CHD on death certificates may have been inaccurate,32 and underestimation or random misclassification of CHD events in this cohort may have attenuated the relationship with HbA1c. There may have been limited power in the models for women because the event rate was low; this may explain the blood pressure coefficients, which do not all fall in the expected direction of effect for models A, B and C. Furthermore, we used logistic AUCs to compare the models, which do not take account of time to event, but we would not expect this to make a large difference to the overall conclusions.
In this large, population-based cohort, we found a significant improvement in the prediction of absolute CHD risk in men with the inclusion of HbA1c in a Framingham-type risk model. We found no significant improvement in prediction for women. Although HbA1c may not be a powerful predictor of CHD in this cohort, the degree of reclassification between treatment threshold categories indicates that it still may be important for individual clinical decision making. Further testing of a Framingham-type model including HbA1c in other cohorts is recommended. It remains unclear whether reduction of blood glucose levels could diminish CHD risk, and randomized controlled trials are needed to assess the effect of lowering blood glucose levels on CHD events in people without diabetes mellitus.
AUTHOR INFORMATION
Correspondence: Simon J. Griffin, DM, MRC Epidemiology Unit, Institute of Metabolic Science, Box 285, Addenbrooke's Hospital, Hills Road, Cambridge CB2 0QQ, England (simon.griffin{at}mrc-epid.cam.ac.uk).
Accepted for Publication: November 19, 2007.
Author Contributions: Dr Simmons had full access to all of the data in the study and takes responsibility for the accuracy of the data analysis, and Drs Khaw and Wareham take responsibility for the integrity of the data. Study concept and design: Boekholdt, Sargeant, Khaw, Wareham, and Griffin. Acquisition of data: Khaw and Wareham. Analysis and interpretation of data: Simmons, Sharp, Boekholdt, Khaw, Wareham, and Griffin. Drafting of the manuscript: Simmons, Sharp, Wareham, and Griffin. Critical revision of the manuscript for important intellectual content: Sharp, Boekholdt, Sargeant, Khaw, Wareham, and Griffin. Statistical analysis: Simmons, Sharp, Boekholdt, and Griffin. Obtained funding: Khaw and Wareham. Administrative, technical, and material support: Khaw and Griffin. Study supervision: Wareham and Griffin.
Financial Disclosure: None reported.
Funding/Support: This study was supported by the Medical Research Council, Cancer Research UK, the British Heart Foundation, the European Union (Europe Against Cancer Programme), the Stroke Association, Wellcome Trust, Research Into Ageing, and the Academy of Medical Sciences.
Role of the Sponsors: The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.
Additional Contributions: We gratefully acknowledge the contributions of the EPIC-Norfolk participants and the EPIC-Norfolk team.
Author Affiliations: MRC Epidemiology Unit, Institute of Metabolic Science, Addenbrooke's Hospital, Cambridge, England (Drs Simmons, Sargeant, Wareham, and Griffin and Mr Sharp); Department of Cardiology, Academic Medical Center, Amsterdam, the Netherlands (Dr Boekholdt); and Department of Clinical Gerontology, University of Cambridge School of Clinical Medicine, Addenbrooke's Hospital, Cambridge (Dr Khaw).
REFERENCES
| |
1. Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837-1847.
FREE FULL TEXT
2. Leaverton PE, Sorlie PD, Kleinman JC; et al. Representativeness of the Framingham risk model for coronary heart disease mortality: a comparison with a national cohort study. J Chronic Dis. 1987;40(8):775-784.
FULL TEXT
|
ISI
| PUBMED
3. Brindle P, Beswick A, Fahey T, Ebrahim SB. Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review. Heart. 2006;92(12):1752-1759.
FREE FULL TEXT
4. Empana JP, Ducimetiere P, Arveiler D; et al. Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? the PRIME Study. Eur Heart J. 2003;24(21):1903-1911.
FREE FULL TEXT
5. Hense HW, Schulte H, Lowel H, Assmann G, Keil U. Framingham risk function overestimates risk of coronary heart disease in men and women from Germany: results from the MONICA Augsburg and the PROCAM cohorts. Eur Heart J. 2003;24(10):937-945.
FREE FULL TEXT
6. Brindle P, Emberson J, Lampe F; et al. Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ. 2003;327(7426):1267.
FREE FULL TEXT
7. Farmer A. Metabolic syndrome and mortality. BMJ. 2006;332(7546):882.
FREE FULL TEXT
8. Coutinho M, Gerstein HC, Wang Y, Yusuf S. The relationship between glucose and incident cardiovascular events: a metaregression analysis of published data from 20 studies of 95,783 individuals followed for 12.4 years. Diabetes Care. 1999;22(2):233-240.
FREE FULL TEXT
9. Levitan EB, Song Y, Ford ES, Liu S. Is nondiabetic hyperglycemia a risk factor for cardiovascular disease? a meta-analysis of prospective studies. Arch Intern Med. 2004;164(19):2147-2155.
FREE FULL TEXT
10. Gerstein HC. Glucose: a continuous risk factor for cardiovascular disease. Diabet Med. 1997;14(suppl 3):S25-S31.
FULL TEXT
|
ISI
| PUBMED
11. Khaw KT, Wareham N. Glycated hemoglobin as a marker of cardiovascular risk. Curr Opin Lipidol. 2006;17(6):637-643.
ISI
| PUBMED
12. Selvin E, Marinopoulos S, Berkenblit G; et al. Meta-analysis: glycosylated hemoglobin and cardiovascular disease in diabetes mellitus. Ann Intern Med. 2004;141(6):421-431.
FREE FULL TEXT
13. Khaw KT, Wareham N, Bingham S, Luben R, Welch A, Day N. Association of hemoglobin A1c with cardiovascular disease and mortality in adults: the European prospective investigation into cancer in Norfolk. Ann Intern Med. 2004;141(6):413-420.
FREE FULL TEXT
14. Selvin E, Coresh J, Golden SH, Brancati FL, Folsom AR, Steffes MW. Glycemic control and coronary heart disease risk in persons with and without diabetes: the atherosclerosis risk in communities study. Arch Intern Med. 2005;165(16):1910-1916.
FREE FULL TEXT
15. Gerstein HC. Glycosylated hemoglobin: finally ready for prime time as a cardiovascular risk factor. Ann Intern Med. 2004;141(6):475-476.
FREE FULL TEXT
16. Day N, Oakes S, Luben R; et al. EPIC-Norfolk: study design and characteristics of the cohort. Br J Cancer. 1999;80(suppl 1):95-103.
PUBMED
17. Boekholdt SM, Peters RJ, Day NE; et al. Macrophage migration inhibitory factor and the risk of myocardial infarction or death due to coronary artery disease in adults without prior myocardial infarction or stroke: the EPIC-Norfolk Prospective Population study. Am J Med. 2004;117(6):390-397.
FULL TEXT
|
ISI
| PUBMED
18. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845.
FULL TEXT
|
ISI
| PUBMED
19. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928-935.
FREE FULL TEXT
20. Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297(6):611-619.
FREE FULL TEXT
21. Pepe MS, Feng Z, Gu JW. Comments on "Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond" by M. J. Pencina et al., Statistics in Medicine. Stat Med. 2008;27(2):173-181.
FULL TEXT
|
ISI
| PUBMED
22. Pencina MJ, Vasan RS, D'Agostino RB Sr. Algorithms for assessing cardiovascular risk in women [letter]. JAMA. 2007;298(2):175-176.
FREE FULL TEXT
23. Wang TJ, Kathiresan S, Lloyd-Jones DM. Algorithms for assessing cardiovascular risk in women [letter]. JAMA. 2007;298(2):176.
FREE FULL TEXT
24. Brindle PM, McConnachie A, Upton MN, Hart CL, Davey Smith G, Watt GC. The accuracy of the Framingham risk-score in different socioeconomic groups: a prospective study. Br J Gen Pract. 2005;55(520):838-845.
ISI
| PUBMED
25. Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ. 2006;332(7549):1080.
FREE FULL TEXT
26. Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999;319(7224):1562-1565.
FREE FULL TEXT
27. Home PD, Pocock SJ, Beck-Nielsen H; et al. Rosiglitazone evaluated for cardiovascular outcomes: an interim analysis. N Engl J Med. 2007;357(1):28-38.
FREE FULL TEXT
28. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352(9131):837-853.
FULL TEXT
|
ISI
| PUBMED
29. Tuomilehto J, Lindstrom J, Eriksson JG; et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344(18):1343-1350.
FREE FULL TEXT
30. Knowler WC, Barrett-Connor E, Fowler SE; et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346(6):393-403.
FREE FULL TEXT
31. Gerstein HC, Yusuf S, Bosch J; et al. Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose tolerance or impaired fasting glucose: a randomised controlled trial. Lancet. 2006;368(9541):1096-1105.
FULL TEXT
| PUBMED
32. Lloyd-Jones DM, Martin DO, Larson MG, Levy D. Accuracy of death certificates for coding coronary heart disease as the cause of death. Ann Intern Med. 1998;129(12):1020-1026.
FREE FULL TEXT
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati
What's this?
RELATED ARTICLE
Evaluation of the Framingham Risk Score in the European Prospective Investigation of Cancer–Norfolk Cohort—Invited Commentary
Michael J. Pencina and Ralph B. D’Agostino
Arch Intern Med. 2008;168(11):1216-1218.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Evaluation of the Framingham Risk Score in the European Prospective Investigation of Cancer-Norfolk Cohort--Invited Commentary
Pencina and D'Agostino
Arch Intern Med 2008;168:1216-1218.
FULL TEXT
|
