 |
|
Validation and Comparison of a Novel Screening Guideline for Kidney Disease: KEEPing SCORED
Heejung Bang, PhD;
Madhu Mazumdar, PhD;
Lisa M. Kern, MD, MPH;
David A. Shoham, PhD;
Phyllis A. August, MD, MPH;
Abhijit V. Kshirsagar, MD, MPH
Arch Intern Med. 2008;168(4):432-435.
Chronic kidney disease (CKD) is one of the world's major public health problems. Nearly 1 in 9 adults (20 million people) in the United States have CKD, and it is estimated that another 20 million are at increased risk.1 Given the asymptomatic nature of kidney disease, affected individuals and health care providers may be unaware of the condition in patients. Identifying individuals with early kidney disease would be a useful first step in preventing progression to end-stage renal disease as well as reducing morbidity and mortality from cardiovascular disease (CVD).
We recently published an instrument (SCreening for Occult REnal Disease [SCORED]) to systematically identify individuals with a high likelihood of prevalent CKD.2 Derived from the National Health and Nutrition Examination Survey (NHANES) 1999-2002 and (partially) validated in the Atherosclerosis Risk in Communities (ARIC) study, SCORED identified 9 demographic and medical variables that could be assigned integer values and then entered into a scoring algorithm. The scoring algorithm was intentionally simplified to be accessible to lay persons and health care providers.
To our knowledge, SCORED is the only algorithm derived from scientific modeling, rather than expert scientific opinion, targeted for general population screening. Alternatively, the National Kidney Foundation has encouraged screening strategies targeted at high-risk groups. Through the Kidney Early Evaluation Program (KEEP), the National Kidney Foundation recommends using the following characteristics to identify individuals with a high likelihood of kidney disease: "if a person is 18 years or older and has one or more of the following: diabetes; high blood pressure; or a family history of diabetes, high blood pressure or kidney disease."3-4
It is unclear how the SCORED algorithm compares with the KEEP guidelines. Such comparisons would be useful for clinicians and others. Since the publication of SCORED, we have performed an additional validation study using new, independent samples: (1) NHANES 2003-2004 and (2) a combined cohort of the ARIC study and Cardiovascular Heart Study (ARIC/CHS). We report the findings herein.
Methods
The ARIC study enrolled 15 792 participants aged 45 to 64 years between 1987 and 1989, and CHS recruited 5201 subjects 65 years and older between 1989 and 1990. Both are community studies, and detailed descriptions have been published previously.2, 5-6 Some data disparities and limitations to be noted are summarized in a footnote of the Table.
|
|
|
|
Table. Diagnostic Performance of SCORED vs KEEP Using Validation Datasets (NHANES 2003-2004 [N = 4298] and ARIC/CHS [N = 21 221])a
|
|
|
SCORED Model
The SCORED model is a multivariable mathematical function that gives an estimated probability of having CKD as follows:
Probability(CKD) = 1/[1 + exp(–β" x x)],
where
β" x x = –5.4 + 1.55 x I(Age 50-59 years) + 2.31 x I(Age 60-69 years) + 3.23 x I(Age  70 years) + 0.29 x I(Female) + 0.93 x I(Anemia) + 0.45 x I(Hypertension) + 0.44 x I(Diabetes) + 0.59 x I(History of CVD) + 0.45 x I(History of Heart Failure) + 0.74 x I(Peripheral Vascular Disease) + 0.83 x I(Proteinuria).
The total integer score is a weighted sum, ranging from 0 to 12, and is calculated as follows:
Total score = 2 x I(Age 50-59 years) + 3 x I(Age 60-69 years) + 4 x I(Age 70 years) + 1 x I(Female) + 1 x I(Anemia) + 1 x I(Hypertension) +1 x I(Diabetes) + 1 x I(History of CVD) + 1 x I(History of Heart Failure) + 1 x I(Peripheral Vascular Disease) + 1 x I(Proteinuria),
where I(A) is an indicator taking 1 for condition A and 0 for otherwise. If the total score is greater than or equal to 4, a confirmatory blood examination (eg, serum creatinine and/or urinalysis) by a physician is strongly recommended. A user-friendly questionnaire is also provided in the original article.2
Measures for Validation and Comparison
Creatinine level was calibrated in all studies as recommended by the National Kidney Disease Education Program and the National Center for Health Statistics.7-9 Kidney function was quantified by glomerular filtration rate from the 4-variable Modification of Diet in Renal Disease study equation, and CKD was defined as a glomerular filtration rate lower than 60 mL/min/1.73m2, which corresponds to stage 3 or higher kidney disease.3, 10
The SCORED model was evaluated in the validation datasets using standard measures: percentage of people identified to be at high risk by the given rule, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC).
Because the ARIC study and CHS did not measure urinary protein level for proteinuria, we created 3 different approaches to handling missing or incomplete information and assessed sensitivity of the estimates of the selected measures. First, we started the analysis with ARIC/CHS datasets without proteinuria, which reflects the situation that no one reported that she or he had proteinuria. Second, we substituted urinary protein level measured at the year 5 follow-up visit for CHS subjects if available or imputed the measurement via a missing data analysis method otherwise.11 Third, we defined the highest 10 percentiles of uric acid level as proteinuria (to compare, 9.5% of subjects had proteinuria in NHANES). (The rationale for using high uric acid level as a surrogate marker for proteinuria, at least for sensitivity checking, is that it is used in the diagnosis and treatment of renal disorders, including renal failure or injury and is associated with proteinuria.12)
We compared SCORED vs KEEP using the same set of diagnostic measures. It is important to note that none of the datasets we examined contained information on "family history of kidney disease," a criterion needed for KEEP. As a result, the SCORED model could not examine the effect of this potentially important factor. The absence of this data was also a problem for the direct comparison between KEEP vs SCORED. To improve comparability, we repeated our analysis in 2 scenarios: (1) ignoring this criterion and (2) after applying the fill-in method based on missing data imputation and replacement by surrogate variables. These statistical strategies, although imperfect, would offer a sound method of sensitivity analysis in practice.
Results
For the NHANES dataset, sensitivity, specificity, positive predictive value, and negative predictive value of SCORED were estimated to be 95%, 65%, 20%, and 99%, respectively, with an AUC of 0.88. The application to ARIC/CHS provided sensitivity of 88% to 90%, specificity of 50% to 52%, positive predictive value of 13% to 14%, and negative predictive value of 98%, with an AUC of 0.78 to 0.80 (Table). In contrast, KEEP tended to place larger proportions of subjects into the high-risk category than SCORED (67%-77% vs 40%-53%), and the sensitivity for detecting CKD was 86% to 92%, with a specificity of 24% to 35%, positive predictive value of 3% to 12%, negative predictive value of 95% to 98%, and an AUC of 0.65 to 0.77. Results were robust under the different assumptions and approaches we adopted. Our comparisons give some indication about the expected performance of the 2 screening programs in real-world settings.
Comment
We report the diagnostic characteristics of the SCORED model and its performance compared with the KEEP guidelines. Ninety-five percent of individuals with CKD were identified by the SCORED algorithm as being at high risk (cutoff of 4); among those who did not have CKD, 65% scored less than 4 in NHANES. Those values were somewhat reduced in the ARIC/CHS samples, possibly owing to differences in population characteristics, variable definitions, and available data.
The SCORED model exhibited improved performances in all the criteria we examined. For KEEP as well as SCORED, NHANES showed better results than ARIC/CHS. Based on the higher AUC of the SCORED model compared with the KEEP guidelines, the improved performance characteristics are likely because of the use of additional variables and different weights for age groups. Indeed, questions about underlying CVD, common in the CKD population as demonstrated by the KEEP data,13 may improve their screening efforts.
There are also some limitations. Some variables that are a part of SCORED or KEEP were unavailable or incomplete in the databases we used, including family history of kidney disease and proteinuria. We addressed this issue with different statistical strategies. The absence of these variables in well-known studies may reflect low awareness of these important risk factors. We encourage the collection of such information in future epidemiological studies including the next iteration of NHANES. Furthermore, there are known negative consequences of screening. Overdiagnosis, misdiagnosis, anxiety, cost, and creating a false sense of security are some adverse effects of screening and render screening in some diseases controversial.14-18 We propose that SCORED would minimize the potential adverse outcomes associated with screening by identifying approximately 25% fewer screeners as being at high risk compared with KEEP. The greater specificity of SCORED may reduce overdiagnosis and enhance resource use.
Screening is a public health strategy for identifying an unrecognized disease in asymptomatic populations, in which subjects are asked questions or offered a test, to identify those individuals who are more likely to be helped than harmed by further tests or treatment to reduce the risk of a disease or its complications. Diseases suitable for screening are those with serious consequences, those in which treatment is more effective at an earlier stage, and conditions with a long preclinical phase.16 Chronic kidney disease fulfills these criteria; however, it is not known whether screening will in fact result in improved outcomes and reduced progression to end-stage renal disease.17 The benefits of screening for CKD and its effectiveness and cost-effectiveness are yet to be determined.19
In conclusion, SCORED and KEEP are simple and inexpensive to use and carry some, albeit, minor, consequences of a false-positive result. Because of its general population sampling, we believe that SCORED has greater applicability and could also serve as an educational tool to raise CKD awareness. There is ample evidence that CKD is underdiagnosed and undertreated and that the burden of CKD is increasing worldwide.20 Not many people are aware that undiagnosed CKD can lead to serious consequences, including kidney failure, dialysis, and even death, and is associated with CVD. We propose that increased screening would identify greater numbers of individuals who may potentially benefit from strategies known to slow the progression of renal disease, a hypothesis that warrants testing in prospective clinical trials. We should "act on the best available evidence—as opposed to waiting for the best possible evidence."21(p111)
AUTHOR INFORMATION
Correspondence: Dr Bang, Division of Biostatistics and Epidemiology, Department of Public Health, Weill Medical College of Cornell University, 411 E 69th St, New York, NY 10021 (heb2013{at}med.cornell.edu).
Author Contributions: Dr Bang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bang, Mazumdar, Shoham, and Kshirsagar. Acquisition of data: Bang. Analysis and interpretation of data: Bang, Mazumdar, Kern, Shoham, August, and Kshirsagar. Drafting of the manuscript: Bang, Shoham, August, and Kshirsagar. Critical revision of the manuscript for important intellectual content: Bang, Mazumdar, Kern, and Kshirsagar. Statistical analysis: Bang, Mazumdar, and Shoham. Obtained funding: Mazumdar. Administrative, technical, and material support: Kern and Kshirsagar. Study supervision: August and Kshirsagar.
Financial Disclosure: None reported.
Funding/Support: The ARIC and CHS studies are conducted and supported by the National Heart, Lung, and Blood Institute (NHLBI) in collaboration with ARIC and CHS investigators. Drs Bang and Mazumdar were partially supported by the Tolly Vinik Trust through the Institute of Clinical Research of Weill Medical College of Cornell University. Dr Mazumdar was additionally supported by grant R25CA105012 (Collaborative Program in Nutrition and Cancer Prevention). Dr Shoham is a postdoctoral trainee supported by NHLBI institutional training grant 5T-32HL007055-29.
Disclaimer: This manuscript was prepared using a limited-access dataset obtained from the NHLBI and does not necessarily reflect the opinions or views of the ARIC/CHS or NHLBI.
Additional Contributions: Kylie Braynt, MS, provided CHS data setup and Sean Coady, MA, at NHLBI assisted in data issues and clarifications. We thank the staff and participants of the NHANES, ARIC study, and CHS for their important contributions and valuable information for health research.
REFERENCES
|
1. Coresh J, Byrd-Holt D, Astor BC; et al. Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol. 2005;16(1):180-188.
FREE FULL TEXT
2. Bang H, Vupputuri S, Shoham DA; et al. SCreening for Occult REnal Disease (SCORED). Arch Intern Med. 2007;167(4):374-381.
FREE FULL TEXT
3. National Kidney Foundation. K/DOQI clinical practice guidelines for chronic kidney disease. Am J Kidney Dis. 2002;39(2)(suppl 1):S1-S266.
FULL TEXT
|
ISI
| PUBMED
4. National Kidney Foundation. Kidney Early Evaluation Program. Am J Kidney Dis. 2005;45(2)(suppl 2):S1-S135.
PUBMED
5. The ARIC investigators. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol. 1989;129(4):687-702.
FREE FULL TEXT
6. Fried LP, Borhani NO, Enright P; et al. The Cardiovascular Health Study: design and rationale. Ann Epidemiol. 1991;1(3):263-276.
PUBMED
7. Coresh J, Astor BC, McQuillan G; et al. Calibration and random variation of the serum creatinine assay as critical elements of using equations to estimate glomerular filtration rate. Am J Kidney Dis. 2002;39(5):920-929.
FULL TEXT
|
ISI
| PUBMED
8. Myers GL, Miller WG, Coresh J; et al. Recommendations for improving serum creatinine measurement. Clin Chem. 2006;52(1):5-18.
FREE FULL TEXT
9. Weiner DE, Tighiouart H, Amin MG; et al. Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol. 2004;15(5):1307-1315.
FREE FULL TEXT
10. Levey AS, Green T, Kusek J, Beck G, MDRD Study Group. A simplified equation to predict glomerular filtration rate from serum creatinine [abstract]. J Am Soc Nephrol. 2000;11(suppl):155A.
11. SAS Institute Inc. SAS/STAT User's Guide Version 8.2. Cary, NC: SAS Institute Inc; 2000.12. Lee JE, Kim Y-G, Choi Y-H, Huh W, Kim DJ, Oh HY. Serum uric acid is associated with microalbuminuria in prehypertension. Hypertension. 2006;47(5):962-967.
FREE FULL TEXT
13. McCullough PA, Jurkovitz CT, Pergola PE; et al. Independent components of chronic kidney disease as a cardiovascular risk state. Arch Intern Med. 2007;167(11):1122-1129.
FREE FULL TEXT
14. McClellan WM, Ramirez SPB, Jurkovitz C. Screening for chronic kidney disease: unresolved issues. J Am Soc Nephrol. 2003;14(7)(suppl 2):S81-S87.
FREE FULL TEXT
15. Gansevoort RT, Bakker SJL, de Jong PE. Early detection of progressive chronic kidney disease: is it feasible? J Am Soc Nephrol. 2006;17(5):1218-1220.
FREE FULL TEXT
16. Wilson J, Jungner G. The Principles and Practice of Screening for Disease. Geneva, Switzerland: World Health Organization; 1968.17. Hallan SI, Dahl K, Oien CM; et al. Screening strategies for chronic kidney disease in the general population. BMJ. 2006;333(7577):1047.
FREE FULL TEXT
18. Freedman DA, Petitti DB, Robins JM. On the efficacy of screening for breast cancer. Int J Epidemiol. 2004;33(1):43-55.
FREE FULL TEXT
19. Bang H. Medical cost analysis: application to colorectal cancer data from the SEER Medicare database. Contemp Clin Trials. 2005;26(5):586-597.
FULL TEXT
|
ISI
| PUBMED
20. Meguid El Nahas A, Bello AK. Chronic kidney disease: the global challenge. Lancet. 2005;365(9456):331-340.
ISI
| PUBMED
21. Kaplah JP, ed, Liverman CT, ed, Kraak VI, ed. Preventing Childhood Obesity: Health in the Balance. Washington, DC: Institute of Medicine; 2004.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Screening for kidney disease in vascular patients: SCreening for Occult REnal Disease (SCORED) experience
Bang et al.
Nephrol Dial Transplant 2009;24:2452-2457.
ABSTRACT
| FULL TEXT
A Simple Algorithm to Predict Incident Kidney Disease
Kshirsagar et al.
Arch Intern Med 2008;168:2466-2473.
ABSTRACT
| FULL TEXT
|
