This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (143)  • Contact me when this article is cited  Related Content  • Similar articles in this journal  Social Bookmarking   What's this?

Khalid S. Khan, FCPS; Salim Daya, MB, MSc; Alejandro R. Jadad, MD, DPhil

Arch Intern Med. 1996;156(6):661-666.

Abstract
Background
Traditional and largely qualitative reviews of evidence are now giving way to much more structured systematic overviews that use a quantitative method to calculate the overall effect of treatment. The latter approach is dependent on the quality of primary studies, which may introduce bias if they are of poor methodologic quality.

Objective
To test the hypothesis that the inclusion of poor-quality trials in meta-analyses would bias the conclusions and produce incorrect estimates of treatment effect.

Methods
An overview of randomized trials of antiestrogen therapy in subfertile men with oligospermia was performed to test the hypothesis. Data sources included online searching of MEDLINE and Science Citation Index databases between 1966 and 1994, scanning the bibliography of known primary studies and review articles, and contacting experts in the field. After independent, blind assessment, nine of 149 originally identified studies met the inclusion criteria and were selected. We assessed study quality independently. Outcome data from each study were pooled and statistically summarized. Results: There was a marginal improvement in pregnancy rate with antiestrogen treatment (odds ratio, 1.6; 95% confidence interval, 0.9 to 2.6). Sensitivity analyses on the basis of methodologic quality demonstrated that poor-quality studies produced a positive effect with treatment, whereas no benefit was observed with high-quality studies.

Conclusions
The results of a meta-analysis are influenced by the quality of the primary studies included. Methodologically, poor studies tend to exaggerate the overall estimate of treatment effect and may lead to incorrect inferences.

(Arch Intern Med. 1996;156:661-666)

Author Affiliations
From the International Scholars Programme, Department of Clinical Epidemiology and Biostatistics (Dr Khan), Departments of Obstetrics and Gynaecology (Dr Daya), and Clinical Epidemiology and Biostatistics (Drs Daya and Jadad), McMaster University, Hamilton, Ontario.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Oral anticoagulants and non-cardioembolic stroke prevention
Schachter et al.
Vasc Med 2008;13:55-62.
ABSTRACT  

Meta-Analysis
Cleophas and Zwinderman
Circulation 2007;115:2870-2875.
FULL TEXT  

Palliative Radiotherapy Trials for Bone Metastases: A Systematic Review
Chow et al.
JCO 2007;25:1423-1436.
ABSTRACT | FULL TEXT  

The Use of Herbal Medicine in Alzheimer's Disease--A Systematic Review
Santos-Neto et al.
Evid Based Complement Alternat Med 2006;3:441-445.
ABSTRACT | FULL TEXT  

Fluoroquinolones vs {beta}-Lactams for Empirical Treatment of Immunocompetent Patients With Skin and Soft Tissue Infections: A Meta-analysis of Randomized Controlled Trials
Falagas et al.
Mayo Clin Proc. 2006;81:1553-1566.
ABSTRACT | FULL TEXT  

The role of aminoglycosides in combination with a {beta}-lactam for the treatment of bacterial endocarditis: a meta-analysis of comparative trials
Falagas et al.
J Antimicrob Chemother 2006;57:639-647.
ABSTRACT | FULL TEXT  

Does immunity regulate ejaculate quality and fertility in humans?
Skau and Folstad
Behav Ecol 2005;16:410-416.
ABSTRACT | FULL TEXT  

Meta-Analyses in Orthopaedic Surgery : A Systematic Review of Their Methodologies
Bhandari et al.
JBJS 2003;83:15-15.
ABSTRACT | FULL TEXT  

Do bacterial infections cause reduced ejaculate quality? A meta-analysis of antibiotic treatment of male infertility
Skau and Folstad
Behav Ecol 2003;14:40-47.
ABSTRACT | FULL TEXT  

Reviews of acupuncture for chronic neck pain: pitfalls in conducting systematic reviews
White et al.
Rheumatology (Oxford) 2002;41:1224-1231.
ABSTRACT | FULL TEXT  

User's Guide to the Orthopaedic Literature: How to Use a Systematic Literature Review
Bhandari et al.
JBJS 2002;84:1672-1682.
FULL TEXT  

Corticosteroid Therapy in Pulmonary Sarcoidosis: A Systematic Review
Paramothayan and Jones
JAMA 2002;287:1301-1307.
ABSTRACT | FULL TEXT  

The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture
Linde et al.
Int J Epidemiol 2001;30:526-531.
ABSTRACT | FULL TEXT  

Drug-Related Adverse Events: A Readers' Guide to Assessing Literature Reviews and Meta-analyses
Ross
Arch Intern Med 2001;161:1041-1046.
FULL TEXT  

A Scale of Methodological Quality for Clinical Studies of Radiologic Examinations
Arrivé et al.
Radiology 2000;217:69-74.
ABSTRACT | FULL TEXT  

How Well Are Randomized Controlled Trials Reported in the Dermatology Literature?
Adetugbo and Williams
Arch Dermatol 2000;136:381-385.
ABSTRACT | FULL TEXT  

A Meta-analysis Comparing Low-Molecular-Weight Heparins With Unfractionated Heparin in the Treatment of Venous Thromboembolism: Examining Some Unanswered Questions Regarding Location of Treatment, Product Type, and Dosing Frequency
Dolovich et al.
Arch Intern Med 2000;160:181-188.
ABSTRACT | FULL TEXT  

Benzodiazepine use in pregnancy and major malformations or oral clefts
Game et al.
BMJ 1999;319:918-918.
FULL TEXT  

The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Juni et al.
JAMA 1999;282:1054-1060.
ABSTRACT | FULL TEXT  

Tacrolimus versus cyclosporin for immunosuppression in renal transplantation: meta-analysis of randomised trials
Knoll and Bell
BMJ 1999;318:1104-1107.
ABSTRACT | FULL TEXT  

Preoperative vs Postoperative Initiation of Low-Molecular-Weight Heparin Prophylaxis Against Venous Thromboembolism in Patients Undergoing Elective Hip Replacement
Hull et al.
Arch Intern Med 1999;159:137-141.
ABSTRACT | FULL TEXT  

Efficacy of Antiseptic-Impregnated Central Venous Catheters in Preventing Catheter-Related Bloodstream Infection: A Meta-analysis
Veenstra et al.
JAMA 1999;281:261-267.
ABSTRACT | FULL TEXT  

Complementary Medicine and the Cochrane Collaboration
Ezzo et al.
JAMA 1998;280:1628-1630.
FULL TEXT  

The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials
Kunz and Oxman
BMJ 1998;317:1185-1190.
ABSTRACT | FULL TEXT  

Should inhaled anticholinergics be added to beta 2 agonists for treating acute childhood and adolescent asthma? A systematic review
Plotnick and Ducharme
BMJ 1998;317:971-977.
ABSTRACT | FULL TEXT  

Guides for Reading and Interpreting Systematic Reviews: III. How Did the Authors Synthesize the Data and Make Their Conclusions?
Moher et al.
Arch Pediatr Adolesc Med 1998;152:915-920.
FULL TEXT  

Guides for Reading and Interpreting Systematic Reviews: II. How Did the Authors Find the Studies and Assess Their Quality?
Jadad et al.
Arch Pediatr Adolesc Med 1998;152:812-817.
FULL TEXT  

Guides for Reading and Interpreting Systematic Reviews: I. Getting Started
Klassen et al.
Arch Pediatr Adolesc Med 1998;152:700-704.
FULL TEXT  

Quantitive systematic review of topically applied non-steroidal anti-inflammatory drugs
Moore et al.
BMJ 1998;316:333-338.
ABSTRACT | FULL TEXT  

Relation Between the Time to Achieve the Lower Limit of the APTT Therapeutic Range and Recurrent Venous Thromboembolism During Heparin Treatment for Deep Vein Thrombosis
Hull et al.
Arch Intern Med 1997;157:2562-2568.
ABSTRACT  

The Importance of Initial Heparin Treatment on Long-term Clinical Outcomes of Antithrombotic Therapy: The Emerging Theme of Delayed Recurrence
Hull et al.
Arch Intern Med 1997;157:2317-2321.
ABSTRACT  

Methodological Studies of Systematic Reviews: Is There Publication Bias?
Laupacis
Arch Intern Med 1997;157:357-357.
ABSTRACT  

Large Trials vs Meta-analysis of Smaller Trials: How Do Their Results Compare?
Cappelleri et al.
JAMA 1996;276:1332-1338.
ABSTRACT