Methods of Meta-Analysis : Correcting Error and Bias in Research Findings

by John E Hunter

• ISBN13: 9781412904797
• ISBN10: 141290479X
• Edition: 2nd
• Format: Paperback
• Copyright: 2004-04-07
• Publisher: SAGE Publications, Inc

Summary

Meta-analysis is arguably the most important methodological innovation in the social and behavioral sciences in the last 25 years. Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, Methods of Meta-Analysis, Second Edition is the most comprehensive text on meta-analysis available today.

Table of Contents

Preface to 2nd Edition

Preface to 1st Edition

Acknowledgements

Introduction to Meta-Analysis

Integration Research Findings Across Studies

General problem and an example

Problems with statistical significance tests

Is statistical power the solution?

Confidence intervals

Meta-analysis

Role of meta-analysis in the behavioral and social sciences

Role of meta-analysis in theory development

Increasing use of meta-analysis

Meta-analysis in industrial-organizational psychology

Wider impact of meta-analysis on psychology

Impact of meta-analysis outside psychology

Meta-analysis and social policy

Meta-analysis and theories of data

Conclusions

Study Artifacts and Their Impact on Study Outcomes

Study Artifacts

Sampling error, statistical power, and the interpretation of research literatures

When and how to cumulate

Undercorrection for artifacts in the corrected standard deviation

Coding study characteristics and capitalization on sampling error in moderator analysis

A look ahead in the book

Meta-Analysis of Correlations

Meta-Analysis of Correlations Corrected Individually for Artifacts

Introduction and Overview

Bare bones meta-analysis: Correcting for sampling error only

Artifacts other than sampling error

Multiple simultaneous artifacts

Meta-analysis of individually corrected correlations

A worked example: Indirect range restriction

Summary of meta-analysis correcting each correlation individually

Exercise 1: Bare bones meta-analysis

Exercise 2: Meta-analysis correcting each correlation individually

Meta-Analysis of Correlations Using Artifact Distributions

Full artifact distribution meta-analysis

Accuracy of corrections for artifacts

Mixed meta-analysis: Partial artifact information in individual studies

Summary of artifact distribution of meta-analysis of correlations

Exercise: Artifact distribution meta-analysis

Technical Questions in Meta-Analysis of Correlations

r versus : Which should be used?

r vs. regression slopes and intercepts in meta-analysis

Technical factors that cause overestimation of

Fixed and random models in meta-analysis

Credibility vs. confidence intervals in meta-analysis

Computing confidence intervals in meta-analysis

Range Restriction in meta-analysis: New technical analysis

Criticisms of meta-analysis procedures for correlations

Meta-Analysis of Experimental Effects and Other Dichotomous Comparisons

Treatment Effects: Experimental Artifacts and Their Impact

Quantification of the treatment effect: The d statistic and the point-biserial correlation

Sampling error in d values: Illustrations

Error of measurement in the dependent variable

Error of measurement in the treatment variable

Variation across studies in treatment strength

Range variation on the dependent variable

Dichotomization of the dependent variable

Imperfect construct validity in the dependent variable

Imperfect construct validity in the treatment variable

Bias in the effect size (d statistic)

Recording, computational, and transcriptional errors

Multiple artifacts and corrections

Meta-Analysis Methods for d Values

Effect size indices: d and r

An Alternative to d: Glass' d

Sampling error in the d statistic

Cumulation and correction of the variance for sampling error

Analysis of moderator variables

Correcting d values for measurement error in the dependent variable

Measurement error in the independent variable in experiments

Other artifacts and their effects

Correcting for multiple artifacts

Summary of meta-analysis of d values

Exercise: Meta-Analysis of d-Values

Technical Questions in Meta-Analysis of d Values

Alternative experimental designs

Within-subjects experimental designs

Meta-analysis and the within-subjects design

Statistical power in the two designs

Threats to internal and external validity

Bias in observed d values

Use of multiple regression in moderation analysis of d values

General Issues in Meta-Analysis

General Technical Issues in Meta-analysis

Fixed effects versus random effects models in meta-analysis

Second order sampling error: General principles

Detecting moderators not hypothesized a priori

Second order meta-analysis

Large N studies and meta-analysis

Second order sampling error: Technical treatment

The detection of moderator variables: Summary

Hierarchical analysis of moderator variables

Exercise: Second order meta-analysis

Cumulation of Findings Within Studies

Fully replicated designs

Conceptual replications

Conceptual replications and confirmatory factor analysis

Conceptual replications: A alternative approach

Analysis of subgroups

Summary

Methods of Integrating Findings Across Studies and Related Software

The traditional narrative procedure

The traditional voting method

Cumulation of p-values across studies

Statistically correct vote counting procedures

Meta-analysis of research studies

Unresolved problems in meta-analysis

Summary of methods of integrating studies

Computer programs for meta-analysis

Locating, Evaluating, and Coding Studies

Conducting a thorough literature search

What to do about studies with "methodological weaknesses"

Coding studies in meta-analysis

What to include in the meta-analysis report

Information needed in reports of primary studies

Appendix: Coding sheet for validity studies

Availability and Source Bias in Meta-Analysis

Some evidence on bias

Effects of methodological quality on mean effect sizes from different sources

Multiple hypotheses and other considerations in availability bias

Methods for detecting availability bias

Methods for correcting for availability bias

Summary of Psychometric Meta-Analysis

Meta-analysis methods and theories of data

What is the ultimate purpose of meta-analysis?

Appendix: Windows Based Meta-Analysis Software Package

References

Author Index

Subject Index

About the Authors