9780387952826

Generalizability Theory

by ; ; ;
  • ISBN13:

    9780387952826

  • ISBN10:

    0387952829

  • Format: Hardcover
  • Copyright: 2001-10-01
  • Publisher: Springer Verlag
  • Purchase Benefits
  • Free Shipping Icon Free Shipping On Orders Over $35!
    Your order must be $35 or more to qualify for free economy shipping. Bulk sales, PO's, Marketplace items, eBooks and apparel do not qualify for this offer.
  • eCampus.com Logo Get Rewarded for Ordering Your Textbooks! Enroll Now
List Price: $169.99 Save up to $119.95
  • eBook
    $108.42
    Add to Cart

    DURATION
    PRICE

Supplemental Materials

What is included with this book?

Summary

Generalizability theory offers an extensive conceptual framework and a powerful set of statistical procedures for characterizing and quantifying the fallibility of measurements. It liberalizes classical test theory, in part through the application of analysis of variance procedures that focus on variance components. As such, generalizability theory is perhaps the most broadly defined measurement model currently in existence. It is applicable to virtually any scientific field that attends to measurements and their errors, and it enables a multifacteted perspective on measurement error and its components. This book provides the most comprehensive and up-to-date treatment of generalizability theory. In addition, it provides a synthesis of those parts of the statistical literature that are directly applicable to generalizability theory. The principal intended audience is measurement practitioners and graduate students in the behavioral and social sciences, although a few examples and references are provided from other fields. Readers will benefit from some familiarity with classical test theory and analysis of variance, but the treatment of most topics does not presume specific background. Robert L. Brennan is E.F. Lindquist Professor of Educational Measurement at the University of Iowa. He is an acknowledged expert in generalizability theory, has authored numerous publications on the theory, and has taught many courses and workshops on generalizability. The author has been Vice-President of the American Educational Research Association and President of the National Council on Measurement in Education (NCME). He has received NCME Awards for Outstanding Technical Contributions to Educational Measurement and Career Contributions to Educational Measurement.

Author Biography

Robert L. Brennan is E.F. Lindquist Professor of Educational Measurement and Director of Iowa Testing Programs at the University of Iowa.

Table of Contents

Preface vii
Principal Notational Conventions xix
Introduction
1(20)
Framework of Generalizability Theory
4(14)
Universe of Admissible Observations and G Studies
5(3)
Infinite Universe of Generalization and D Studies
8(5)
Different Designs and/or Universes of Generalization
13(4)
Other Issues and Applications
17(1)
Overview of Book
18(1)
Exercises
19(2)
Single-Facet Designs
21(32)
G Study p x i Design
22(2)
G Study Variance Components for p x i Design
24(5)
Estimating Variance Components
25(3)
Synthetic Data Example
28(1)
D Studies for the p x I Design
29(10)
Error Variances
31(3)
Coefficients
34(1)
Synthetic Data Example
35(2)
Real-Data Examples
37(2)
Nested Designs
39(6)
Nesting in Both the G and D Studies
40(4)
Nesting in the D Study, Only
44(1)
Summary and Other Issues
45(5)
Other Indices and Coefficients
47(1)
Total Score Metric
48(2)
Exercises
50(3)
Multifacet Universes of Admissible Observations and G Study Designs
53(42)
Two-Facet Universes and Designs
54(6)
Venn Diagrams
54(3)
Illustrative Designs and Universes
57(3)
Linear Models, Score Effects, and Mean Scores
60(7)
Notational System for Main and Interaction Effects
61(2)
Linear Models
63(3)
Expressing a Score Effect in Terms of Mean Scores
66(1)
Typical ANOVA Computations
67(7)
Observed Mean Scores and Score Effects
67(2)
Sums of Squares and Mean Squares
69(1)
Synthetic Data Examples
70(4)
Random Effects Variance Components
74(11)
Defining and Interpreting Variance Components
74(2)
Expected Mean Squares
76(3)
Estimating Variance Components Using the EMS Procedure
79(1)
Estimating Variance Components Directly from Mean Squares
80(3)
Synthetic Data Examples
83(1)
Negative Estimates of Variance Components
84(1)
Variance Components for Other Models
85(7)
Model Restrictions and Definitions of Variance Components
86(3)
Expected Mean Squares
89(1)
Obtaining σ2(α\M) from σ2(α)
89(1)
Example: APL Program
90(2)
Exercises
92(3)
Multifacet Universes of Generalization and D Study Designs
95(46)
Random Models and Infinite Universes of Generalization
96(14)
Universe Score and Its Variance
97(3)
D Study Variance Components
100(1)
Error Variances
100(4)
Coefficients and Signal-Noise Ratios
104(3)
Venn Diagrams
107(1)
Rules and Equations for Any Object of Measurement
108(1)
D Study Design Structures Different from the G Study
109(1)
Random Model Examples
110(10)
p x I x O Design with Synthetic Data Set No. 3
110(3)
p x (I:O) Design with Synthetic Data Set No. 3
113(2)
p x (R:T) Design with Synthetic Data Set No. 4
115(2)
Performance Assessments
117(3)
Simplified Procedures for Mixed Models
120(5)
Rules
121(1)
Venn Diagrams
122(3)
Mixed Model Examples
125(10)
p x I x O Design with Items Fixed
125(1)
p x (R:T) Design with Tasks Fixed
125(2)
Perspectives on Traditional Reliability Coefficients
127(3)
Generalizability of Class Means
130(2)
A Perspective on Validity
132(3)
Summary and Other Issues
135(1)
Exercises
136(5)
Advanced Topics in Univariate Generalizability Theory
141(38)
General Procedures for D Studies
141(12)
D Study Variance Components
142(2)
Universe Score Variance and Error Variances
144(1)
Examples
145(4)
Hidden Facets
149(4)
Stratified Objects of Measurement
153(4)
Relationships Among Variance Components
155(1)
Comments
156(1)
Conventional Wisdom About Group Means
157(2)
Two Random Facets
157(2)
One Random Facet
159(1)
Conditional Standard Errors of Measurement
159(6)
Single-Facet Designs
160(4)
Multifacet Random Designs
164(1)
Other Issues
165(10)
Covariances as Estimators of Variance Components
166(2)
Estimators of Universe Scores
168(3)
Random Sampling Assumptions
171(3)
Generalizability and Other Theories
174(1)
Exercises
175(4)
Variability of Statistics in Generalizability Theory
179(36)
Standard Errors of Estimated Variance Components
180(10)
Normal Procedure
181(1)
Jackknife Procedure
182(3)
Bootstrap Procedure
185(5)
Confidence Intervals for Estimated Variance Components
190(6)
Normal Procedure
190(1)
Satterthwaite Procedure
190(1)
Ting et al. Procedure
191(4)
Jackknife Procedure
195(1)
Bootstrap Procedure
196(1)
Variability of D Study Statistics
196(5)
Absolute Error Variance
197(1)
Feldt Confidence Interval for Eρ2
198(2)
Arteaga et al. Confidence Interval for φ
200(1)
Some Simulation Studies
201(7)
G Study Variance Components
201(4)
D Study Statistics
205(3)
Discussion and Other Issues
208(3)
Exercises
211(4)
Unbalanced Random Effects Designs
215(34)
G Study Issues
216(11)
Analogous-ANOVA Procedure
217(3)
Unbalanced i:p Design
220(2)
Unbalanced p x (i:h) Design
222(3)
Missing Data in the p x i Design
225(2)
D Study Issues
227(13)
Unbalanced I:p Design
228(3)
Unbalanced p x (I:H) Design
231(2)
Unbalanced (P:c) x I Design
233(2)
Missing Data in the p x I Design
235(2)
Metric Matters
237(3)
Other Topics
240(7)
Estimation Procedures
241(4)
Computer Programs
245(2)
Exercises
247(2)
Unbalanced Random Effects Designs-Examples
249(18)
ACT Science Reasoning
249(2)
District Means for ITED Vocabulary
251(6)
Clinical Clerkship Performance
257(5)
Testlets
262(3)
Exercises
265(2)
Multivariate G Studies
267(34)
Introduction
268(5)
G Study Designs
273(11)
Single-Facet Designs
275(3)
Two-Facet Crossed Designs
278(3)
Two-Facet Nested Designs
281(3)
Defining Covariance Components
284(2)
Estimating Covariance Components for Balanced Designs
286(9)
An Illustrative Derivation
287(2)
General Equations
289(4)
Standard Errors of Estimated Covariance Components
293(2)
Discussion and Other Topics
295(3)
Interpreting Covariance Components
295(2)
Computer Programs
297(1)
Exercises
298(3)
Multivariate D Studies
301(46)
Universes of Generalization and D Studies
301(9)
Variance-Covariance Matrices for D Study Effects
302(1)
Variance-Covariance Matrices for Universe Scores and Errors of Measurement
303(2)
Composites and A Priori Weights
305(1)
Composites and Effective Weights
306(1)
Composites and Estimation Weights
307(1)
Synthetic Data Example
308(2)
Other Topics
310(18)
Standard Errors of Estimated Covariance Components
310(2)
Optimality Issues
312(2)
Conditional Standard Errors of Measurement for Composites
314(3)
Profiles and Overlapping Confidence Intervals
317(3)
Expected Within-Person Profile Variability
320(4)
Hidden Facets
324(2)
Collapsed Fixed Facets in Multivariate Analyses
326(2)
Computer Programs
328(1)
Examples
328(15)
ACT Assessment Mathematics
328(6)
Painting Assessment
334(5)
Listening and Writing Assessment
339(4)
Exercises
343(4)
Multivariate Unbalanced Designs
347(44)
Estimating G Study Covariance Components
347(20)
Observed Covariance
349(1)
Analogous TP Terms
349(4)
CP Terms
353(6)
Compound Means
359(1)
Variance of a Sum
360(3)
Missing Data
363(3)
Choosing a Procedure
366(1)
Examples of G Study and D Study Issues
367(21)
ITBS Maps and Diagrams Test
367(6)
ITED Literary Materials Test
373(7)
District Mean Difference Scores
380(8)
Discussion and Other Topics
388(1)
Exercises
388(3)
Multivariate Regressed Scores
391(40)
Multiple Linear Regression
392(3)
Estimating Profiles Through Regression
395(20)
Two Independent Variables
395(6)
Synthetic Data Example
401(3)
Variances and Covariances of Regressed Scores
404(3)
Standard Errors of Estimate and Tolerance Intervals
407(2)
Different Sample Sizes and/or Designs
409(3)
Expected Within-Person Profile Variability
412(3)
Predicted Composites
415(11)
Difference Scores
417(2)
Synthetic Data Example
419(2)
Different Sample Sizes and/or Designs
421(1)
Relationships with Estimated Profiles
422(2)
Other Issues
424(2)
Comments
426(1)
Exercises
427(4)
Appendices 431(76)
A. Degrees of Freedom and Sums of Squares for Selected Balanced Designs
431(4)
B. Expected Mean Squares and Estimators of Random Effects Variance Components for Selected Balanced Designs
435(4)
C. Matrix Procedures for Estimating Variance Components and Their Variability
439(6)
D. Table for Simplified Use of Satterthwaite's Procedure
445(4)
E. Formulas for Selected Unbalanced Random Effects Designs
449(4)
F. Mini-Manual for GENOVA
453(18)
G. urGENOVA
471(2)
H. mGENOVA
473(2)
I. Answers to Selected Exercises
475(32)
References 507(14)
Author Index 521(4)
Subject Index 525

Rewards Program

Reviews for Generalizability Theory (9780387952826)