The Kernel Method of Test Equating

by Von Davier, Alina A.; Holland, Paul W.; Thayer, Dorothy T.

• ISBN13: 9780387019857
• ISBN10: 0387019855
• Format: Hardcover
• Copyright: 2003-10-01
• Publisher: Springer Verlag

Summary

Kernel Equating (KE) is a powerful, modern and unified approach to test equating. It is based on a flexible family of equipercentile-like equating functions and contains the linear equating function as a special case. Any equipercentile equating method has five steps or parts. They are: 1) pre-smoothing; 2) estimation of the score-probabilities on the target population; 3) continuization; 4) computing and diagnosing the equating function; 5) computing the standard error of equating and related accuracy measures. KE brings these steps together in an organized whole rather than treating them as disparate problems. KE exploits pre-smoothing by fitting log-linear models to score data, and incorporates it into step 5) above. KE provides new tools for diagnosing a given equating function, and for comparing two or more equating functions in order to choose between them. In this book, KE is applied to the four major equating designs and to both Chain Equating and Post-Stratification Equating for the Non-Equivalent groups with Anchor Test Design. This book will be an important reference for several groups: (a) Statisticians and others interested in the theory behind equating methods and the use of model-based statistical methods for data smoothing in applied work; (b) Practitioners who need to equate testsa??including those with these responsibilities in testing companies, state testing agencies and school districts; and (c) Instructors in psychometric and measurement programs. The authors assume some familiarity with linear and equipercentile test equating, and with matrix algebra. From the reviews: "The book is nicely laid out, is extremely well written, and is an excellent text for a semester course or a short coursea? The book is highly recommended." Short Book Reviews of the International Statistical Institute, December 2004 "This book is well-written and the presentation is clear, rigorous, and concise...A rich set of applications is used to illustrate the methods...This book is a gem! I highly recommend it to any statistician or psychometrician who has even a passing interest in test equating." Pscyhometrika, March 2006 "This is a great book, and it is the first to focus on the kernel method of test equating." Applied Psychological Measurement, September 2005

Author Biography

Alina A. von Davier is Associate Research Scientist in the Center for Statistical Theory and Practice, at Educational Testing Service.

Table of Contents

Preface	vii
List of Notation	xv
Introduction and Notation	1	(16)
Introduction	1	(4)
The Notation Used in This Book	5	(3)
The Linear Equating Function	8	(1)
The Equipercentile Equating Function	9	(2)
The Relationship Between LinY(x) and EquiY(x)	11	(2)
Data Collection Designs	13	(1)
Sample Estimates	14	(1)
A Summary of the New Material in This Book	15	(2)
Part I---The Kernel Method of Test Equating: Theory	17	(80)
Data Collection Designs	19	(26)
The Equivalent-Groups Design (EG)	21	(1)
The Single-Group Design (SG)	22	(5)
The Counterbalanced Design (CB)	27	(5)
Non-Equivalent groups with Anchor Test Design (NEAT)	32	(11)
Chain Equating (CE)	35	(4)
Post-Stratification Equating (PSE)	39	(2)
Internal Anchor Tests and Structural Zeros	41	(2)
The EG Design with an Anchor Test	43	(1)
Random versus Spiraled Samples	43	(1)
Summary	44	(1)
Kernel Equating: Overview, Pre-smoothing, and Estimation of r and s	45	(10)
The Five Steps of Kernel Equating: Overview	45	(2)
Pre-smoothing Using Log-Linear Models	47	(5)
Estimating a Univariate Score Distribution	49	(3)
Estimation of the Score Probabilities	52	(3)
Kernel Equating: Continuization and Equating	55	(12)
Continuization	55	(9)
Gaussian Kernel Smoothing	56	(5)
Choice of the Bandwidth	61	(3)
Equating	64	(3)
Kernel Equating: The SEE and the SEED	67	(20)
Introduction	68	(1)
The δ-Method Divides the Problem in Three	69	(4)
The SEE and the SEED for Kernel Equating	73	(8)
Computing JeY for Kernel Equating	73	(3)
The SEE for Specific Equating Designs	76	(3)
The SEED	79	(2)
The SEE and SEED for Chain Equating	81	(6)
Kernel Equating versus Other Equating Methods	87	(10)
KE versus Linear Equating	88	(2)
KE versus the Percentile Rank Method	90	(3)
The Percentile Rank Method	90	(1)
Some Facts About PRM	91	(1)
Distributional Characteristics of PRM	92	(1)
The Issue of the Finite Range	92	(1)
Viewing PRM from the KE Perspective	93	(1)
Advantages of KE over PRM	94	(3)
Part II---The Kernel Method of Test Equating: Applications	97	(100)
The Equivalent-Groups Design	99	(14)
Pre-smoothing	101	(2)
Estimation of the Score Probabilities	103	(1)
Continuization	104	(2)
Equating	106	(2)
Standard Error of Equating	108	(2)
Deciding Between eY(x) and LinY(x)	110	(3)
The Single-Group Design	113	(18)
Pre-smoothing	116	(5)
Estimation of the Score Probabilities	121	(2)
Continuization	123	(1)
Equating	124	(2)
Standard Error of Equating	126	(2)
Deciding Between eY(x) and LinY(x)	128	(3)
The Counterbalanced Design	131	(24)
Pre-smoothing	133	(6)
Estimation of the Score Probabilities	139	(2)
Continuization	141	(2)
Equating	143	(2)
Standard Error of Equating	145	(3)
Deciding Between eY1(x) and eY 1/2(x)	148	(1)
Diagnosis of the Equating Process	149	(1)
Deciding Between eY 1/2 and LinY 1/2 (x)	150	(2)
Appendix: The Data Used in This Chapter	152	(3)
The NEAT Design: Chain Equating	155	(24)
Pre-smoothing	159	(8)
Estimation of the Score Probabilities	167	(1)
Continuization	168	(2)
Equating	170	(4)
Standard Error of Equating	174	(1)
Deciding Between eY (CE)(x) and LinY(x)	175	(4)
The NEAT Design: Post-Stratification Equating	179	(18)
Estimation of the Score Probabilities	181	(2)
Continuization	183	(1)
Equating	184	(3)
Standard Error of Equating	187	(1)
The Choice of the Target Population	188	(2)
Deciding Between e 1/2Y (x) and Lin 1/2Y (x)	190	(2)
Comparing the KE Functions for PSE and CE	192	(2)
CE versus PSE: Which One to Choose?	194	(3)
Appendix	197	(19)
A The δ-Method	197	(2)
B Bivariate Smoothing	199	(4)
B.1 Assessing the Fit of the Log-Linear Models	201	(1)
B.1.1 Covariance Matrix of the Parameters	201	(2)
C Other Univariate Moments	203	(2)
D Review of the Use of Matrices in This Book	205	(11)
Bibliography	216	(7)
Author Index	223	(2)
Subject Index	225