Making Sense of Test-Based Accountability in Education 2002

by Hamilton, Laura; Stecher, Brian M.; Klein, Stephen P.

• ISBN13: 9780833031617
• ISBN10: 0833031619
• Format: Paperback
• Copyright: 2002-11-18
• Publisher: RAND Corporation

Summary

Examining the state of the art in achievement testing and how testing and accountabiltiy can be used most effectively to promote positive educational outcomes.

Table of Contents

Preface	iii
Figures	ix
Tables	xi
Summary	xiii
Acknowledgments	xxi
Introduction	1	(12)
Defining the Terms	2	(2)
Tests Are Used Widely and for Several Purposes	4	(1)
New Federal Legislation Makes Testing Mandatory	5	(2)
The Rationale for Test-Based Accountability Systems Is Compelling	7	(2)
Test-Based Accountability Commands Substantial Resources	9	(1)
About This Book	10	(3)
Tests and Their Use in Test-Based Accountability Systems	13	(38)
A Brief History of Recent Trends in Large-Scale Assessment	14	(6)
Using Assessments to Monitor Aggregate Academic Achievement	16	(1)
Evolving from Minimum-Competency Testing to Measurement-Driven Instruction	17	(1)
The Education Reform Movement and Increased Use of Standardized Tests	17	(1)
The Second Wave of Reform: Testing Higher-Order Skills	18	(2)
How Large-Scale Tests Are Used Today	20	(9)
Content and Performance Standards	21	(1)
Norm-Referenced Score Reporting	22	(2)
Criterion-Referenced Score Reporting	24	(1)
Comparing the Two Types of Score Reporting	25	(2)
Disadvantages to Reporting Performance Solely as Norm or Criterion Referenced	27	(2)
Aligning Tests and Standards	29	(10)
Types of Assessments	31	(3)
Reporting the Results of Large-Scale Assessments	34	(5)
Setting Targets for Performance	39	(4)
Measuring Status	40	(2)
Measuring Change	42	(1)
Rewards and Sanctions	43	(5)
School-Level Incentives	44	(1)
Student-Level Incentives	45	(3)
Summary	48	(3)
Technical Criteria for Evaluating Tests	51	(28)
First, Some Background	51	(1)
Score Reliability	52	(8)
Factors That Affect Score Reliability	53	(2)
Quantifying Score Reliability	55	(3)
Score Reliability and Decisionmaking	58	(2)
Validity	60	(8)
Forms of Validity Evidence	61	(5)
Validity of Scores and Gains Under High-Stakes Conditions	66	(2)
Fairness	68	(6)
Bias-Free Items and Tests	68	(2)
Comparable Opportunities to Demonstrate Knowledge	70	(4)
Trade-Offs: Testing Time, Costs, and Other Factors	74	(2)
Conclusion	76	(3)
Consequences of Large-Scale, High-Stakes Testing on School and Classroom Practice	79	(22)
The Importance of the Effects of Testing	79	(4)
The Effects of Testing and Test Validity	80	(1)
Broadened Use of High-Stakes Testing to Promote Changes in School Practice	81	(2)
Gathering Evidence About the Effects of High-Stakes Testing	83	(2)
The Positive and Negative Effects of High-Stakes Testing	85	(14)
Teacher Response to High-Stakes Testing	87	(3)
Positive Classroom Effects	90	(1)
Neutral and Negative Classroom Effects	91	(5)
School-Level Effects	96	(2)
Effects on Equity	98	(1)
Summary	99	(2)
Accountability as Seen Through a Political Lens	101	(20)
Standardized Testing Facilitates Political Oversight of Education	103	(2)
Which Groups Are Interested in Testing and Assessment?	105	(3)
What Factors Influence Testing Policy?	108	(5)
Public Opinion	108	(3)
The Need to Show Results Quickly	111	(2)
Testing as a Route to Achieving Other Policy Goals	113	(1)
The Influence of Politics on How Tests Are Used	114	(3)
Reconciling Policymaking Imperatives with Good Testing Standards	117	(4)
Recommendations for Policymakers	117	(2)
Recommendations for Testing and Measurement Professionals	119	(2)
Improving Test-Based Accountability	121	(24)
First, Some General Advice	125	(1)
Improving Technical Quality	126	(7)
Conduct Ongoing Evaluation of Technical Quality	126	(2)
Incorporate Multiple Measures and Multiple Formats	128	(1)
Incorporate Findings from Cognitive Psychology	129	(1)
Collect and Maintain Rich Student-Level Data	130	(1)
Adopt Stable Statistical Indicators	131	(2)
Designing Systems That Have Positive Effects on Practice	133	(6)
Conduct Ongoing Research and Evaluation of Effects on Practice	133	(1)
Align Teacher Professional Development and Curriculum with Standards and Assessments	134	(1)
Clarify the Desired Balance Between Accountability Purposes and Instructional Purposes	135	(1)
Create Incentives That Promote Desired Goals	136	(2)
Consider Each Policy's Potential Effects on Equity	138	(1)
Political Considerations	139	(3)
Facilitate Integration of Political and Professional Perspectives	139	(1)
Incorporate Parents' Perspectives	140	(1)
Explore Costs and Benefits of Alternative Approaches	141	(1)
Conclusion	142	(3)
Resources	145	(4)
References	149	(20)
Contributors	169