Evaluation

by Weiss, Carol H.

• ISBN13: 9780133097252
• ISBN10: 0133097250
• Edition: 2nd
• Format: Paperback
• Copyright: 1997-12-08
• Publisher: Pearson

Summary

This timely and accessible guide on the conduct of evaluation studies explores how we apply research methods in evaluating social programs, illustrating its points with reference to a variety of fields, including education, social services, and criminal justice.Offers practical advice on understanding the reasons for the study, identifying key questions to be answered, and planning and implementing the overall design of the study, including measurement, qualitative methods of inquiry, data collection, analysis, reporting, and dissemination. Stresses that understanding the underlying theory of the program is essential to developing the most appropriate evaluation, and emphasizes the need to take ethical considerations into account all along the course of the study. Covers meta-analysis, cost-benefit analysis and includes a non-technical discussion of the logic of data analysis. Includes references to further sources on measurement, existing longitudinal data sets, statistics, and qualitative analysis.For professionals in psychology, sociology, public health, public policy/affairs, education, and criminal justice.

Table of Contents

Preface	ix
Chapter 1 Setting the Scene	1	(19)
Domain of Evaluation	3	(1)
Definition of Evaluation	4	(2)
What Is Evaluated? An Excursus on Terminology	6	(2)
Outcome and Process Evaluation	8	(2)
Contributions of Evaluation	10	(1)
History of Evaluation	10	(5)
Comparison between Evaluation and Other Research	15	(3)
Summary	18	(2)
Chapter 2 Purposes of Evaluation	20	(26)
Who Wants Evaluation?	20	(1)
Overt and Covert Purposes	21	(3)
Conditions Unfavorable for Evaluation	24	(1)
Evaluation for Decision Making	25	(1)
Evaluation as Organizational Learning	26	(2)
Intended Uses for Evaluation	28	(3)
Formative and Summative Evaluation	31	(1)
Formative-Summative and Process-Outcome: Different Constructs	32	(1)
Compatibility of Purposes	33	(1)
How Evaluations Are Commissioned	34	(3)
Inside versus Outside Evaluation	37	(2)
Level in the Structure	39	(2)
Whose Use Shall Be Served?	41	(3)
Summary	44	(2)
Chapter 3 Understanding the Program	46	(26)
Why Program Knowledge Pays	46	(2)
Characterizing the Program	48	(2)
Getting to Know the Program	50	(1)
What Is the Program Trying to Achieve?	51	(4)
How the Program Works: Surfacing the Program's Theories of Change	55	(2)
Program Theory and Implementation Theory	57	(1)
Organizing the Evaluation to Investigate Program Theories	58	(3)
Building Program Theory	61	(1)
Using Theories of Change as a Guide to Evaluation	62	(4)
Comparing Program Theory to Actual Developments	66	(1)
Advantages of Explicating Theories of Change	67	(2)
Critiques of Program Theory	69	(1)
Summary	70	(2)
Chapter 4 Planning the Evaluation	72	(25)
The Right Time to Evaluate	73	(1)
Types of Evaluation Questions	74	(2)
Additional Decisions in the Planning Period	76	(1)
How to Decide Which Questions to Pursue	77	(4)
Specifying the Questions	81	(1)
Quantitative or Qualitative Study	82	(5)
Design of the Evaluation	87	(2)
One Study or a Fleet of Studies?	89	(1)
Designs for Different Types of Programs	90	(1)
Some Practical Plans	91	(1)
Ethical Issues	92	(20)
Summary	112
Chapter 5 Roles for the Evaluator	97	(17)
Types of Participative Roles	99	(1)
Reasons for Alternative Evaluator Roles	100	(3)
Implementing a Participatory Evaluation	103	(2)
Balance of Pros and Cons	105	(3)
Additional Limitations to Participatory Approaches	108	(1)
Ethical Issues	109	(3)
Summary	112	(2)
Chapter 6 Developing Measures	114	(38)
Measurement	115	(2)
Program Outcomes	117	(6)
Choices among Measures	123	(3)
Unanticipated Consequences	126	(1)
Interim Markers of Progress	127	(3)
Program Processes	130	(2)
Program Inputs, Resources, and Environment	132	(4)
Multiple Measures	136	(1)
How Many Measures Are Enough?	137	(1)
Measurement of Variables	137	(2)
Developing New Measures	139	(5)
Desirable Characteristics of Measures in Evaluation	144	(6)
Summary	150	(2)
Chapter 7 Collecting Data	152	(28)
Sources of Data	152	(11)
Sampling	163	(3)
Interviewing	166	(2)
Coding Responses	168	(2)
Existing Statistical Data	170	(5)
Ethical Issues in Collecting Data	175	(3)
Summary	178	(2)
Chapter 8 Design of the Evaluation	180	(35)
Designing the Process Evaluation	181	(1)
Designing the Outcome Evaluation	182	(2)
Important Concepts	184	(4)
Designs	188	(1)
Informal Designs	188	(3)
Formal Designs	191	(22)
Summary	213	(2)
Chapter 9 The Randomized Experiment	215	(20)
Random Assignment	215	(6)
Planned Variations	221	(2)
Procedures for Random Assignment	223	(3)
Coping with Difficulties That Arise	226	(2)
Conditions That Make Randomized Experiments Problematic	228	(1)
Criticisms of the Randomized Experiment	229	(4)
Summary	233	(2)
Chapter 10 Extensions of Good Design	235	(17)
Replication	235	(1)
Meta-Analysis	236	(8)
Cost-Benefit and Cost-Effectiveness Analysis	244	(6)
Summary	250	(2)
Chapter 11 Qualitative Methods	252	(19)
Design of Qualitative Evaluations	253	(3)
Collecting Data	256	(7)
Fieldwork and Analysis	263	(2)
Qualitative Evaluation and Program Theory	265	(1)
Ethical Issues	266	(1)
Combining Qualitative and Quantitative Approaches	267	(2)
Summary	269	(2)
Chapter 12 Analyzing and Interpreting the Data	271	(23)
Introduction	271	(1)
Analytic Tasks in Evaluation	272	(11)
General Strategies of Analysis	283	(5)
An Example of Program Theory as a Guide to Analysis	288	(2)
Books on Analytic Methods	290	(2)
Ethical Issues	292	(1)
Summary	292	(2)
Chapter 13 Writing the Report and Disseminating Results	294	(26)
Reporting	295	(10)
Dissemination	305	(5)
Utilization of Evaluation Results	310	(10)
Chapter 14 Evaluating with Integrity	320	(7)
Know the Program	321	(1)
Maintain High Technical Quality and Relevance	321	(2)
Use Balance and Judgment	323	(1)
Consider Use from the Start	324	(1)
Behave Ethically throughout the Study	325	(1)
Survive and Reap Rewards	325	(2)
Glossary	327	(13)
References	340	(25)
Index	365