Program Evaluation An Introduction

by Royse, David; Thyer, Bruce A.; Padgett, Deborah K.; Logan, T.K.

• ISBN13: 9780534508272
• ISBN10: 0534508278
• Edition: 4th
• Format: Paperback
• Copyright: 2005-03-10
• Publisher: Brooks Cole

Summary

Praised by instructors and students alike, PROGRAM EVALUATION: AN INTRODUCTION helps your students evaluate services and programs that they will encounter in their professional practice. In the process of learning evaluation techniques and skills, students will become proficient at critically analyzing evaluation studies conducted by others. The authors present and simplify all the essentials needed for a critical appreciation of evaluation issues and methodology. The text's student-friendly writing style and clear presentation of concepts, as well as its hands-on and applied focus, will guide students on how to gather evidence and demonstrate that their interventions and programs are effective in improving clients' lives.

Table of Contents

Preface	xi
CHAPTER 1 Introduction	1	(32)
The Importance of Program Evaluation	1	(4)
What Is a Program?	5	(1)
Characteristics of "Good" Programs	5	(6)
Program Evaluation Defined	11	(1)
Reasons Why Programs Are Evaluated	12	(1)
Motivations for Program Evaluation	13	(4)
Overcoming the Subjective Perspective	17	(4)
Philosophical Assumptions of Program Evaluation	21	(5)
More on Positivism	26	(2)
Chapter Recap	28	(5)
CHAPTER 2 Ethical Issues in Program Evaluation	33	(20)
Historical Background: Why IRBs Are Necessary	33	(1)
Ethical Guidelines	34	(6)
How Ethical Guidelines Get Operationalized	40	(3)
Research with Special Populations	43	(1)
Potential Problems in Conducting Ethical Research	44	(3)
The Practitioner's Ethical Responsibilities	47	(3)
Chapter Recap	50	(3)
CHAPTER 3 Needs Assessment	53	(34)
What Is Needs Assessment?	53	(1)
The Need for Needs Assessment	54	(1)
Definitions of Need	55	(2)
Planning a Needs Assessment	57	(5)
Selecting a Needs Assessment Approach	62	(10)
Convergent Analysis and Multimethod Approaches	72	(3)
Thinking Creatively about Needs Assessment	75	(5)
Community Readiness	80	(7)
CHAPTER 4 Qualitative Methods in Evaluation	87	(29)
Introduction	87	(1)
What Is "Qualitative Evaluation"?	88	(1)
When Is Qualitative Evaluation Useful?	89	(2)
Qualitative Methods and Single System Evaluation	91	(1)
Variants in Qualitative Evaluation	91	(3)
Methods of Qualitative Evaluation	94	(8)
Managing and Organizing Qualitative Data	102	(1)
Data Analysis	102	(3)
Quality Control	105	(1)
Writing the Report	106	(2)
An Important Final Step: Dissemination and Advocacy	108	(1)
Examples of Qualitative Evaluations	108	(8)
CHAPTER 5 Formative and Process Evaluation	116	(45)
Formative Evaluation	116	(2)
Conducting a Formative Evaluation	118	(4)
Other Uses of Formative Evaluation	122	(2)
Process Evaluation	124	(1)
Program Description	125	(5)
Program Monitoring	130	(1)
Becoming a Program Monitor	131	(2)
Mission Statements, Goals, and Objectives	133	(5)
Writing Program Objectives	138	(2)
Hierarchy of Goals, Objectives, and Activities	140	(1)
What Should Be Monitored?	141	(6)
Quality Assurance	147	(3)
Total Quality Management	150	(3)
Chapter Recap	153	(8)
CHAPTER 6 Single System Research Designs	161	(33)
What Are Single System Research Designs?	161	(2)
Selecting Outcome Measures	163	(2)
Assessing Measures over Time	165	(1)
Needs Assessments	166	(1)
Notation and General Principles	167	(2)
Formative Evaluations	169	(4)
Quality Assurance Studies	173	(2)
Summative Evaluation Designs	175	(2)
Experimental Designs	177	(7)
External Validity	184	(1)
Inferential Statistics	185	(2)
How to Prepare Graphs	187	(1)
Ethics of Single System Research Designs	188	(2)
Chapter Recap	190	(4)
CHAPTER 7 Goal Attainment Scaling	194	(13)
Introductory Scenario	194	(1)
What Is Goal Attainment Scaling?	195	(3)
Illustration of Goal Attainment Scaling	198	(1)
Goal Attainment Scaling and Program Evaluation	199	(2)
Variations of Goal Attainment Scaling	201	(1)
Problems with Goal Attainment Scaling	202	(5)
CHAPTER 8 Client Satisfaction	207	(24)
The Importance of Monitoring Consumer Satisfaction	207	(2)
The Problem with Client Satisfaction Studies	209	(3)
A Sampling of Recent Client Satisfaction Studies	212	(1)
Annotations about Client Satisfaction	212	(3)
Explanations for High Ratings	215	(1)
Recommendations for Client Satisfaction Studies	216	(6)
Determining Sample Size	222	(9)
CHAPTER 9 Group Research Designs	231	(43)
What Are Group Research Designs?	231	(1)
Starting an Outcome Evaluation	232	(2)
Outcome Evaluation Designs	234	(2)
General Principles of Group Research Designs	236	(1)
Pre-Experimental Research Designs	237	(11)
Quasi-Experimental Research Designs	248	(6)
Threats to the Internal Validity	254	(3)
Protection against Alternative Explanations	257	(1)
Experimental Designs	258	(7)
Efficacy and Effectiveness Studies	265	(1)
A Note on the Term Experiment	266	(1)
Chapter Recap	267	(7)
CHAPTER 10 Cost-Effectiveness and Cost Analysis Designs	274	(16)
Cost as an Evaluative Criterion	274	(2)
Example of a Cost-Effectiveness Evaluation	276	(1)
How to Do a Cost-Effectiveness Study	277	(2)
Whose Point of View?	279	(2)
Cost-Benefit Analysis	281	(4)
Chapter Recap	285	(5)
CHAPTER 11 Measurement Tools and Strategies	290	(33)
Importance of Measurement	290	(2)
Deciding What to Measure	292	(3)
Reliability	295	(7)
Validity	302	(4)
Locating Appropriate Instruments	306	(2)
Constructing "Good" Evaluation Instruments	308	(8)
Levels of Measurement	316	(7)
CHAPTER 12 Illustrations of Instruments	323	(16)
Child Abuse Potential Inventory	324	(3)
Clinical Anxiety Scale	327	(2)
CES-D Scale	329	(2)
The Hope Scale	331	(2)
Rosenberg Self-Esteem Scale	333	(2)
Adult-Adolescent Parenting Inventory	335	(4)
CHAPTER 13 Data Analysis	339	(32)
What Does It Mean to Analyze Data?	339	(1)
Data Analysis and the Computer	340	(1)
Univariate Analysis	341	(7)
Bivariate Analysis	348	(10)
Multivariate Analysis	358	(1)
Myths about Statistical Significance	359	(2)
Understanding Trends	361	(3)
Using Statistics in Reports	364	(1)
Type I and Type II Errors	365	(6)
CHAPTER 14 Pragmatic Issues	371	(23)
Treatment Fidelity	371	(2)
Fidelity Nightmares	373	(4)
Program Drift and the Transfer of Programs	377	(1)
Political Nature of Evaluation	378	(3)
"Threat" of Evaluation	381	(2)
Guidelines for Evaluation in Politically Charged Arenas	383	(3)
Culturally Sensitive Evaluation Practice	386	(8)
CHAPTER 15 Writing Evaluation Proposals, Reports, and Journal Articles	394	(23)
Components of the Evaluation Proposal and Report	395	(9)
Mistakes to Avoid in Planning and Writing Evaluation Reports	404	(5)
Checklist for Writing and Assessing Evaluation Reports	409	(1)
The Utilization of Evaluation Reports	410	(2)
Writing for Professional Publication	412	(5)
Index	417