Program Evaluation An Introduction

by Royse, David; Thyer, Bruce A.; Padgett, Deborah K.; Logan, T.K.

• ISBN13: 9780830415366
• ISBN10: 083041536X
• Edition: 3rd
• Format: Paperback
• Copyright: 2000-08-02
• Publisher: Brooks Cole

Summary

Praised by instructors and students alike, PROGRAM EVALUATION: AN INTRODUCTION helps your students evaluate services and programs using scientifically credible research tools. In the process of learning these evaluation techniques and skills, they will also become proficient at critically analyzing evaluation studies conducted by others. The authors present and simplify all the essentials needed for a critical appreciation of evaluation issues and methodology. From this text, students will learn how to gather evidence and demonstrate that their interventions and programs are effective in improving clients' lives. This text is known for its student-friendly writing style and clear presentation of concepts, as well as its hands-on and applied focus.

Table of Contents

Preface	xi
Introduction	1	(29)
The Importance of Program Evaluation	1	(4)
What Is a Program?	5	(1)
Characteristics of ``Good'' Programs	5	(6)
Program Evaluation Defined	11	(1)
Reasons Why Programs Are Evaluated	12	(4)
Overcoming the Subjective Perspective	16	(5)
Philosophical Assumptions of Program Evaluation	21	(9)
Ethical Issues in Program Evaluation	30	(21)
Historical Background	30	(2)
Ethical Guidelines	32	(6)
How Ethical Guidelines Get Operationalized	38	(3)
Research with Special Populations	41	(1)
Potential Problems in Conducting Ethical Research	42	(3)
The Practitioner's Ethical Responsibilities	45	(1)
Ethical Dilemmas	45	(6)
Needs Assessment	51	(31)
What Is Needs Assessment?	51	(1)
The Need for Needs Assessment	52	(1)
Definitions of Need	53	(2)
Planning a Needs Assessment	55	(3)
Selecting a Needs Assessment Approach	58	(12)
Thinking Creatively About Needs Assessment	70	(4)
Community Readiness	74	(8)
Qualitative Methods in Evaluation	82	(26)
Introduction	82	(1)
What Is ``Qualitative Evaluation''?	83	(1)
When Is Qualitative Evaluation Useful?	83	(2)
Qualitative Methods and Single System Evaluation	85	(2)
Designing a Qualitative Evaluation	87	(2)
Action (Participatory) Evaluation	89	(1)
Gaining Access to the Site	90	(1)
Reciprocity, Payback, and Feedback	90	(1)
Sampling in Qualitative Evaluation	91	(1)
Data Collection	92	(4)
Analyzing Qualitative Data	96	(2)
Quality Control	98	(1)
Writing the Report	99	(2)
Examples of Qualitative Evaluations	101	(7)
Formative and Process Evaluation	108	(42)
Formative Evaluation	108	(2)
Conducting a Formative Evaluation	110	(4)
Other Uses of Formative Evaluation	114	(2)
Process Evaluation	116	(5)
Program Monitoring	121	(2)
Mission Statements, Goals, and Objectives	123	(4)
Writing Program Objectives	127	(3)
What Should Be Monitored?	130	(5)
Quality Assurance	135	(5)
Total Quality Management	140	(10)
Single System Research Designs	150	(31)
What Are Single System Research Designs?	150	(1)
Selecting Outcome Measures	151	(2)
Assessing Measures Over Time	153	(1)
Needs Assessments	154	(1)
Notation and General Principles	154	(3)
Formative Evaluations	157	(2)
Quality Assurance Studies	159	(2)
Summative Evaluation Designs	161	(4)
Experimental Designs	165	(5)
External Validity	170	(2)
Inferential Statistics	172	(3)
How to Prepare Graphs	175	(1)
Ethics of Single System Research Designs	176	(5)
Goal Attainment Scaling	181	(10)
Introductory Scenario	181	(1)
What Is Goal Attainment Scaling?	182	(1)
Illustration of Goal Attainment Scaling	183	(1)
Goal Attainment Scaling and Program Evaluation	184	(2)
Variations of Goal Attainment Scaling	186	(1)
Problems with Goal Attainment Scaling	187	(4)
Client Satisfaction	191	(22)
Arguments for Monitoring Consumer Satisfaction	191	(2)
The Problem with Client Satisfaction Studies	193	(2)
A Sampling of Recent Client Satisfaction Studies	195	(2)
Annotations About Client Satisfaction	197	(1)
Explanations for High Ratings	198	(1)
Recommendations for Client Satisfaction Studies	199	(4)
Determining Sample Size	203	(10)
Group Research Designs	213	(36)
What Are Group Research Designs?	213	(1)
Starting an Outcome Evaluation	214	(2)
Outcome Evaluation Designs	216	(2)
General Principles of Group Research Designs	218	(1)
Pre-Experimental Research Designs	219	(7)
Quasi-Experimental Research Designs	226	(1)
Quasi-Experimental Designs	227	(5)
Threats to the Internal Validity	232	(3)
Protection Against Alternative Explanations	235	(1)
Experimental Designs	236	(13)
Cost-Effectiveness and Cost Analysis Designs	249	(15)
Cost as an Evaluative Criterion	249	(2)
Example of a Cost-Effectiveness Evaluation	251	(1)
How to Do a Cost-Effectiveness Study	252	(3)
Whose Point of view?	255	(1)
Cost-Benefit Analysis	256	(8)
Measurement Tools and Strategies	264	(32)
Importance of Measurement	264	(2)
Deciding What to Measure	266	(3)
Reliability	269	(6)
Validity	275	(3)
Locating Appropriate Instruments	278	(4)
Constructing ``Good'' Evaluation Instruments	282	(7)
Levels of Measurement	289	(7)
Illustrations of Instruments	296	(28)
Child Abuse Potential Inventory	297	(2)
Clinical Anxiety Scale	299	(2)
CES-D Scale	301	(3)
The Hope Scale	304	(2)
The UCLA Loneliness Scale (Version 3)	306	(2)
Rosenberg Self-Esteem Scale	308	(2)
Adult-Adolescent Parenting Inventory	310	(2)
Brief Symptom Inventory™	312	(2)
Mental Adjustment to HIV Scale	314	(3)
Scales of Perceived Social Support	317	(3)
The Well-Being Scale	320	(4)
Pragmatic Issues	324	(20)
Treatment Fidelity	324	(2)
Fidelity Nightmares	326	(4)
Program Drift and the Transfer of Programs	330	(1)
Political Nature of Evaluation	331	(2)
``Threat'' of Evaluation	333	(1)
Guidelines for Evaluation in Politically Charged Arenas	334	(4)
Culturally Sensitive Evaluation Practice	338	(6)
Making Sense of Evaluation Data	344	(30)
What Does It Mean to Analyze Data?	344	(1)
Data Analysis and the Computer	345	(1)
Univariate Analysis	346	(7)
Bivariate Analysis	353	(8)
Multivariate Analysis of Data	361	(1)
Myths About Statistical Significance	362	(3)
Understanding Trends	365	(2)
Using Statistics in Reports	367	(3)
Type I and II Errors	370	(4)
Writing Evaluation Proposals, Reports, and Journal Articles	374	(23)
Components of the Evaluation Proposal and Report	375	(10)
Mistakes to Avoid in Writing Evaluation Reports	385	(5)
Checklist for Writing and Assessing Evaluation Reports	390	(1)
The Utilization of Evaluation Reports	391	(2)
Writing for Professional Publication	393	(4)
Index	397