Program Evaluation: Methods and Case Studies

by Posavac; Emil J.

• ISBN13: 9780205804979
• ISBN10: 0205804977
• Edition: 8th
• Format: Hardcover
• Copyright: 2010-07-04
• Publisher: Routledge

Summary

For upper-level undergraduate/ graduate-level courses in Program Evaluation, Program Planning, Program Administration, and Public Administration.     Comprehensive yet accessible, this text provides a practical introduction to the skills, attitudes, and methods required to assess the worth and value of human services offered in public and private organizations in a wide range of fields. Students are introduced to the need for such activities, the methods for carrying out evaluations, and the essential steps in organizing findings into reports. The text focuses on the work of people who are closely associated with the service to be evaluated, and is designed to help program planners, developers, and evaluators to work with program staff members who might be threatened by program evaluation.

Author Biography

Emil J. Posavac (Ph.D., University of Illinois, Champaign) is Professor Emeritus of Psychology at Loyola University of Chicago where he served as director of the Applied Social Psychology Graduate Program and chairman of the Psychology Department. He has consulted with a number of public and private organizations. He has published over sixty papers and chapters, edited or co-edited six volumes on program evaluation and applied social psychology, and written numerous evaluation reports for health care and educational institutions. He has written a textbook (with Eugene B. Zechmeister) on statistical analysis based on emerging orientations that emphasize a more complete understanding and presentation of data. In 1990, he was awarded the Myrdal Award by the American Evaluation Association for his contributions to the advancement of program evaluation practice. Since retirement he has taught at the University of Rochester (Simon Graduate School of Business), Rochester Institute of Technology, and Trevecca Nazarene University. He teaches English as a Second Language at his church in Franklin, Tennessee.

Table of Contents

TABLE OF CONTENTS

 

1     Program Evaluation: An Overview    

Evaluation Tasks That Need to Be Done    

Common Types of Program Evaluations    

Activities Often Confused with Program Evaluation    

Different Types of Evaluations for Different Kinds of Programs    

Purpose of Program Evaluation    

 

2     Planning an Evaluation    

An Overview of the Core Purposes of Evaluation

Steps in Preparing to Conduct an Evaluation   

Dysfunctional Attitudes Making Program Evaluation Challenging

 

3     Developing and Using a Theory of the Program

Developing a Program Theory

Evaluation Questions Flow from Program Theory

Cautions in Choosing Evaluation Criteria

Identifying  Goals and Objectives

Some Practical Limitations in Selecting Evaluation Criteria    

 

4     Developing Measures of Implementation and Outcomes

Sources of Data for Evaluation    

Gathering Information Wisely

Case Study 1: Using Multiple Measures in an Evaluation of a Summer Community Program for Youth    

Types of Measures of Evaluation Criteria    

Preparing Special Surveys    

 

5     Ethics in Program Evaluation    

Standards for the Practice of Evaluation    

Ethical Issues Involved in the Treatment of People    

Role Conflicts Facing Evaluators    

Recognizing the Needs of Different Stakeholders    

The Validity of Evaluations    

Avoiding Possible Negative Side Effects of Evaluation Procedures   

Institutional Review Boards and Program Evaluation    

Ethical Problems Evaluators Report   

 

6     The Assessment of Need    

Definitions of Need    

Sources of Information for the Assessment of Need    

Inadequate Assessment of Need    

Using Needs Assessments in Program Planning    

 

7     Monitoring the Implementation and Operation of Programs    

Monitoring Programs as a Means of Evaluating Programs    

What to Summarize with Information Systems    

Program Records and Information Systems    

Avoiding Common Problems in Implementing an Information System    

 

8     Qualitative Evaluation Methods    

Evaluation Settings Best Served by Qualitative Evaluations   

Gathering Qualitative Information    

Case Study 2: Using Qualitative Methods in an Evaluation of a University Library    

Carrying Out Naturalistic Evaluations    

Coordinating Qualitative and Quantitative Methods    

Philosophical Assumptions    

 

9     Outcome Evaluations with One Group   

One-Group Evaluation Designs    

Uses of One-Group, Descriptive Designs    

Case Study 3: A Pretest-Posttest Design to Evaluate a Peer-Based Program to Prevent Skin Cancer    

Threats to Internal Validity    

Construct Validity in Pretest-Posttest Designs    

Overinterpreting the Results of One-Group Designs    

Usefulness of One-Group Designs as Initial Approaches to Program Evaluation    

 

10 Quasi-Experimental Approaches to Outcome Evaluation    

Increasing the Number of Times  Observations Are Made   

Observing Other Groups   

Case Study 4: Nonequivalent Control Groups Used to Evaluate an Employee Incentive Plan    

Regression-Discontinuity Design    

Observing Other Dependent Variables    

Combining Designs to Increase Internal Validity    

 

11 Using Experiments to Evaluate Programs    

Experiments in Program Evaluation    

Objections to Experimentation    

The Most Desirable Times to Conduct Experiments    

Case Study 5: Teaching Doctors Communication Skills: An Evaluation with Random Assignment and Pretests    

Successfully Implementing and Interpreting an Experimental Design    

 

12 Analyses of Costs and Outcomes    

Cost Analyses and Budgets    

Comparing Outcomes to Costs    

Some Details of Cost Analyses    

Case Study 6: The Value of Providing Smoking Cessation Clinics for Employees on Company Time    

Major Criticisms of Cost Analyses    

 

13 Evaluation Reports: Interpreting and Communicating Findings    

Developing a Communication Plan    

Personal Presentations of Findings    

Content of Formal Written Evaluation Reports    

Provide Progress Reports and Press Releases    

 

14 How to Encourage Utilization    

Obstacles to Effective Utilization    

Dealing with Mixed Findings    

Using Evaluations When an Innovative Program Seems No Better than Other Treatments    

Case Study 7: Evaluations of the Outcomes of Boot Camp Prisons: The Value of Finding No Differences Between Program and Comparison Groups    

Developing a Learning Culture    

The Evaluation Attitude    

 

References

Index