Program Evaluation and Performance Measurement : An Introduction to Practice

by James C. McDavid

• ISBN13: 9781412906685
• ISBN10: 1412906687
• Format: Paperback
• Copyright: 2005-08-19
• Publisher: SAGE Publications, Inc

Summary

Program Evaluation and Performance Measurement: An Introduction to Practice offers a conceptual, as well as practical, introduction to program evaluation and performance measurement for public and non-profit organizations. This introductory text discusses topics in a detailed fashion, making it a useful guide for practitioners who are constructing and implementing performance measurement systems, as well as for students. Authors James C. McDavid and Laura R. L. Hawthorn guide readers through conducting quantitative and qualitative program evaluations and needs assessments, as well as constructing and implementing performance measurement systems. Key Features: Highlights the importance of sound professional judgment throughout the book and the final chapter offers ways that evaluation professionals can develop their professional judgment Integrates performance measurement into evaluation practice to demonstrate how these skills are important foundations for performance measurement and essential for building program evaluation expertise Discusses the realities of organizational politics and balances the importance of good evaluation methods with the resource and organizational constraints that almost always affect the choices evaluators make as they do their work Provides reader friendly checklists to underscore key points Program Evaluation and Performance Measurement is designed for advanced undergraduate and graduate courses in program evaluation, performance measurement, and performance management. The book does not assume a thorough understanding of research methods and design, making it an ideal text for students in Public Administration and Management, Social Work, the health professions, and other disciplines where research methods are not a central focus. The book is complemented by an Instructor's CD-ROM to facilitate effective classroom use.

Table of Contents

Dedication	ii
Acknowledgments	xv
Preface	xvii
Key Concepts and Issues in Program Evaluation and Performance Measurement	1	(38)
Introduction	3	(7)
A Typical Program Evaluation: Assessing the Neighbourhood Integrated Service Team Program	10	(5)
What Is a Program?	15	(1)
Key Concepts in Program Evaluation	15	(2)
Key Evaluation Questions	17	(4)
Formative and Summative Program Evaluations	21	(1)
Ex Ante and Ex Post Program Evaluations	22	(1)
Analyzing Cause and Effect Linkages in Program Evaluations	23	(1)
The Process of Conducting a Program Evaluation	24	(10)
Summary	34	(1)
Discussion Questions	35	(1)
References	36	(3)
Understanding and Applying Program Logic Models	39	(40)
Introduction	41	(3)
A Basic Logic Modeling Approach	44	(3)
Logic Models That Categorize and Specify Intended Causal Linkages	47	(10)
Flow Charts	57	(1)
Constructing Program Logics in Program Evaluations	58	(4)
Program Technologies	62	(3)
Program Objectives, Program Environments, and Organizational Objectives	65	(2)
Strengths and Limitations of Program Logics	67	(2)
Summary	69	(1)
Discussion Questions	70	(2)
Appendix 1: Applying What You Have Learned in Chapter 2: Developing a Logic Model for the Compass Program	72	(4)
Answer Key	76	(1)
References	77	(2)
Research Designs for Program Evaluations	79	(50)
Introduction	81	(2)
What Is Research Design?	83	(6)
Why Pay Attention to Experimental Designs?	89	(2)
Using Experimental Designs to Evaluate Programs: The Elmira Nurse Home Visitation Program	91	(4)
Establishing Validity in Research Designs	95	(1)
Defining and Working with the Four Kinds of Validity	96	(18)
Testing the Causal Linkages in Program Logic Models	114	(4)
Research Designs and Performance Measurement	118	(2)
Summary	120	(2)
Discussion Questions	122	(3)
References	125	(4)
Measurement in Program Evaluation	129	(36)
Introduction	131	(2)
Measurement Procedures	133	(1)
Illustrating Measurement Terminology	133	(5)
Measurement Validity	138	(4)
Levels of Measurement	142	(3)
Units of Analysis	145	(1)
Sources of Data in Program Evaluations and Performance Measurement Systems	146	(10)
Using Surveys to Estimate the Incremental Effects of Programs	156	(1)
Survey Designs and Research Designs	157	(1)
Validity of Measures and the Validity of Causes and Effects	158	(2)
Summary	160	(1)
Discussion Questions	161	(1)
References	161	(4)
Applying Qualitative Evaluation Methods	165	(36)
Introduction	167	(1)
Comparing and Contrasting Different Approaches to Qualitative Evaluation	168	(4)
Qualitative Evaluation Methods: Some Basics	172	(7)
Structuring Qualitative Program Evaluations	179	(12)
The Credibility and Generalizability of Qualitative Findings	191	(3)
Connecting Qualitative Evaluation Methods to Performance Measurement	194	(2)
The Power of Case Studies	196	(1)
Summary	197	(1)
Discussion Questions	198	(1)
References	198	(3)
Assessing the Need for Programs	201	(40)
Introduction	203	(3)
What Are Needs?	206	(2)
Benchmarking Needs: Criteria for Establishing the Existence and Magnitude of Needs	208	(4)
Steps in Conducting Needs Assessments	212	(18)
The St. Columba Collaboration Project: Needs Assessment in a Newark Neighborhood	230	(5)
Summary	235	(1)
Discussion Questions	236	(1)
Appendix: Designing a Needs Assessment for a Small Nonprofit Organization	236	(2)
References	238	(3)
Concepts and Issues in Economic Evaluation	241	(40)
Introduction	243	(4)
Three Types of Economic Evaluation	247	(2)
Economic Evaluation in the Performance Management Cycle	249	(1)
Historical Developments in Economic Evaluation	250	(2)
Cost-Effectiveness Analysis	252	(10)
Cost-Utility Analysis	262	(1)
Cost-Benefit Analysis	263	(3)
Cost-Effectiveness Analysis Example: A Study of Falls Prevention Among the Elderly	266	(6)
Strengths and Limitations of Economic Evaluation	272	(4)
Summary	276	(1)
Discussion Questions	277	(1)
References	278	(3)
Performance Measurement as an Approach to Evaluation	281	(24)
Introduction	282	(1)
Growth of Performance Measurement	283	(5)
Metaphors That Support and Sustain Performance Measurement	288	(4)
Comparing Program Evaluation and Performance Measurement	292	(8)
Summary	300	(1)
Discussion Questions	301	(1)
References	302	(3)
Design and Implementation of Performance Measurement Systems	305	(32)
Introduction	307	(1)
Key Steps in Designing and Implementing a Performance Measurement System	307	(23)
Summary	330	(1)
Discussion Questions	331	(3)
References	334	(3)
Using and Sustaining Performance Measurement Systems	337	(36)
Introduction	339	(1)
Comparing Ideal and Real Organizations: The Role of Organizational Politics in Designing and Implementing Performance Measurement Systems	340	(3)
Intended Uses of Performance Measurement Systems	343	(6)
Actual Uses of Performance Measurement	349	(5)
Problems and Issues in Implementing and Sustaining Performance Measurement Systems	354	(13)
Summary	367	(1)
Discussion Questions	368	(1)
References	368	(5)
Program Evaluation and Program Management: Joining Theory and Practice	373	(28)
Introduction	375	(1)
Can Management and Evaluation Be Joined?	376	(2)
Prospects for Building Cultures That Support Evaluation	378	(2)
Learning Organizations as Self-Evaluating Organizations	380	(3)
Can Program Managers Evaluate Their Own Programs?	383	(2)
How Should Evaluators Relate to Managers? Striving for Objectivity in Program Evaluations	385	(1)
Can Program Evaluators Be Objective?	386	(3)
Criteria for Best Practices in Program Evaluation: Assuring Stakeholders That Evaluations Are High Quality	389	(2)
Ethics and Evaluation Practice	391	(3)
Summary	394	(1)
Discussion Questions	394	(1)
Appendix 1. A Case Study in Ethical Decision Making	395	(2)
References	397	(4)
The Nature and Practice of Professional Judgment in Program Evaluation	401	(34)
Introduction	403	(1)
The Nature of the Evaluation Enterprise	403	(9)
Professionalism and Evaluation Practice	412	(3)
Understanding Professional Judgment	415	(7)
Improving Professional Judgment in Evaluation Practice	422	(5)
Evaluation as a Craft: Implications for Professionalizing Evaluation	427	(1)
Teamwork and Professional Judgment	428	(1)
Summary	429	(1)
Discussion Questions	430	(1)
References	431	(4)
Glossary	435	(18)
Index	453	(24)
About the Authors	477

Excerpts

"Program Evaluation and Performance Measurement: An Introduction to Practice offers a conceptual, as well as practical, introduction to program evaluation and performance measurement for public and non-profit organizations. This introductory text discusses topics in a detailed fashion, making it a useful guide for practitioners who are constructing and implementing performance measurement systems, as well as for students. Throughout the text, program evaluation and performance measurement are portrayed as complementary ways of supporting evidence-based management and decision-making. Authors James C. McDavid and Laura R. L. Hawthorn guide readers through conducting quantitative and qualitative program evaluations and needs assessments, as well as constructing and implementing performance measurement systems." "Program Evaluation and Performance Measurement is designed for advanced undergraduate and graduate courses in program evaluation, performance measurement, and performance management. The book does not assume a thorough understanding of research methods and design, making it an ideal text for students in Public Administration and Management. Social Work, the health professions, and other disciplines in which research methods are not always a central focus. It is also an excellent resource for practicing public and non-profit managers who are not research specialists."--BOOK JACKET.