Evaluation Essentials, First Edition From A to Z

by Alkin, Marvin C.; Vo, Anne T.

• ISBN13: 9781606238998
• ISBN10: 160623899X
• Edition: 1st
• Format: Hardcover
• Copyright: 2010-09-07
• Publisher: The Guilford Press

Summary

Written in a refreshing conversational style, this text thoroughly prepares students, program administrators, and new evaluators to conduct evaluations or to use them in their work. The book's question-driven focus and clear discussions about the importance of fostering evaluation use by building collaborative relationships with stakeholders set it apart from other available texts. In 26 concise sections, Marvin C. Alkin explores how to articulate answerable evaluation questions, collect and analyze data using both quantitative and qualitative methods, and deal with contingencies that might alter the traditional sequence of an evaluation. Student-friendly features include handy bulleted recaps of each section, "Thinking Ahead" and "Next Steps" pointers, cautionary notes, annotated suggestions for further reading, and an in-depth case study that provides the basis for end-of-chapter exercises.

Author Biography

Marvin C. Alkin is Emeritus Professor in the Social Research Methodology Division of the Graduate School of Education and Information Studies at the University of California, Los Angeles. He has been a member of the UCLA faculty since 1964 and has, at various times, served as Chair of the Education Department and Associate Dean of the school. Dr. Alkin was one of the founders of the Center for the Study of Evaluation and was its Director for 7 years. The Center, established in 1966 by the U.S. government to engage in research on appropriate methods for conducting evaluation of educational programs, continues to be an integral part of the UCLA Graduate School of Education and Information Studies. Dr. Alkin is a leading authority in the field of evaluation. He has published important research studies on the use of evaluation information in decision making and on comparative evaluation theory. His publications include five books and more than 150 journal articles, book chapters, monographs, and technical reports. Dr. Alkin is currently Associate Editor of Studies in Educational Evaluation and co-section Editor of the American Journal of Evaluation. He has been a consultant to numerous national governments and has directed program evaluations in 14 different countries.

Table of Contents

Overview Structure A. What Is Evaluation? Professional Program Evaluation Evaluation and Research Evaluation Definition A Confusion of Terms Evaluation Purposes B. Why Do Evaluations? Making Decisions Issues for Professional Evaluation • Time-Out: The RUPAS Case The Rural Parents\u2019 Support Program (RUPAS): A Community Well-Being Case Study,Nicole Eisenberg C. Who Does Evaluations? Evaluator Settings Multiple Orientations to Doing Evaluation My View D. Who Are the Stakeholders for an Evaluation? Stakeholders—Not Audience Who Are the Stakeholders? Focus on Primary Stakeholders Differences in Stakeholder Participation E. How Are Positive Stakeholder Relationships Maintained? Gaining RTC (Respect, Trust, Credibility) Concluding Note F. What Is the Organizational, Social, and Political Context? Organizational Context Social Context Political Context My Advice Thinking Ahead G. How Do You Describe the Program? Program Components Program Size and Organizational Location Program Differences What Should We Know about Programs Learning about the Program H. How Do You \u201cUnderstand\u201d the Program? Theory of Action Logic Models Why Is This Important? What Does a Logic Model Look Like? A Partial Logic Model Getting Started I. What Are the Questions/Issues to Be Addressed? Kinds of Evaluation Questions Getting Started on Defining Questions Some Next Steps J. Who Provides Data? Again, Be Clear on the Questions Focus of the Data Selecting Individuals K. What Are Instruments for Collecting Quantitative Data? Instruments for Attaining Quantitative Data Acquisition of Data Existing Data Finding Existing Instruments Developing New Instruments Questionnaire Construction Measuring Achievement Achievement Test Construction Observation Protocols L. What Are Instruments for Collecting Qualitative Data? Developing New Instruments Observations Interviews and Focus Groups Surveys and Questionnaires M. What Are the Logistics of Data Collection? Gaining Data Access Collecting Data Quality of Data Understanding the Organization\u2019s Viewpoints My Advice N. Are the Questions Evaluable (Able to Be Evaluated)? Stage of the Program Resources Nature of the Question Establishing Standards Technical Issues Ethical Issues Political Feasibility My Advice O. What Is the Evaluation Plan (Process Measures)? The Evaluation Design Process Measures Program Elements Program Mechanisms Question P. What Is the Evaluation Plan (Outcome Measures)? "An Exercise to Assist Us Toward Stronger Causal Models Descriptive Designs Mixed Methods Summary My Advice Q. What Is the Evaluation Plan (Procedures and Agreements)? Evaluation Activities: Past, Present, and Upcoming The Written Evaluation Plan The Contract My Advice R. How Are Quantitative Data Analyzed? Types of Data A First Acquaintance with the Data Measures of Central Tendency Measures of Variability Getting Further Acquainted with the Data Descriptive and Inferential Statistics Are the Results Significant? Appropriate Statistical Techniques My Warning S. How Are Qualitative Data Analyzed? Refining the Data Testing the Validity of the Analysis T. How Do Analyzed Data Answer Questions? Difficulties in Valuing Valuing in a Formative Context \u201cValuing\u201d Redefined A Final Note U. How Are Evaluation Results Reported? Communication Reporting The Final Written Report Nature and Quality of Writing V. What Is the Evaluator\u2019s Role in Helping Evaluations to Be Used? A Word about \u201cUse,\u201d What Is Use? What Can You Do? Guard against Misuse W. How Are Evaluations Managed? Acquiring the Evaluation Contract/Agreement Budget Operational Management X. What Are the Evaluation Standards and Codes of Behavior? Judging an Evaluation The Program Evaluation Standards American Evaluation Association Guiding Principles Y. How Are Costs Analyzed? Cost-Effectiveness Analysis Cost–Benefit Analysis Cost–Utility Analysis And Now to Costs How to Determine Cost Z. How Can You Embark on a Program to Learn More about Evaluation? Getting Feedback on Evaluation Taking Full Advantage of This Volume Gaining Evaluation Expertise Beyond This Volume • Appendix. An Evaluation Lesson, \u201cUnknown Student\u201d