Brain Korea 21 Phase II: A New Evaluation Mode

by Seong, Somi; Popper, Steven W.; Goldman, Charles A.; Evans, David K.; Grammich, Clifford A.

• ISBN13: 9780833043214
• ISBN10: 0833043218
• Format: Paperback
• Copyright: 2008-04-08
• Publisher: RAND Corporation

Summary

The Brain Korea 21 Program (BK21), which seeks to make Korean research universities globally competitive and to produce more high-quality researchers in Korea, provides funding to graduate students and professors who belong to research groups at top universities. The authors develop quantitative and qualitative models to evaluate how well BK21 is fulfilling its goals and make suggestions for further stimulating Korean university research.

Table of Contents

Preface	p. iii
Figures	p. xi
Tables	p. xiii
Summary	p. xv
Acknowledgments	p. xxv
Abbreviations	p. xxvii
Introduction	p. 1
Background	p. 1
BK21 Program Structure	p. 1
Phase I and Phase II Comparison	p. 3
Significance of the Program	p. 4
Objective of the Study	p. 5
Approach	p. 6
Outline of the Monograph	p. 7
Implications of Phase I Program Evaluations for Phase II Evaluations	p. 9
Previous Performance Evaluations of BK21 Phase I	p. 9
Annual and Interim Evaluations	p. 9
Program-Level Evaluations	p. 11
Findings from Previous Evaluations	p. 14
Changes Resulting from BK21 Phase I	p. 14
Suggestions for Improving the Evaluation System	p. 16
Limitations of Previous Evaluations	p. 17
Implications for Designing the Phase II Evaluation Model	p. 19
A Logic Model for Evaluating BK21	p. 21
Literature Review and Focus Group Interviews	p. 21
Developing the Logic Model	p. 23
Identifying the Goals and Missions	p. 26
Goals	p. 27
Missions	p. 29
Performance Milestones	p. 31
Goals and Missions Identified for the Logic Model	p. 33
Program Inputs	p. 34
BK21 Funding	p. 34
Strategy	p. 37
Unit of Support	p. 40
Recipient Selection Criteria	p. 41
Performance Evaluation Criteria	p. 42
Other Operational Mechanisms	p. 42
Summary	p. 43
Incentives, Activities, Outputs, and Outcomes of the Program	p. 44
Incentives	p. 44
Activities	p. 47
Outputs	p. 48
Outcomes	p. 54
Other Observations	p. 55
Summary	p. 56
Quantitative Model to Assess BK21 Program Outcomes	p. 59
Introduction	p. 59
Summary of Phase I Statistics	p. 61
Potential Approaches	p. 64
Unadjusted Comparisons of BK21 Winners and Losers	p. 67
Ordinary Least Squares, Matching, and Propensity Score	p. 69
Fixed-Effects and Difference-in-Differences Models	p. 71
Regression Discontinuity	p. 74
Other Strategies	p. 76
Quantitative Model	p. 77
Heterogeneous Funding	p. 80
The Power to Identify Impacts	p. 81
Data for Use in Validating the Model	p. 81
ISI Data	p. 81
Ministry of Education Data	p. 83
Validation of the Model	p. 83
Pre-Treatment Trends	p. 84
Another Possible Comparison Group: Nonapplicants	p. 88
Validation	p. 90
Variations on the Specification	p. 96
Quality	p. 97
Social Science and Humanities	p. 100
Other Aspects of the Logic Model	p. 100
Simultaneous Equations Methods	p. 102
International Competitiveness	p. 104
Simulation: Ability to Precisely Estimate Effects at Phase II Funding Levels	p. 105
Simulation: Outside Funding	p. 107
Implications for Data Collection	p. 109
Summary	p. 110
Metrics and Measurement for Program Evaluation	p. 111
General Issues of Measurement	p. 112
Reliability	p. 113
Validity	p. 114
Direct and Indirect Measures	p. 114
Cost Versus Benefit	p. 115
Strategy for Framing Evaluation Measures	p. 116
Matching Measures to Goals	p. 117
Selecting Among Candidate Measures	p. 118
Adding to Korea's Research Manpower Pool	p. 132
Highly Educated Workforce Employment	p. 134
High-Quality Globally Competitive Research	p. 135
Measuring Quality of Research Output	p. 135
Consistent Measures of Quality Within Fields	p. 138
Measures of Quality Across All Fields	p. 144
Relative Research Quality Measures	p. 146
Changes to University/Department Infrastructure	p. 146
Fostering a Competitive Environment Within Korea	p. 147
Other Measures of Prestigious and Competitive Graduate Research	p. 150
Measuring Enhanced University-Industry Collaboration	p. 154
Measuring Enhanced Regional University Standing and Industry Linkage	p. 156
Industrial Research Quantity, Quality, and National Labor Productivity	p. 159
Database Design to Support BK21 Program Evaluation and Measurement	p. 161
Priority Among Proposed Measures	p. 161
Database Design and Content	p. 166
Format of Database Design Entries	p. 173
Future Development of BK21 and University Research in Korea	p. 177
A Good Model for an Allocation System	p. 178
Applying the Model to Korea	p. 179
Research Project Funding	p. 179
Portable Student Fellowships	p. 181
Facilities and Infrastructure Funding	p. 182
Promotion of Specific Regions or Fields	p. 183
BK21 Program Management	p. 183
Requirements for University and Department Governance	p. 183
Changing the Korean System	p. 184
Appropriate Scale and Number of Research Universities	p. 185
Analogy to the U.S. System	p. 185
Applying the Proposed Measurement Tools	p. 189
Summary	p. 191
Conclusions	p. 195
Directions for a Phase II Evaluation System	p. 195
Phase II Program Goals and Characteristics Identified by the Logic Model	p. 196
Quantitative Model and Data Requirements	p. 196
Metrics, Measures, and Database to Assess Program Effects and to Monitor Progress	p. 197
Assessing the Quality of Program Outputs and Outcomes	p. 199
Quality of Research	p. 199
Quality of the Highly Educated Workforce	p. 200
Concentration Strategy and Competition Rule	p. 201
Suggestions for Long-Term Changes in the Korean Scientific Research and Education System	p. 202
Balancing Competition and Stability in Funding	p. 202
Portable Student Fellowships	p. 203
Infrastructure Funding	p. 203
Improving University and Department Governance	p. 204
Preconditions for the Proposed Changes	p. 204
Enhancing Program Evaluation and Management Capabilities	p. 205
Appendix
Performance Evaluation Metrics Used in Previous Evaluations of Phase I	p. 207
Interview Guides for Representatives of Participants, Policymakers, and Users Related to the University Research and Education System	p. 209
Selection Criteria for Applied Science Research Groups: Phase II	p. 217
Table with Lagged Outcomes	p. 219
Research Data Series	p. 221
Education Data Series	p. 229
Industry Data Series	p. 237
General Data Series	p. 243
Bibliography	p. 245
Table of Contents provided by Ingram. All Rights Reserved.