Research Methods : Learning to Become a Critical Research Consumer

by Martella, Ronald C.; Nelson, Ronald; Marchand-Martella, Nancy E.

• ISBN13: 9780205271252
• ISBN10: 0205271251
• Format: Paperback
• Copyright: 1999-01-01
• Publisher: Allyn & Bacon

Summary

Becoming a critical research consumer is one of the most important skills a research professional must acquire. To become a consumer of critical research, one must develop certain critical thinking skills about research, such as the ability to read about and understand an investigation; the knowledge to determine the value of the obtained data; the intuitive skills to determine the adequacy of the research design; the experience to determine if there are other explanations for the obtained results; and finally, the good judgment to assess the adequacy of conclusions and claims.This text is designed to produce just such a knowledgeable and informed critical research consumer. Each chapter builds on the information in preceding ones to focus on the importance of research methodology and interpretation rather than mere statistical calculations and analyses. The thorough and extensive coverage of all research methods combined with a focus on critical thinking makes this text truly unique.Educators, researchers, scientists, research and data analysts, and psychologists.

Table of Contents

PREFACE	xvii
PART ONE UNDERSTANDING RESEARCH	1	(32)
CHAPTER 1 THINKING CRITICALLY ABOUT RESEARCH	3	(30)
Objectives	1	(1)
Overview	1	(4)
What Is Science and the Scientific Method?	5	(1)
What Are the Purposes of Science?	6	(2)
Description	6	(1)
Prediction	6	(1)
Improvement	7	(1)
Explanation	8	(1)
What Is a Scientific Theory?	8	(3)
Good Versus Bad Scientific Theories	10	(1)
Testability of a Theory	10	(1)
What Are the Types of Scientific Logic?	11	(3)
Inductive Logic	11	(1)
Deductive Logic	12	(1)
Combination of Logic Forms	13	(1)
In What Ways Do We Gain Information?	14	(3)
Tenacity	14	(1)
Intuition	15	(1)
Authority	15	(1)
Empiricism	16	(1)
Rationalism	16	(1)
What Are Constraint Levels in Educational and Psychological Research?	17	(5)
Experimental Research	18	(2)
Causal-Comparative Research	20	(1)
Correlational Research	20	(1)
Case Study Research	20	(1)
Naturalistic or Descriptive Research	21	(1)
What Are the Differences between Basic and Applied Research?	22	(1)
Basic Research	22	(1)
Applied Research	22	(1)
What Is Replication Research?	23	(6)
Reasons for Replications	23	(4)
Types of Replications	27	(2)
Replication as the "Big Picture"	29	(1)
Summary	29	(1)
Discussion Questions	29	(4)
PART TWO CRITICAL ISSUES IN RESEARCH	33	(58)
CHAPTER 2 FUNDAMENTAL ISSUES FOR INTERPRETING RESEARCH	35	(28)
Objectives	35	(1)
Overview	35	(1)
What Is Variability?	36	(2)
Sources of Variability	36	(2)
What Is Internal Validity?	38	(8)
Maturation	38	(2)
Selection	40	(1)
Selection by Maturation Interaction	41	(1)
Statistical Regression	41	(1)
Mortality	42	(1)
Instrumentation	43	(1)
Testing	43	(1)
History	44	(1)
Resentful Demoralization of the Control Group	44	(1)
Diffusion of Treatment	45	(1)
Compensatory Rivalry by the Control Group	45	(1)
Compensatory Equalization of Treatments	46	(1)
What Is External Validity?	46	(10)
Population Validity	47	(2)
Ecological Validity	49	(7)
What Are Statistical and Social Validities?	56	(2)
Statistical Validity	56	(1)
Social Validity	57	(1)
Summary	58	(1)
Discussion Questions	59	(1)
Interpretation Exercises	59	(4)
CHAPTER 3 RELIABILITY, VALIDITY, AND INTEROBSERVER AGREEMENT	63	(28)
Objectives	63	(1)
Overview	63	(2)
What Are Reliability and Validity Issues in Quantitative Research?	65	(1)
How Do Researchers Assess the Reliability of Measurement Devices?	66	(7)
Types of Reliability Coefficients	66	(5)
Standard Error of Measurement	71	(2)
How Do Researchers Determine the Validity of Measurement Devices?	73	(4)
Types of Validity	74	(3)
What Are Reliability and Validity Issues in Qualitative Research?	77	(2)
Reliability Issues	78	(1)
Validity Issues	78	(1)
Illustrative Example of Reliability and Validity Procedures	78	(1)
What Is Interobserver Agreement?	79	(1)
What Are the Methods of Establishing Interobserver Agreement?	80	(5)
Measures of Permanent Products	81	(1)
Event Recording	82	(1)
Latency and Duration Recording	82	(1)
Interval Recording	83	(2)
What Are the Factors that Influence Interobserver Agreement?	85	(1)
Reactivity	85	(1)
Observer Drift	85	(1)
Complexity of the Measurement System	86	(1)
Observer Expectations	86	(1)
What Are the Factors to Consider when Assessing Measurement Devices in the Context of a Study?	86	(1)
Description of the Measurement Device	86	(1)
Adjustments to the Measurement Device	86	(1)
Appropriateness of the Measurement Device	87	(1)
Cooperation of Participants	87	(1)
Summary	87	(1)
Discussion Questions	88	(1)
Practice Exercises	89	(2)
PART THREE QUANTITATIVE RESEARCH METHODS	91	(162)
CHAPTER 4 BASIC STATISTICAL CONCEPTS AND SAMPLING PROCEDURES	93	(37)
Objectives	93	(1)
Overview	93	(1)
What Are the Scales of Measurement?	94	(1)
Nominal Scale	94	(1)
Ordinal Scale	94	(1)
Interval Scale	94	(1)
Ratio Scale	95	(1)
What Is a Univariate Frequency Distribution?	95	(1)
How Can We Describe Data Sets?	95	(3)
Measures of Central Tendency	95	(1)
Measures of Variation	96	(1)
Shape of the Distribution	97	(1)
What Role Do Hypotheses Play in Research?	98	(4)
Inferential Statistics	98	(1)
Role of Hypotheses	99	(1)
Null and Alternative Hypotheses	99	(1)
Directional and Nondirectional Alternative Hypotheses	100	(2)
What Are Parametric and Nonparametric Tests of Statistical Significance?	102	(1)
Parametric Tests	102	(1)
Nonparametric Tests	102	(1)
What Is Statistical Significance?	102	(3)
Errors Made in Interpretations of the Significance Level	104	(1)
What Are Type I and Type II Errors and Power?	105	(6)
Type I Errors	105	(1)
Type II Errors	106	(1)
Relative Seriousness of Type I or Type II Errors	107	(1)
Power	108	(1)
Five Methods of Increasing Power	108	(3)
What Are the Types of Statistical Significance Testing Methods?	111	(8)
Parametric Tests of Statistical Significance	111	(5)
Nonparametric Tests of Statistical Significance	116	(3)
What Are the Different Sampling Methods?	119	(6)
Probability Sampling	120	(4)
Nonprobability Sampling	124	(1)
What Are the Sampling Decisions Researchers Make?	125	(1)
What Is Sampling Error?	126	(1)
Summary	127	(1)
Discussion Questions	128	(2)
CHAPTER 5 EXPERIMENTAL DESIGNS	130	(44)
Objectives	130	(1)
Overview	130	(2)
What Are True Experimental Designs?	132	(1)
What Are the Most Common True Experimental Designs?	133	(8)
Pretest-Posttest Control-Group Design	133	(3)
Posttest-Only Control-Group Design	136	(2)
Solomon Four-Group Design	138	(3)
What Are Factorial Experimental Designs?	141	(3)
Analysis of Data	141	(3)
What Are Quasi-Experimental Designs?	144	(1)
What Are the Common Quasi-Experimental Designs?	144	(9)
Static-Group Comparison	144	(2)
Nonequivalent Control-Group Designs	146	(3)
Counterbalanced Designs	149	(2)
Time-Series Designs	151	(2)
What Are Preexperimental Designs?	153	(1)
What Are the Types of Preexperimental Designs?	153	(3)
One-Shot Case Study	153	(2)
One-Group Pretest-Posttest Design	155	(1)
When Should Researchers Use Each Experimental Research Design?	156	(1)
Discussion Questions	157	(1)
Sample Article: "The Effects of Learning Strategy Instruction on the Completion of Job Applications by Students with Learning Disabilities"	158	(16)
J. Ron Nelson
Deborah J. Smith
John M. Dodd
CHAPTER 6 CAUSAL-COMPARATIVE RESEARCH	174	(26)
Objectives	174	(1)
Overview	174	(1)
What Is the Causal-Comparative Research Method?	175	(1)
What Should Researchers Consider when Designing a Causal-Comparative Study?	176	(6)
Development of Hypotheses	176	(1)
Selection of Groups	176	(1)
Analysis of Data	177	(3)
Research Examples	180	(2)
When Should Researchers Use the Causal-Comparative Research Design?	182	(1)
Discussion Questions	182	(1)
Sample Article: "The Current and Future Outcomes of Interpersonal Social Interactions: The Views of Students with Behavioral Disorders"	183	(17)
J. Ron Nelson
Marcy Drummond
Ron Martella
Nancy Marchand-Martella
CHAPTER 7 CORRELATIONAL RESEARCH	200	(53)
Objectives	200	(1)
Overview	200	(1)
What Is the Correlational Research Method?	201	(1)
What Are the Issues in Designing a Correlational Study?	201	(29)
Types of Variables in Correlational Research	201	(1)
Critical Issues in Correlational Research	202	(1)
Analysis of Data	203	(24)
Research Examples	227	(3)
When Should Researchers Use the Correlational Research Method?	230	(1)
Discussion Questions	231	(1)
Sample Article: "Peer Victimization: Cause or Consequence of School Maladjustment?"	232	(21)
Becky J. Kochenderfer
Gary W. Ladd
PART FOUR QUALITATIVE RESEARCH METHODS	253	(72)
CHAPTER 8 QUALITATIVE RESEARCH: METHODS	255	(26)
Objectives	255	(1)
Overview	255	(2)
What Are the Characteristics of Qualitative Research?	257	(1)
What Are the Differences between Qualitative and Quantitative Research?	258	(7)
Source of Data	258	(5)
Nature of Data	263	(1)
Process of Change	264	(1)
Type of Reasoning	264	(1)
Focus of Research	265	(1)
What Are Qualitative Research Procedures?	265	(5)
Phases	265	(2)
Sampling	267	(3)
What Is Understanding in Qualitative Research?	270	(3)
Descriptive Validity	270	(1)
Interpretive Validity	271	(1)
Theoretical Validity	271	(1)
Generalizability	272	(1)
Evaluative Validity	273	(1)
What Are the Evaluative Criteria for Judging the Reliability and Validity of Qualitative Research?	273	(4)
Criterion 1: Completeness of Information	274	(1)
Criterion 2: Adequacy of Interpretation	274	(1)
Criterion 3: Determination of Inconsistencies in Data	274	(1)
Criterion 4: Adequacy of Metaphors, Pictures, or Diagrams	274	(1)
Criterion 5: Collaboration with Participants	274	(1)
Criterion 6: Multiple Methods to Gather Data	274	(2)
Criterion 7: Disqualification of Interpretations	276	(1)
What Are the Types of Triangulation Methods?	277	(2)
Data Sources Triangulation	277	(1)
Analyst Triangulation	277	(1)
Theory/Perspective Triangulation	278	(1)
Methods Triangulation	278	(1)
Summary	279	(1)
Discussion Questions	279	(2)
CHAPTER 9 QUALITATIVE RESEARCH: DATA COLLECTION AND DESIGNS	281	(44)
Objectives	281	(1)
Overview	281	(1)
What Are Field-Oriented Studies?	282	(18)
Case Study	282	(1)
Observation Studies	282	(3)
Participant Observation Study	285	(1)
Nonparticipant Observation Study	286	(1)
Natualistic Observation Study	287	(1)
Simulations	287	(1)
Interview Studies	288	(3)
Ethnography	291	(2)
Document Analysis	293	(1)
Analysis of Data	294	(4)
Research Examples	298	(2)
When Should Researchers Use Each Qualitative Research Design?	300	(1)
Discussion Questions	301	(1)
Sample Article: "Successful Mainstreaming in Elementary Science Classes: A Qualitative Study of Three Reputational Cases"	302	(23)
Thomas E. Scruggs
Margo A. Mastropieri
PART FIVE SINGLE-CASE RESEARCH METHODS	325	(122)
CHAPTER 10 WITHDRAWAL AND ASSOCIATED DESIGNS	327	(36)
Objectives	327	(1)
Overview	327	(3)
What Are Graphing Methods in Withdrawal Designs?	330	(1)
What Are Withdrawal and Associated Designs?	331	(16)
A-B Design	331	(2)
Withdrawal Design	333	(1)
A-B-A-B Design	334	(1)
B-A-B Design	334	(1)
A-B-C-B Design	335	(2)
Analysis of Data	337	(3)
Research Examples	340	(7)
When Should Researchers Use Each Withdrawal and Associated Design?	347	(3)
Discussion Questions	350	(1)
Sample Article: "Reducing Negative Comments through Self-Monitoring and Contingency Contracting"	351	(12)
Pamela S. Courson-Krause
Nancy Marchand-Martella
Ronald C. Martella
Brenda Schmitt
CHAPTER 11 MULTIPLE-BASELINE DESIGNS	363	(42)
Objectives	363	(1)
Overview	363	(1)
What Are Graphing Methods in Multiple-Baseline Designs?	364	(1)
What Are Multiple-Baseline Designs?	364	(21)
Multiple-Baseline Design across Behaviors	366	(1)
Multiple-Baseline Design across Participants	366	(1)
Multiple-Baseline Design across Settings	366	(1)
Multiple-Probe Design	367	(2)
Analysis of Data	369	(10)
Research Examples	379	(6)
When Should Researchers Use Each Multiple-Baseline Design?	385	(2)
Discussion Questions	387	(1)
Sample Article: "Generalized Effects of a Peer-Delivered First Aid Program for Students with Moderate Intellectual Disabilities"	388	(17)
Nancy E. Marchand-Martella
Ronald C. Martella
Martin Agran
Charles L. Salzberg
K. Richard Young
Daniel Morgan
CHAPTER 12 ADDITIONAL SINGLE-CASE DESIGNS	405	(42)
Objectives	405	(1)
Overview	405	(1)
What Is a Changing-Criterion Design?	405	(8)
Analysis of Data	407	(5)
Research Example	412	(1)
What Is a Multitreatment Design?	413	(5)
Analysis of Data	416	(2)
Research Example	418	(1)
What Is an Alternating Treatments Design?	418	(6)
Analysis of Data	422	(1)
Research Example	423	(1)
What Are Combination Designs?	424	(4)
Analysis of Data	428	(1)
Research Example	428	(1)
When Should Researchers Use Each of the Additional Single-Case Designs?	428	(2)
Discussion Questions	430	(1)
Sample Article: "Problem Solving to Prevent Work Injuries in Supported Employment"	431	(16)
Ronald C. Martella
Martin Agran
Nancy E. Marchand-Martella
PART SIX ADDITIONAL RESEARCH METHODS	447	(76)
CHAPTER 13 SURVEY, HISTORICAL, AND PROGRAM EVALUATION RESEARCH METHODS	449	(40)
SURVEY RESEARCH	449	(11)
Objectives	449	(1)
Overview	449	(1)
What Are the Purposes of Survey Research?	450	(1)
Description	450	(1)
Explanation	450	(1)
Exploration	451	(1)
What Are the Different Types of Surveys?	451	(1)
What Are the Factors in Choosing a Survey Method?	451	(1)
Sampling Procedures	451	(1)
Sampling Population	451	(1)
Question Format	452	(1)
Content of Questions	453	(1)
Response Rate	453	(1)
Time and Money	453	(1)
How Is Survey Research Designed?	453	(1)
Sampling	453	(3)
Developing the Survey Instrument	456	(2)
Analysis of Data	458	(1)
Research Example	459	(1)
When Should Researchers Use Survey Research?	459	(1)
HISTORICAL RESEARCH	460	(9)
Objectives	460	(1)
Overview	460	(1)
What Are the Characteristics and Purposes of Historical Research?	461	(1)
Characteristics	461	(1)
Purposes	461	(1)
How Is Historical Research Designed?	462	(1)
Research Question or Hypothesis	462	(1)
Data Collection	463	(1)
Analysis of Data	464	(4)
Research Example	468	(1)
When Should Researchers Use Historical Research?	469	(1)
PROGRAM EVALUATION	469	(8)
Objectives	469	(1)
Overview	469	(1)
What Are the Goal and Objectives of Program Evaluation?	470	(1)
Goal	470	(1)
Objectives	470	(1)
What Are the Types of Program Evaluations?	471	(1)
Needs Assessments	472	(1)
Process Evaluations	473	(1)
Outcome Evaluations	474	(1)
Efficiency Evaluations	474	(1)
How Is a Program Evaluation Conducted?	475	(1)
Establishing Boundaries	475	(1)
Selecting Evaluation Methods	475	(1)
Collecting and Analyzing Data	476	(1)
Reporting of Findings	476	(1)
Analysis of Data	476	(1)
Evaluation Example	476	(1)
When Should Program Evaluations Be Conducted?	477	(1)
Discussion Questions	477	(1)
Sample Article: "Faculty Willingness to Accommodate Students with Learning Disabilities: A Comparison among Academic Divisions"	478	(11)
J. Ron Nelson
John M. Dodd
Deborah J. Smith
CHAPTER 14 CONDUCTING RESEARCH SYNTHESES	489	(34)
Objectives	489	(1)
Overview	489	(1)
What Are the Purposes of Research Syntheses?	490	(1)
Establishing Cause-and-Effect Relationships	490	(1)
Determining the Degree of Relationship between Two Variables	490	(1)
Developing Theories	490	(1)
Assessing the External Validity of Findings	491	(1)
What Are Systematic and Unsystematic Research Syntheses?	491	(2)
Unsystematic Research Syntheses	491	(1)
Systematic Research Syntheses	491	(1)
Comparison of Unsystematic and Systematic Research Syntheses	492	(1)
What Are the Considerations for Conducting Research Syntheses?	493	(1)
Identifying Primary Sources	493	(1)
Identifying Secondary Sources	493	(1)
Overcoming Selection Biases	494	(1)
Focusing the Literature Search	494	(1)
How Do Researchers Plan and Execute Research Syntheses?	494	(21)
Formulating a Precise Research Question	494	(1)
Defining Critical Terms	495	(2)
Formulating a Literature Search Framework	497	(1)
Searching the Literature	497	(3)
Coding Study Characteristics	500	(2)
Determining the Magnitude of Intervention Outcomes	502	(6)
Relating Study Characteristics to Intervention Outcomes	508	(4)
Reporting the Results	512	(1)
Analysis of Data	513	(2)
When Should Researchers Conduct Research Syntheses?	515	(1)
Discussion Questions	515	(1)
Sample Article: "The Moral Reasoning of Juvenile Delinquents: A Meta-Analysis"	516	(7)
J. Ron Nelson
Deborah J. Smith
John Dodd
PART SEVEN ACTION RESEARCH	523	(30)
CHAPTER 15 ACTION RESEARCH: MOVING FROM CRITICAL RESEARCH CONSUMER TO RESEARCHER	525	(28)
Objectives	525	(1)
Overview	525	(1)
What Is Action Research?	526	(1)
Dimensions of Action Research	527	(1)
What Are the Characteristics of Action Research?	527	(1)
Purposes and Value Choice	527	(1)
Contextual Focus	527	(1)
Change-Based Data and Sense Making	528	(1)
Participation in the Research Process	528	(1)
Knowledge Diffusion	528	(1)
How Are Quantitative Methods Used in Action Research?	528	(1)
Research Example	529	(1)
How Are Qualitative Methods Used in Action Research?	529	(2)
Research Example	530	(1)
How Are Single-Case Methods Used in Action Research?	531	(1)
Research Example	531	(1)
How Are Survey, Historical, and Program Evaluation Methods Used in Action Research?	531	(1)
Survey Research	532	(1)
Historical Research	532	(1)
Program Evaluation	532	(1)
What Are the Ethical Principles and Codes of Conduct for Research?	532	(6)
American Educational Research Association	533	(1)
American Psychological Association	533	(5)
Ethical Conduct of Action Researchers	538	(1)
How Does One Write a Research Article?	538	(4)
Contents	538	(3)
A Cautionary Note	541	(1)
How Are Articles Submitted for Publication?	542	(1)
When Should One Conduct Action Research?	542	(1)
Discussion Questions	543	(1)
Sample Article: "Dyads and Data in Peer Coaching"	544	(9)
Cynthia O. Vail
Jennifer M. Tschantz
Alicia Bevill
GLOSSARY	553	(12)
REFERENCES	565	(10)
INDEX	575