Evaluating Scientific Research : Separating Fact from Fiction

by Leavitt, Fred

• ISBN13: 9781577663454
• ISBN10: 1577663454
• Format: Paperback
• Copyright: 2004-03-01
• Publisher: Waveland Pr Inc

Summary

Acknowledging the profound role science plays in our lives, this book uses a non-technical vocabulary to explain the research process and discuss many controversial issues. It emphasizes creative and critical thinking, challenging many traditional ways of doing research and covering six major problem areas--limitaions of science, preparing for research, measurement, research designs, data analysis, and philosophical issues . Chapter topics include reading and reviewing scientific literature; conflicts of interest and bias; variations on the simple experiment; case studies; using existing data; using animals in research; and philosophical challenges. For anyone who wants to do more than simply accept scientific research at face value.

Table of Contents

Preface	ix
Checklist for Evaluating Research	xv
1 What Is Science?	1	(18)
Scientific Theories Are Testable	2	(3)
Scientists Organize Facts to Solve Meaningful Problems	5	(1)
Additional Views on Philosophy of Science	6	(1)
A Few Noncritical Features	7	(1)
Limitations of Science	7	(6)
Scientific Inference	13	(6)
2 Reading and Reviewing the Scientific Literature	19	(9)
Immensity of the Literature	20	(1)
How to Find the Appropriate Materials	21	(1)
Publication Bias	22	(2)
Read Efficiently, Critically, and Creatively	24	(4)
3 Conflicts of Interest and Bias	28	(18)
Funding Sources	30	(2)
Peer Review of Proposals for Funding	32	(1)
Individual Scientists	33	(5)
Articles Published Without Peer Review	38	(1)
Peer Review of Articles Submitted for Publication	39	(3)
Replication	42	(1)
The Media	42	(3)
Other Tactics	45	(1)
4 Finding Interesting Problems and Studying Them Creatively	46	(11)
Be Attentive to Both Routine and Unusual Occurrences	47	(1)
Read Popular Materials	48	(1)
Don't Just Read	48	(1)
Question	48	(2)
Types of Questions	50	(4)
Asking Questions From Multiple Perspectives	54	(3)
5 Selecting and Measuring Variables	57	(18)
The Difficulty of Measuring Well	58	(1)
Should a New Measure Be Constructed or a Previously Used One Recycled?	58	(4)
Operational Definitions	62	(1)
Constructs	63	(1)
Properties of Good Measures	64	(6)
Strategies for Working with Independent Variables	70	(1)
Use Multiple Measures	71	(1)
Use Nonreactive Measures	72	(1)
Use Surrogate Measures	73	(2)
6 Choosing the Best Research Design	75	(15)
Multiple Methods	76	(1)
Data Collection	77	(1)
Research Designs	78	(2)
Advantages of Multiple Methods	80	(3)
An Example: Cigarette Smoking and Lung Cancer	83	(4)
Why Choose One Method Over Another?	87	(1)
Additional Comments	88	(2)
7 Experimenting: Two Groups	90	(18)
The Logic of Experimentation	91	(1)
Form Groups	91	(1)
Treat Subjects Exactly the Same Except for the Independent Variable	92	(4)
Reasons for Experimenting	96	(2)
Experiments Do Not Eliminate the Need for Judgment	98	(6)
Variability in Experiments	104	(4)
8 Variations on the Simple Experiment	108	(10)
Studying an Independent Variable at More Than One Level or Studying More than One IV	109	(1)
Factorial Designs	110	(6)
Within-Subject Designs	116	(2)
9 Comparing Existing Groups	118	(8)
Reasons for Doing Comparative Studies	119	(2)
Pitfalls in Comparative Research	121	(3)
Systematic Comparisons	124	(2)
10 Correlational Strategies to Predict and Assess Relationships	126	(9)
Interpreting Correlation Coefficients	127	(3)
When Should a Correlational Study Be Considered?	130	(2)
Three Potential Problems	132	(1)
Several Correlation Coefficients	133	(1)
Partial Correlation	133	(1)
Correlational and Causation-Additional Comments	134	(1)
11 Case Studies	135	(15)
Unplanned, Isolated Cases	136	(4)
Case Studies as Planned, After-the-Fact, Research Designs	140	(2)
Single-Case Experimental Designs	142	(8)
12 Observing	150	(11)
Reasons for Doing Observational Studies	151	(1)
Becoming a Good Observer	152	(7)
Some Problems	159	(2)
13 Surveys	161	(19)
Writing Questions	163	(1)
Assembling the Questionnaire	163	(1)
Selecting the Sample	164	(4)
Nonresponse	168	(1)
Administering the Questions	168	(2)
Research on Survey Validity	170	(1)
Unique Qualities of Surveys	170	(4)
Retrospective Recall and Consistency	174	(1)
Why Survey Houses Disagree	174	(4)
Alternatives to Surveys	178	(2)
14 Other Types of Self-Report	180	(9)
The Existence of Self-Reports is Objective and Verifiable	182	(1)
The Correspondence Between Descriptions of Inner Experiences and Other Data Can Often Be Evaluated	182	(1)
Hypotheses Generated From Introspective Data Can Be Falsified	183	(1)
Some Uses of Self-Report Methods	183	(3)
Methods of Collecting Self-Report Data	186	(3)
15 Using Existing Data	189	(12)
Reasons for Using Existing Data Sources	190	(1)
Locating the Appropriate Materials	191	(1)
Types of Archival Research	191	(8)
Weaknesses of Archival Research	199	(2)
16 Using Animals in Research	201	(9)
Why Use Animals in Research	203	(7)
17 Data Analysis	210	(17)
Null Hypothesis Significance Testing	211	(1)
Types of Errors	212	(2)
Misinterpretations by Professionals	214	(2)
Implications for Public Policy	216	(1)
Theory Testing	217	(1)
Incentive to Cheat	218	(1)
A Few Useful Statistical Procedures	218	(1)
Exploratory Data Analysis	219	(5)
Three Alternatives to Null-Hypothesis Testing	224	(3)
18 Philosophical Challenges	227	(8)
Hempel's Paradox	228	(1)
Induction: Hume, Russell, Goodman	228	(1)
The Fundamental Uncertainty	229	(4)
Comments on the Philosophical Challenges	233	(2)
References	235	(20)
Name Index	255	(8)
Subject Index	263