Evaluating Information : A Guide for Users of Social Science Research

by KATZER JEFFREY

• ISBN13: 9780070343092
• ISBN10: 0070343098
• Edition: 4th
• Format: Paperback
• Copyright: 1997-06-01
• Publisher: McGraw-Hill Humanities/Social Sciences/Languages

Summary

This book introduces students to social science research from the consumer's point of view. The authors believe that while social science may not require that every student of the discipline will have to conduct research studies, it is still essential for students to successfully read, understand and evaluate the research published in their field. To that end, the authors of this text have streamlined their narrative, omitting the burden of technical jargon, and focusing on the broad elements common to all kinds of social science research, such as experimental, survey, and case study. Questions to Ask, found at the end of each chapter, highlight the specific criteria to consider when evaluating research and offer a clear and accessible presentation of the general principles in social science research.

Table of Contents

SECTION ONE INTRODUCTION: HOW DOES ANYBODY KNOW ANYTHING?	1	(22)
Chapter 1 Purpose: What We Are Trying to Do	3	(9)
Students	3	(1)
Professionals	3	(1)
General Public	4	(1)
THE LITERATURE IS PROBABLY WORSE THAN YOU THINK	5	(1)
WHAT THIS BOOK CAN DO FOR YOU AND HOW WE DO IT	6	(4)
The Error Model	7	(1)
External Criteria	8	(1)
Internal Criteria	9	(1)
QUESTIONS TO ASK	10	(1)
PRACTICE EVALUATIONS	11	(1)
FOR FURTHER THOUGHT	11	(1)
FOR FURTHER READING	11	(1)
Chapter 2 Assumptions: What We Believe about How We Know	12	(11)
WAYS OF KNOWING	13	(3)
Common Sense	13	(1)
Advice of Colleagues	13	(1)
Do Your Own Research	14	(2)
SCIENCE AS A WAY OF KNOWING	16	(2)
Reliance on Evidence	17	(1)
Need for Controlling Error	17	(1)
Standard of Publicness	17	(1)
ARGUMENTS SUPPORTING RESEARCH CONCLUSIONS	18	(2)
Reasoning and Plausibility	18	(1)
Replications	18	(1)
Research Methods	19	(1)
QUESTIONS TO ASK	20	(1)
PRACTICE EVALUATIONS	20	(1)
FOR FURTHER THOUGHT	20	(1)
FOR FURTHER READING	21	(2)
SECTION TWO DISCOVERY AND COMMUNICATION OF RESEARCH FINDINGS: WHERE DO ERRORS COME FROM?	23	(30)
Chapter 3 Observation: Seeing Is Not Believing	25	(11)
YOU COULDN'T SEE IT ALL EVEN IF YOU WANTED TO	26	(2)
The Instruments We Use to Observe	27	(1)
YOUR INTERPRET EVERYTHING YOU SEE	28	(3)
Learning to See	28	(1)
The Context	29	(1)
The "Halo Effect"	30	(1)
YOU PRODUCE SOME OF WHAT YOU SEE	31	(1)
The World Changes	31	(1)
Expectations	31	(1)
WHAT CAN BE MADE OF ALL THIS?	32	(1)
Develop Sensitivity to Error	32	(1)
QUESTIONS TO ASK	33	(1)
PRACTICE EVALUATIONS	33	(1)
FOR FURTHER THOUGHT	34	(1)
FOR FURTHER READING	34	(2)
Chapter 4 Communication: Writing Adds Other Problems	36	(10)
WHAT'S LEFT OUT OF THE ARTICLE?	36	(4)
Conscious and Unconscious Decisions	37	(1)
Evaluating What's Not There	38	(1)
Slightly Distorted Reconstructions	38	(1)
Accepting Author's Assumptions	39	(1)
Findings Are Based on Assumptions	39	(1)
Conclusions Are Also Based on Assumptions	40	(1)
HOW WAS MEANING CONVEYED?	40	(1)
Effect of Words on Meanings	40	(1)
Language as a Smokescreen	41	(1)
WHY WAS THE ARTICLE WRITTEN?	41	(3)
Creation of Phony Data	42	(1)
Researchers as Safeguards	42	(1)
The "Art" of Factifuging	42	(1)
What Do You Know about the Journal?	43	(1)
QUESTIONS TO ASK	44	(1)
PRACTICE EVALUATIONS	44	(1)
FOR FURTHER THOUGHT	44	(1)
FOR FURTHER READING	45	(1)
Chapter 5 Interpretation:...And Then You Read It	46	(7)
WHO, ME?	46	(1)
YOU PROBABLY DON'T READ IT ALL	46	(1)
YOU ALSO HAVE TO INTERPRET IT	46	(1)
THE AURA OF THE PRINTED WORD	47	(1)
SOURCE CREDIBILITY	47	(1)
YOUR APPROACH TO THINKING	48	(2)
QUESTIONS TO ASK	50	(1)
PRACTICE EVALUATIONS	50	(1)
FOR FURTHER THOUGHT	50	(1)
FOR FURTHER READING	51	(2)
SECTION THREE THE NATURE OF ERROR: WHAT KINDS ARE THERE?	53	(20)
Chapter 6 Bias: A Systematic Error	55	(9)
Identifying Sources of Bias	56	(1)
BIASES DUE TO THE RESEARCHER	57	(2)
The Researcher's Expectations	57	(1)
Age and Sex	58	(1)
Inadequate Training	58	(1)
Intentional Deception	58	(1)
BIASES DUE TO THE BEHAVIOR OF SUBJECTS	59	(1)
The Hawthorne Effect	59	(1)
BIASES DUE TO THE RESEARCH PLAN	60	(1)
Use of Pretests	60	(1)
The Selection Process	61	(1)
BIAS AND ERROR	61	(1)
QUESTIONS TO ASK	62	(1)
PRACTICE EVALUATIONS	62	(1)
FOR FURTHER THOUGHT	62	(1)
FOR FURTHER READING	63	(1)
Chapter 7 Noise: The Other Type of Error	64	(9)
Keep the Noise Down	65	(1)
CHARACTERISTICS OF NOISE	66	(1)
Pervasive and Inevitable	66	(1)
Unsystematic Error	66	(1)
Mix the Biases; Live with the Noise	66	(1)
NOISE AND RESEARCH	67	(1)
View Noisy Results as Estimates	67	(1)
Learn to "See" through Noise	68	(1)
SOURCES OF NOISE	68	(2)
Differences among People	68	(1)
Fuzzy Criteria	69	(1)
The "Mixing" Process	70	(1)
QUESTIONS TO ASK	70	(1)
PRACTICE EVALUATIONS	70	(1)
FOR FURTHER THOUGHT	70	(1)
FOR FURTHER READING	71	(2)
SECTION FOUR FACTUALLY ACCURATE INFORMATION: CAN YOU BELIEVE IT?	73	(90)
Chapter 8 Subject Matter: What Is Being Studied?	75	(18)
PROBLEM STATEMENT	76	(3)
Is It Relevant?	77	(1)
Limits and Restrictions	77	(1)
The Author's Orientation	77	(1)
The "Importance Fallacy"	78	(1)
Compare the Findings with the Problem	78	(1)
DEFINITION	79	(6)
Circular Definitions	80	(1)
Specific Definitions	80	(1)
Vague Definitions	81	(1)
Definitional Goals	82	(1)
Operational Definitions	82	(1)
Fruitfulness	83	(2)
A Brief Summary	85	(1)
REVIEW OF RELATED RESEARCH	85	(4)
Reviews Differ	85	(1)
Replications Are Seldom Identical	86	(1)
Criteria for Inclusion	87	(1)
Prior Predictions	88	(1)
SUMMARY	89	(1)
QUESTIONS TO ASK	89	(1)
PRACTICE EVALUATIONS	90	(1)
FOR FURTHER THOUGHT	91	(1)
FOR FURTHER READING	91	(2)
Chapter 9 Measurement: How Does It Size Up?	93	(15)
TAKE SOME TIME TO MEASURE THE AREA	94	(2)
A Closer Look	94	(1)
A Range of Possible Values	95	(1)
Differences and "Real" Differences	95	(1)
NOISE AND BIAS IN MEASUREMENT	96	(2)
Measurement as a Process	96	(1)
The Measuring Instrument	97	(1)
Other Measurement Factors	97	(1)
Acceptable Error: How Much?	98	(1)
RELIABILITY	98	(3)
Degrees of Reliability	98	(1)
Cautions in Reporting	99	(1)
Classification and Reliability	100	(1)
A Restatement	101	(1)
VALIDITY	101	(3)
Prediction as Validity	101	(1)
Construct Validity	102	(1)
Combined Measures	103	(1)
RELIABILITY AND VALIDITY	104	(1)
QUESTIONS TO ASK	105	(1)
PRACTICE EVALUATIONS	106	(1)
FOR FURTHER THOUGHT	106	(1)
FOR FURTHER READING	106	(2)
Chapter 10 Description: Are the Results Summarized Fairly?	108	(10)
SOME GUIDELINES	109	(1)
How Summaries Are Determined	109	(1)
The Alternatives That Exist	109	(1)
What Might Be Left Out	109	(1)
VERBAL DESCRIPTIONS	110	(1)
AVERAGES	111	(2)
PERCENTAGES	113	(3)
Percentiles	114	(2)
QUESTIONS TO ASK	116	(1)
PRACTICE EVALUATIONS	116	(1)
FOR FURTHER THOUGHT	116	(1)
FOR FURTHER READING	117	(1)
Chapter 11 Relationships: More Informative, But More Difficult to Understand	118	(12)
THE IMPORTANCE OF RELATIONSHIPS	119	(8)
Third Variables	120	(3)
Crossing Levels	123	(1)
Freedom to Vary	124	(1)
Causality and Relationships	125	(2)
QUESTIONS TO ASK	127	(1)
PRACTICE EVALUATIONS	128	(1)
FOR FURTHER THOUGHT	128	(1)
FOR FURTHER READING	128	(2)
Chapter 12 Control: Rival Explanations...Is Something Else at Work?	130	(16)
Underlying Patterns of Thought	130	(1)
What's Wrong with A...B...C?	131	(1)
Possible Rivals to B	131	(2)
LOOKING FOR RIVAL EXPLANATIONS	133	(4)
Time	133	(1)
Process of Measurement	134	(1)
People Being Studied	134	(1)
A Classroom Test Example	135	(1)
Dropout Bias	136	(1)
CONTROLLING RIVAL EXPLANATIONS	137	(3)
Using Two or More Groups	138	(1)
Random Assignment to Groups	139	(1)
Other General Remedies	140	(1)
SOME REMAINING PROBLEMS	140	(2)
Identify the "Precise" Cause	141	(1)
Discounting Rival Explanations	142	(1)
SUMMARY	142	(1)
QUESTIONS TO ASK	143	(1)
PRACTICE EVALUATIONS	144	(1)
FOR FURTHER THOUGHT	144	(1)
FOR FURTHER READING	145	(1)
Chapter 13 Inference: Are the Results Real...Or Could They Have Been Caused by Noise?	146	(17)
STATISTICS AND RIVAL EXPLANATIONS	147	(3)
Discounting Random Error as Explanation	147	(1)
An Example of Statistical Inference	148	(1)
Determining "Significance"	149	(1)
The Researcher's Reasoning	150	(1)
THE NULL HYPOTHESIS	150	(1)
REPORTING THE RESULTS	151	(2)
READING AND UNDERSTANDING INFERENTIAL STATISTICS	153	(3)
A MORE GENERAL DEFENSE	156	(4)
1. Are the Results Worth Interpreting Statistically?	156	(1)
2. Do You Judge the Results to Be Real?	157	(1)
3. Are the Results Meaningful to You?	158	(2)
QUESTIONS TO ASK	160	(1)
PRACTICE EVALUATIONS	161	(1)
FOR FURTHER THOUGHT	161	(1)
FOR FURTHER READING	162	(1)
SECTION FIVE USEFUL INFORMATION: SHOULD YOU APPLY IT?	163	(22)
Chapter 14 Generality: Do the Results Apply to You?	165	(12)
Examining the Differences	166	(1)
Discounting Important Differences	167	(1)
CHOOSING PEOPLE	167	(4)
Just One? Or Everyone?	167	(1)
Representativeness and Bias	168	(1)
Random Sampling	169	(1)
How Many? Rule of Thumb 1	169	(1)
How Many? Rule of Thumb 2	170	(1)
How Many? Rule of Thumb 3	170	(1)
CONFIDENCE INTERVALS	171	(2)
What Confidence Intervals Tell You	171	(1)
How Wrong Can the Findings Be?	172	(1)
OTHER CONSIDERATIONS	173	(1)
A QUICK REVIEW	174	(1)
QUESTIONS TO ASK	175	(1)
PRACTICE EVALUATIONS	175	(1)
FOR FURTHER THOUGHT	176	(1)
FOR FURTHER READING	176	(1)
Chapter 15 Being Practical: Going Beyond Error	177	(8)
Differences among Readers' Objectives	178	(1)
Consequences of Being Wrong	179	(1)
Major Decisions for Research Users	180	(2)
QUESTIONS TO ASK	182	(1)
PRACTICE EVALUATIONS	182	(1)
FOR FURTHER THOUGHT	182	(1)
FOR FURTHER READING	183	(2)
SECTION SIX APPLICATIONS: HOW TO DO EVALUATIONS	185	(80)
Chapter 16 A Step-by-Step Guide for Evaluation	187	(3)
FOR FURTHER READING	189	(1)
Chapter 17 Questions to Ask	190	(7)
Chapter 18 Practicing	197	(3)
DO I REALLY NEED DIRECTIONS ABOUT HOW TO PRACTICE?	197	(3)
Questions Focusing on the Evaluator	197	(1)
Questions Focusing on the Article	198	(2)
Chapter 19 Sample Evaluations	200	(30)
"DETERMINANTS OF NONSTATIONARY PERSONAL SPACE INVASION," AN ARTICLE FROM THE JOURNAL OF SOCIAL PSYCHOLOGY	202	(14)
"FACULTY PERCEPTIONS OF A TEACHER CAREER LADDER PROGRAM," AN ARTICLE FROM CONTEMPORARY EDUCATION	216	(14)
Chapter 20 Practice Articles	230	(26)
"THE PORTRAYAL OF CHILDREN ON PRIME-TIME SITUATION COMEDIES," AN ARTICLE FROM THE JOURNAL OF POPULAR CULTURE	231	(7)
"DOES ATTACK ADVERTISING DEMOBILIZE THE ELECTORATE?" AN ARTICLE FROM THE AMERICAN POLITICAL SCIENCE REVIEW	238	(18)
A FINAL WORD	256	(1)
NOTES	257	(7)
GLOSSARY	264	(23)
BIBLIOGRAPHY	287	(2)
INDEX	289