Identifying Relevant Information for Testing Technique Selection

by Vegas, Sira; Juristo, Natalia; Basili, Victor R.

• ISBN13: 9781402074356
• ISBN10: 1402074352
• Format: Hardcover
• Copyright: 2003-04-01
• Publisher: Kluwer Academic Pub

Summary

The importance of properly selecting testing techniques is widely accepted in the software engineering community today. However, there are chiefly two reasons why the selections now made by software developers are difficult to evaluate as correct. First, there are several techniques with which the average developer is unfamiliar, often leaving testers with limited knowledge of all the techniques currently available. Second, the available information regarding the different testing techniques is primarily procedure (focused on how to use the technique), rather than pragmatic (focused on the effect and appropriateness of using the technique). The problem addressed in this book is aimed at improving software testing technique selection. Identifying Relevant Information for Testing Technique Selection: An Instantiated Characterization Schema will train its readers how to use the conceptual tool presented here in various ways. Developers will improve their testing technique selection process by systematically and objectively selecting the testing techniques for a software project. Developers will also build a repository containing their own experience with the application of various software testing techniques. Researchers will focus their research on the relevant aspects of testing technique when creating it, and when comparing different techniques. Identifying Relevant Information for Testing Technique Selection: An Instantiated Characterization Schema is designed to meet the needs of a professional audience in software engineering. This book is also suitable for graduate-level students in computer science and engineering.

Table of Contents

List of Figures	xi
List of Tables	xviii
Foreword	xix
Preface	xxi
Acknowledgments	xxiii
Introduction	1	(10)
Basics on Software Testing	1	(4)
Dynamic vs. Static Evaluation of Code	1	(1)
Faults and Failures	2	(1)
Testing Process	3	(2)
The Problem of Selecting Testing Techniques	5	(3)
Getting a Characterisation Schema for Testing Techniques	8	(2)
Organisation of the Book	10	(1)
State of the Practice	11	(26)
Areas Surveyed	11	(2)
Testing Area	13	(13)
Descriptive Criteria	13	(3)
Related Work in the Testing Area	16	(10)
Conclusions on the Testing Area	26	(1)
Area of Characterisation	26	(10)
Descriptive Criteria	26	(4)
Related Work in the Characterisation Area	30	(4)
Conclusions on the Area of Characterisation	34	(2)
Conclusions on the State of the Practice	36	(1)
Research Goals	37	(12)
Description of the Problem	37	(5)
Problem-Solving Approach	42	(1)
Hypotheses	43	(6)
Working Hypotheses	44	(1)
Evaluating the Working Hypotheses	45	(4)
Problem Solving Process	49	(16)
Research Methods	49	(1)
Objectives of Technological Research	50	(3)
Applying the Scientific Method to Technological Research	53	(4)
Expert Peer Review versus Experimental Testing	57	(2)
The Problem-Solving Process Used in this Book	59	(6)
First Generative Iteration: Deductive Theoretical Schema	60	(1)
Second Generative Iteration: Inductive Empirical Schema	61	(1)
Synthesis of Perspectives: Proposal of a Preliminary Schema	62	(1)
Improvement of the Schema: Expert Peer Review	63	(1)
Empirical Evaluation	63	(1)
Experimental Evaluation	63	(2)
First Generativeiteration: Deductive Theoretical Schema	65	(32)
Description of the Research Stage: Generation of a Theoretical Schema	65	(1)
Testing Process	65	(2)
Stratification of Testing-Related Information	67	(3)
Testing Process Elements	70	(3)
Tactical Level	70	(1)
Operational Level	71	(2)
Attributes of the Theoretical Schema	73	(14)
Operational Level	73	(12)
Tactical Level	85	(2)
Result of the Construction of the Theoretical Schema	87	(1)
Use and Evolution of the Characterisation Schema	87	(10)
Primary Use of the Repository: Selection	89	(4)
Secondary Uses of the Repository: Evolution	93	(3)
Secondary Uses of the Repository: Research	96	(1)
The Role of the Librarian in the Repository	96	(1)
Second Generative Iteration: Inductive Empirical Schema	97	(38)
Description of the Research Stage: Generation of an Empirical Schema	97	(1)
Data Collection	98	(4)
Characterisation of the Population of Respondents	98	(1)
Survey Coverage	99	(2)
Form Building	101	(1)
Data Analysis	102	(11)
Creation of the Reference Set	103	(1)
Analysis of the Reference Set	103	(10)
Result of Building the Empirical Schema	113	(1)
Study of the Evolution of the Empirical Schema	113	(22)
Schema Growth	125	(3)
Importance of Each Schema Attribute	128	(5)
Schema Evolution	133	(2)
Synthesis of Perspectives: Proposal of the Preliminary Schema	135	(12)
Description of the Research Stage: Synthesis of Schemas	135	(1)
Rules of Synthesis	135	(2)
Synthesis of the Theoretical and Empirical Schemas	137	(4)
First Rule of Synthesis: Levels and Elements of the Preliminary Schema	137	(1)
Second Rule of Synthesis: Attributes of a Schema	138	(1)
Third Rule of Synthesis: Equal Attributes of Two Schemas	139	(1)
Fourth Rule of Synthesis: Similar Attributes in Two Schemas	139	(2)
Result of Schema Synthesis	141	(1)
Study of Schema Synthesis	141	(6)
Improvement of the Schema: Expert Peer Review	147	(20)
Description of the Research Stage: Improvement	147	(1)
Questionnaire for Experts	148	(1)
Questionnaire Analysis Method	149	(2)
Analysis of Responses	151	(12)
Part 1: Respondents	151	(1)
Part 2: Generic Questions	151	(2)
Part 3: Attributes	153	(5)
Part 4: Elements	158	(1)
Part 5: Levels	159	(1)
Part 6: Other Remarks	160	(3)
Improved Schema	163	(4)
Empirical Evaluation	167	(10)
Objective of the Empirical Evaluation	167	(1)
Choice of the Workload	168	(1)
Schema Instantiation	168	(1)
Schema Use	168	(1)
Analysis of the Results	168	(7)
Feasibility	169	(6)
Schema Flexibility	175	(1)
Conclusions on the Empirical Evaluation	175	(2)
Experimental Evaluation	177	(64)
Objective of the Experiment	177	(2)
Experiment Planning	179	(8)
Parameters	182	(1)
Factors and Their Alternatives	183	(2)
Response Variables	185	(2)
Experimental Design	187	(7)
Two-Factor Design with Replication	189	(1)
Experimental Procedure	190	(2)
Threats to Validity	192	(2)
Data Analysis	194	(43)
Characteristics of the Subjects	195	(1)
Schema Efficiency	196	(5)
Schema Usability	201	(10)
Schema Completeness	211	(13)
Schema Effectiveness	224	(8)
Schema Satisfaction	232	(1)
Conclusions on Groups	233	(4)
Conclusions on Experimental Evaluation	237	(1)
Characterisation Schema Improvement	238	(3)
Conclusions	241	(3)
Appendices	244	(33)
Forms Used to Obtain the Empirical Schema	245	(2)
Questionnaires Used in Expert Peer Review	247	(4)
Schema Instantiation	251	(14)
Experiment Forms	265	(12)
Index	277