Rapid Testing

by Culbertson, Robert; Brown, Chris; Cobb, Gary

• ISBN13: 9780130912947
• ISBN10: 0130912948
• Edition: 1st
• Format: Paperback
• Copyright: 2001-12-29
• Publisher: Prentice Hall

Summary

Rapid Testingaddresses the growing need for constant, early, and efficient testing in today's fast-paced software development environment. Covers all aspects of testing, including bug tracking and reporting, verification and validation, test planning, test design and development, metrics, effort estimation, and reporting.

Author Biography

Robert Culbertson has over 25 years of experience in engineering, software development, software testing, and project management. He has worked at Cisco Systems, Texas Instruments, IBM, the University of Texas, and DSC Communications.

Chris Brown has worked more than 20 years in the software and computer industry. Chris has experience with Cisco Systems, Advanced Micro Devices, Compaq Computer Corporation, Dataserv/BellSouth, and IBM.

Gary Cobb teaches for the Software Quality Institute at the University of Texas at Austin where he previously taught in the Departments of Mathematics, Computer Science, and Electrical and Computer Engineering. His industrial experience includes Texas Instruments, Lockheed Martin, and Dell Computer Corporation.

Table of Contents

Preface	xv
Part I The Rapid Testing Process	1	(148)
Introduction to Rapid Testing	3	(86)
Basic Definitions for Software Testing	4	(2)
What Is Rapid Testing?	6	(3)
People	7	(1)
Integrated Test Process	8	(1)
Static Testing	8	(1)
Dynamic Testing	8	(1)
Developing a Rapid Testing Strategy	9	(1)
The Software Development Process	9	(3)
A Waterfall Test Process	12	(6)
Requirements Analysis	14	(1)
Test Planning	15	(1)
Test Design, Implementation, and Debugging	16	(1)
System Test	16	(1)
Acceptance Test	17	(1)
Maintenance	17	(1)
Tying Testing and Development Together	18	(3)
What's Next	21	(1)
References	21	(2)
Requirements Analysis & Testing
The Requirements Process	23	(4)
Eliciting Requirements	27	(8)
Requirements Definition	30	(3)
Requirements Specification	33	(2)
Requirements Traceability Matrix	35	(1)
Requirements Testing	36	(6)
What to Look for When Testing Requirements	37	(2)
The use of Prototypes	39	(1)
Testing in an Evolutionary Prototyping Life Cycle	40	(2)
What's Next	42	(1)
References	43	(4)
Test Planning
Test Strategy	47	(1)
Define the Scope of Testing	48	(10)
Define the Testing Approach	51	(2)
Define Testing Criteria and Quality Checkpoints	53	(1)
Define Automation Strategy	54	(4)
Defining the Test System	58	(7)
Test Architecture	58	(2)
Test Tools	60	(1)
Test Environment	61	(3)
Test Configurations	64	(1)
Estimating the Test Effort	65	(11)
Identify Tasks	67	(2)
Determine Effort	69	(4)
Determine Duration and Construct Schedule	73	(2)
Assess Schedule Risk	75	(1)
Preparing and Reviewing the Test Plan Documents	76	(10)
Test Plan Format	76	(9)
Test Plan Reviews	85	(1)
What's Next	86	(1)
References	86	(3)
Test Design and Development	89	(20)
Test Design	90	(6)
Define Test Objectives	93	(1)
Define Input Specifications	94	(1)
Define Test Configurations	94	(1)
Test Design Document	95	(1)
Developing Test Cases	96	(9)
Develop Detailed Test Procedures	98	(2)
Define Expected Results	100	(1)
Setups and Cleanups---Testing from a Known State	101	(1)
Test Case Template	102	(2)
Test Case Configuration Management	104	(1)
Verifying and Debugging Tests	105	(1)
Test Case Automation	106	(1)
What's Next	107	(1)
References	108	(1)
System Testing	109	(24)
Finding and Tracking Bugs	111	(11)
Define Bug States	111	(3)
Bug Tracking Essentials	114	(4)
Write Bug Reports	118	(1)
Bug Reviews	119	(3)
Running Tests	122	(4)
Entry into System Testing	122	(1)
Test Cycles	122	(2)
Log Test Results	124	(2)
Reporting Test Results	126	(4)
Test Status Report	126	(1)
Bug Status Report	127	(1)
Test Summary Report	128	(2)
Exit Criteria and Release Readiness	130	(1)
What's Next	131	(2)
Putting It All Together: People and Process	133	(16)
The People Side of Testing	134	(6)
Traits of a Successful Test Engineer	134	(2)
Traps to Avoid	136	(1)
Interviewing Tips	137	(3)
Improving the Test Process	140	(7)
The Software Engineering Institute's Capability Maturity Model (CMM)	141	(3)
Relationship of the CMM to Rapid Testing	144	(2)
An Approach to Process Improvement	146	(1)
What's Next	147	(1)
References	148	(1)
Part II Rapid Testing Techniques and Tips	149	(144)
Introduction to Testing Techniques and Tips	151	(14)
The Scope of Testing Techniques	151	(1)
Development Life Cycle	152	(3)
Benefits of Rapid Testing	155	(1)
Definition of Static Testing	156	(1)
Definition of Dynamic Testing	157	(1)
Life Cycle of a Bug	158	(2)
Formal Test Phases	160	(1)
Responsibilities of Test Team Members	161	(1)
What's Next	161	(4)
Joint Application Requirements (JAR): A Method for Eliciting Requirements Using Rapid Testing	165	(12)
JAR Methodology	166	(7)
Roles of the Test Engineer in a JAR	173	(2)
Summary	175	(2)
Static Testing Techniques and Tips	177	(30)
Cyclomatic Complexity and Its Relationship to Test Executions	178	(1)
Example of Graphing the Design of a Module	179	(3)
Formal Reviews	182	(3)
Using Checklists	185	(1)
Audits	186	(2)
Inspections/Walkthroughs/Peer Reviews	188	(1)
Inspection Team Roles and Responsibilities	189	(3)
Reporting on the Inspection Process	192	(1)
Metrics for the Inspection Process	192	(1)
Using e-mail or an e-App to Streamline Inspections	193	(1)
Formal Verification	194	(1)
Specification-Based Languages	195	(1)
Automated Theorem Proofs	195	(1)
Testing Automation/Tools	196	(1)
Requirements Traceability	196	(1)
Physical Units Checker	197	(1)
Symbolic Execution	197	(1)
Cross-Reference Listings	198	(1)
Pretty Printers	199	(1)
Version Comparators	199	(1)
Algorithm Testing	200	(3)
Test Support Facilitator	203	(1)
Shared Issues Databases	204	(1)
Summary	205	(1)
References	205	(2)
Dynamic Testing Techniques and Tips	207	(32)
Functional Testing and Analysis	209	(1)
Equivalence Partitioning	210	(1)
Boundary Value Analysis	211	(1)
Negative Testing	211	(2)
Risk-Based Testing	213	(2)
Path Coverage Testing	215	(6)
Use-Case Testing	221	(1)
Bebugging/Mutation	222	(1)
Tracing/Tracebacks/Snap Dumps/Post-Mortem Dumps	223	(1)
Breakpointing/Patching	224	(2)
Data Flow Testing	226	(1)
Testing for Memory Leaks	227	(2)
Human--Computer Interface Testing	229	(1)
Load Performance Testing	230	(4)
Platform Configuration Testing	234	(2)
Summary	236	(1)
References	237	(2)
Developing and Using Testing Metrics: Modeling and Forecasting Bugs	239	(24)
Definition of Metric and Measurement	241	(9)
Using Standard Metrics to Make Improvements	250	(3)
Metrics for Testing	253	(2)
Bug Density (Bugs/KSLOC)	254	(1)
Project-Based Bug Model	255	(2)
Software Error Estimation Program (SWEEP)	257	(4)
Summary	261	(1)
References	262	(1)
Testing Effort Estimation Techniques and Tips	263	(30)
Using Mathematics to Estimate Software Developments	267	(20)
Function Point Technology	287	(3)
Summary	290	(1)
References	291	(2)
Part III Rapid Testing Examples	293	(92)
Example Requirements	295	(20)
Test Management Toolkit Release 1.0 Requirements Definition	297	(18)
Example Test Plan	315	(16)
Test Management Toolkit Release 1.0 Test Plan	318	(13)
Test Design and Development Examples	331	(44)
Test Management Toolkit Release 1.0 Test Procedure Specification	336	(39)
Example System Test Summary Report	375	(10)
Test Management Toolkit Release 1.0 Test Summary Report	377	(8)
Index	385

Excerpts

Preface This book presents a practical approach to software testing, placing emphasis on a test process that is geared to today's aggressive pace of software development. It is designed for use by practicing test engineers and test managers, and includes tips, techniques, and examples that can be used to improve the efficiency and speed of software testing. The book should also be suitable for people who are beginning a career in software testing. It includes an extensive set of references that are intended to support established test professionals as well as those just starting out. The speed and efficiency of software testing depends how well the test process integrates with the overall development life cycle and upon the efficiency of the test techniques used. This book shows how to improve the speed and efficiency of your testing effort by focusing on three areas: Beginning the testing life cycle simultaneously with the project's requirements phase, so that bugs can be caught as early as possible and so test planning and test case development can start as soon as possible. Presenting efficient static testing techniques such as inspections and walkthroughs that can be used to test work products that are generated during the development life cycle. Presenting efficient dynamic testing techniques that can be used to find bugs during integration, system, and acceptance test phases. Key Features The following features of this book are intended to help you improve your software testing efficiency: An emphasis is placed on tailoring the test processes to meet aggressive time-to-market goals without sacrificing product quality. Software testing is placed in the context of the overall software development life cycle. The development life cycle is viewed from the vantage point of the test engineer. Development models such as evolutionary prototyping and spiral development models are considered, as well as the traditional waterfall. Static testing techniques are presented that can be used to get the test team involved early in the development life cycle. Static testing will find bugs early in the life of the product and will allow you to define test plans and test cases as soon as possible. Includes examples of key work products of the test process. How the Book is Organized This book consists of three parts that are organized as follows: Part I. The Rapid Testing Process--This part defines the basic concepts and terms related to software testing. It describes rapid testing processes that are closely integrated with the overall software development life cycle. The traditional waterfall development model is considered, as well as life cycles based on incremental delivery and evolutionary prototyping. Each phase of the software development process is examined from the viewpoint of the software test engineer, and methods of bug detection and prevention are described as means of improving test efficiency. Part II. Rapid Testing Techniques and Tips--This part describes in detail the tips and techniques that can be used to develop a rapid test process. Methods are presented for eliciting and analyzing requirements, estimating and scheduling the test effort, conducting inspections and reviews, designing black box tests, and reporting test results. A variety of dynamic testing methodologies are discussed, including functional analysis, equivalence partitioning, boundary value analysis, testing for memory leaks, use-case testing, and performance tests. Part III. Rapid Testing Examples--Part III contains a set of examples for the process and techniques discussed in the first two parts of the book. The examples are based on the Test Management Toolkit (TMT), which is a pedagogical application that allows test managers and engi