Solid Software

by Pfleeger, Shari Lawrence; Hatton, Les; Howell, Charles C.

• ISBN13: 9780130912985
• ISBN10: 0130912980
• Edition: 1st
• Format: Paperback
• Copyright: 2001-07-02
• Publisher: PEARSO

Summary

Preface

They constantly try to escape...by dreaming of systems so perfect that no one will need to be good.

T. S. Eliot, Choruses from The Rock, VI

You're in charge. The buck or pound or peso stops with you. Your developers are to build a safety- or business-critical system, and you have a lot of questions to answer. How solid is the software supposed to be? How will you be able to demonstrate to the clients that it is as solid as they wish it? How will your developers be able to demonstrate to you that the software will be solid and (eventually) is solid, so that you can give assurances to your boss and to the clients? You know that there is (unfortunately) no easy solution to the challenges you face-no "eat all the cake you want and still lose weight diet" for developing critical software. But you can take advantage of the experience of others in a wide range of critical software projects.

There are many books for developers and much research about the theoretical ways to build software that does what it is supposed to do (and nothing more, like a virus or Trojan horse) and does it in a consistent, predictable, and safe way. There are theoretical books about how to evaluate the software before you field it or deliver it. But with safety-critical systems, many of which would need over 100,000 years of failure-free testing to confirm required reliability, theory is not enough. You need to know what is practical, what is available right now, and what can give you confidence in the quality of the requirements, design, code, and test procedures.

This is the book for you. In Solid Software we describe the problem and suggest what you can and cannot expect from your developers, their techniques and tools, and their software. We discuss what you should know about software quality-not just about the faults and failures but also how the quality affects your company's bottom line. Then we introduce eight techniques, one chapter at a time, that can help to increase your confidence—and that of your clients—in how the software will perform:

• Hazard analysis
• Testing
• Design analysis
• Prediction
• Reviews
• Static code analysis
• Configuration management and change control
• Tools

None of these techniques is foolproof, but each one helps you to manage the risks inherent in producing such critical code. When properly applied, each one gives you added confidence that you have addressed key points of vulnerability. When used in concert, these techniques stabilize the software, making it less likely to fail and more easy to change and expand.

Author Biography

SHARI LAWRENCE PFLEEGER is President of Systems/Software, Inc., a leading software engineering consultancy. She has been founder/director of Howard University's Center for Research in Evaluating Software Technology, visiting scientist at the City University (London) Centre for Software Reliability, principal scientist at MITRE Corporation's Software Engineering Center, and manager of the measurement program at Contel Technology Center.

LES HATTON is managing partner at Oakwood Consulting, advising clients such as Ford and Philips on software system safety.

CHARLES C. HOWELL, Chief Engineer at MITRE Corporation, has served as Director of Consulting Services at Reliable Software Technologies and as Java Technologist at Sun Microsystems.

Table of Contents

Preface	xv
Why Is This Book Needed?	1	(18)
Software: The Universal Weak Link?	1	(6)
Unreliability and Unavailability	3	(1)
Lack of Security	4	(1)
Unpredictable Performance	5	(1)
Difficulty in Upgrading	5	(1)
Trade-offs and Correlation in Aspects of Fragility	6	(1)
Why Is This So Hard?	7	(5)
Programmer Optimism and Gutless Estimating	7	(1)
Discrete versus Continuous Systems	8	(1)
Immaturity Combined with Rapid Change	9	(1)
Repeating Our Mistakes	10	(2)
Solid, Survivable Software	12	(1)
Critical Systems	12	(1)
Stakeholders	13	(1)
Surviving a Software Project	13	(1)
The Road Ahead	14	(3)
References	17	(2)
Defining Quality: What Do You Want?	19	(22)
Five Views of Quality	19	(3)
Risky Business	22	(3)
Risk and Quality	25	(2)
Consequences of Failure	27	(10)
Product Failure	27	(4)
Process Failure	31	(3)
Resource Failure	34	(3)
Rules of the Road	37	(2)
References	39	(2)
Hazard Analysis	41	(24)
The Rewards of Caution	41	(2)
What Is Hazard Analysis?	43	(2)
Hazop	45	(2)
Fault-Tree Analysis	47	(3)
Failure Modes and Effects Analysis	50	(2)
How to Describe Problems	52	(7)
Failure Modes	52	(2)
Consequences and Probability	54	(5)
Planning for Hazard Analysis	59	(3)
Who Performs the Hazard Analysis?	59	(1)
When Are You Done?	60	(2)
For Additional Information	62	(1)
References	62	(3)
Testing	65	(46)
Types of Faults	66	(6)
Orthogonal Defect Classification	69	(3)
Testing Strategies	72	(11)
Types of Testing	72	(2)
Approaches to Unit Testing	74	(3)
Approaches to Integration Testing	77	(2)
Comparison of Integration Strategies	79	(1)
Approaches to Acceptance Testing	80	(2)
Results of Acceptance Tests	82	(1)
Approaches to Installation Testing	83	(1)
Test Cases and Results	83	(2)
Keeping Test Cases and Data	85	(1)
Who Should Test?	85	(2)
Automated Testing Tools	87	(6)
Code Analysis Tools	88	(2)
Test Execution Tools	90	(3)
Testing: Good and Bad	93	(5)
How Much Testing Is Enough?	98	(5)
Test Planning	98	(4)
Stopping Criteria	102	(1)
Assessing Testing Risk and Trade-offs	103	(4)
Comparing Techniques	105	(2)
Cost and Return on Investment	107	(1)
References	107	(4)
Software Design	111	(34)
The Audience for Design	112	(1)
The Meaning of Good Design	113	(8)
Modularity, Levels of Abstraction, and Information Hiding	113	(2)
Component Independence	115	(6)
Issues to Consider in Good Design	121	(10)
Collaborative Design	122	(2)
Designing the User Interface	124	(4)
Concurrency	128	(3)
Design Leverage Points	131	(10)
Fault Tolerance Philosophy	131	(3)
Error-Handling Design	134	(5)
Design Rationale and History	139	(1)
Design Patterns	140	(1)
References	141	(4)
Prediction	145	(46)
Predicting Software Characteristics	146	(6)
The Jelinski-Moranda Model	149	(1)
The Littlewood Model	149	(1)
Importance of the Operational Environment	150	(2)
Predicting Effort	152	(8)
Expert Judgment	153	(3)
Algorithmic Methods	156	(1)
Machine Learning Methods	157	(3)
Evaluating Model Accuracy	160	(2)
Predicting and Evaluating Return on Investment	162	(10)
Technical Quality Alone Is Misleading	162	(4)
Customer Satisfaction Alone Is Inadequate	166	(2)
Market Leadership Alone Is Inappropriate	168	(1)
Using Economic Value to Support Investment Decisions	168	(4)
Predicting and Managing Risk	172	(15)
What Is a Risk?	173	(1)
Risk Management Activities	174	(4)
Cautions about Risk	178	(7)
Cleaning Up Our Risks	185	(1)
Acknowledgments	186	(1)
References	187	(4)
Peer Reviews	191	(24)
What Is a Review?	191	(3)
Review Effectiveness	194	(3)
Product Inspection	197	(4)
Planning	198	(2)
Individual Preparation	200	(1)
Logging Meeting	200	(1)
Reworking	201	(1)
Reinspection	201	(1)
Process Improvement	201	(4)
Reviewing Speed	202	(2)
Fault Discovery: What the Reviewers Find	204	(1)
Fault Evasion: What the Reviewers Miss	204	(1)
How to Improve Review Results: The Psychological Basis	205	(2)
Automating the Review Process	207	(2)
Pitfalls of the Review Process	209	(1)
The Role of Checklists	210	(2)
Example Checklist for Source Code Inspections	210	(2)
References	212	(3)
Static Analysis	215	(18)
Static Fault versus Dynamic Failure	216	(1)
When Faults Cause Failures	216	(4)
Early versus Late Detection	220	(1)
Measurements for Static Analysis	220	(3)
Coverage: How Much Is Enough?	223	(1)
Approaches to Static Analysis	224	(6)
Static Analysis of Designs	224	(1)
Using Automation to Find Code Faults	224	(6)
Code Faults That Cannot Be Found by Automation	230	(1)
Static Noise	230	(2)
References	232	(1)
Configuration Management	233	(30)
Constant Change	233	(4)
Corrective Changes	235	(1)
Adaptive Changes	235	(1)
Perfective Changes	236	(1)
Preventive Changes	236	(1)
Worth the Effort?	237	(2)
Getting Control	239	(2)
Versions, Releases, and the Challenge of Commercial Components	241	(3)
The Four Facets of SCM	244	(5)
Configuration Identification	244	(1)
Configuration Control and Change Management	245	(3)
Configuration Auditing	248	(1)
Status Accounting	249	(1)
Applying the Principles: Regression Testing	249	(1)
Change Control Boards	250	(2)
Impact Analysis	252	(4)
One Size Does Not Fit All	256	(1)
Tool Support	256	(3)
Text Editors	256	(1)
File Comparators	257	(1)
Compilers and Linkers	257	(1)
Cross-reference Generators	257	(1)
Static Code Analyzers	257	(1)
Configuration Management Repositories	258	(1)
Which Tools to Use?	258	(1)
Begin with the End, but Start Where You Are	259	(1)
References	260	(3)
Using Appropriate Tools	263	(14)
How Tools Develop	264	(1)
The Evolution of Software Tools	265	(3)
Tool Properties	268	(1)
The Anatomy of a Valuable Tool	269	(3)
The Unix Pipeline	269	(2)
TCL/TK	271	(1)
SKS	271	(1)
Tool Quality	272	(1)
Compiler Validation	272	(1)
Tooling and Process	273	(2)
SAM 2000	274	(1)
Tooling and the Organization	275	(1)
References	276	(1)
Trust but Verify	277	(32)
Where We Are	277	(2)
Learning from Mistakes	279	(7)
The Survey	280	(2)
Objective Information	282	(1)
Debriefing Meeting	282	(1)
Project History Day	283	(1)
Publishing the Results	284	(2)
The Importance of Being Human	286	(3)
Best Practices	289	(2)
Making Decisions	291	(10)
Group Decision Making	294	(2)
How We Really Decide	296	(2)
How Groups Really Make Decisions	298	(1)
Deciding What Is Right for Your Situation	299	(2)
What's Next?	301	(6)
S-systems	302	(1)
P-systems	303	(1)
E-systems	304	(3)
References	307	(2)
Index	309

Excerpts

Preface They constantly try to escape...by dreaming of systems so perfect that no one will need to be good. T. S. Eliot, Choruses fromThe Rock,VIYou're in charge. The buck or pound or peso stops with you. Your developers are to build a safety- or business-critical system, and you have a lot of questions to answer. How solid is the software supposed to be? How will you be able to demonstrate to the clients that it is as solid as they wish it? How will your developers be able to demonstrate to you that the software will be solid and (eventually) is solid, so that you can give assurances to your boss and to the clients? You know that there is (unfortunately) no easy solution to the challenges you face-no "eat all the cake you want and still lose weight diet" for developing critical software. But you can take advantage of the experience of others in a wide range of critical software projects.There are many books for developers and much research about the theoretical ways to build software that does what it is supposed to do (and nothing more, like a virus or Trojan horse) and does it in a consistent, predictable, and safe way. There are theoretical books about how to evaluate the software before you field it or deliver it. But with safety-critical systems, many of which would need over 100,000 years of failure-free testing to confirm required reliability, theory is not enough. You need to know what is practical, what is available right now, and what can give you confidence in the quality of the requirements, design, code, and test procedures.This is the book for you. InSolid Softwarewe describe the problem and suggest what you can and cannot expect from your developers, their techniques and tools, and their software. We discuss what you should know about software quality-not just about the faults and failures but also how the quality affects your company's bottom line. Then we introduce eight techniques, one chapter at a time, that can help to increase your confidence--and that of your clients--in how the software will perform: Hazard analysis Testing Design analysis Prediction Reviews Static code analysis Configuration management and change control ToolsNone of these techniques is foolproof, but each one helps you to manage the risks inherent in producing such critical code. When properly applied, each one gives you added confidence that you have addressed key points of vulnerability. When used in concert, these techniques stabilize the software, making it less likely to fail and more easy to change and expand.