Beautiful Testing

by Riley, Tim

• ISBN13: 9780596159818
• ISBN10: 0596159811
• Edition: 1st
• Format: Paperback
• Copyright: 2009-10-30
• Publisher: Oreilly & Associates Inc

Summary

Successful software depends as much on scrupulous testing as it does on solid architecture or elegant code. But testing is not a routine process, it's a constant exploration of methods and an evolution of good ideas. Beautiful Testingoffers 23 essays from 27 leading testers and developers that illustrate the qualities and techniques that make testing an art. Through personal anecdotes, you'll learn how each of these professionals developed beautiful ways of testing a wide range of products -- valuable knowledge that you can apply to your own projects. Here's a sample of what you'll find inside:Microsoft's Alan Page knows a lot about large-scale test automation, and shares some of his secrets on how to make it beautiful Scott Barber explains why performance testing needs to be a collaborative process, rather than simply an exercise in measuring speed Karen Johnson describes how her professional experience intersected her personal life while testing medical software Rex Black reveals how satisfying stakeholders for 25 years is a beautiful thing Mathematician John D. Cook applies a classic definition of beauty, based on complexity and unity, to testing random number generatorsAll author royalties will be donated to the Nothing But Nets campaign to save lives by preventing malaria, a disease that kills millions of children in Africa each year. This book includes contributions from:Adam Goucher Linda Wilkinson Rex Black Martin Schroder Clint Talbert Scott Barber Kamran Khan Emily Chen Brian Nitz Remko Troncon Alan Page Neal Norwitz Michelle Levesque Jeffrey Yasskin John D. Cook Murali Nandigama Karen N. Johnson Chris McMahon Jennitta Andrea Lisa Crispin Matt Heusser Andreas Zeller David Schuler Tomasz Kojm Adam Christian Tim Riley Isaac Clerencia

Author Biography

*Tim Riley is the Director of Quality Assurance at Mozilla. He has tested software for 18 years including everything from spacecraft simulators, ground control systems, high security operating systems, language platforms, application servers, hosted services and open source web applications. He has managed software testing teams in startups to large corporations consisting of 3 to 120 people in size and in up to 6 countries. He has a software patent for a testing execution framework which matches test suites to available test systems. * Adam Goucher has been testing software professionally for over ten years. In that time he has worked with start-ups, large multi-nationals and ones in between in both traditional and agile testing environments. A believer in the communication of ideas big and small, he writes frequently at http://adam.goucher.ca and teaches testing skills at a Toronto area technical college.

Table of Contents

Preface	p. xiii
Beautiful Testers
Was It Good For You	p. 3
Beautiful Testing Satifies Stakeholders	p. 15
For Whom Do We Test?	p. 16
What Satisfies?	p. 18
What Beauty Is External?	p. 20
What Beauty Is Internal?	p. 23
Conclusions	p. 25
Building Open Source Qa Communities	p. 27
Communication	p. 27
Volunteers	p. 28
Coordination	p. 29
Events	p. 32
Conclusions	p. 35
Collaboration Is The Cornerstone of Beautiful Performance Testing	p. 37
Settingthe Stage	p. 38
100%?!? Fail	p. 38
The Memory Leak Thai Wasn't	p. 45
Can't Handle the Load? Change the UI	p. 46
It Can't Be the Network	p. 48
Wrap-Up	p. 51
Beautiful Process
Just Peachy: Making Office Software More Reliable With Fuzz Testing	p. 55
User Expectations	p. 55
What Is Fuzzing?	p. 57
Why Fuzz Test?	p. 57
Fuzz Testing	p. 60
Future Considerations	p. 65
Bug Management and Test Case Effectiveness	p. 67
Bug Management	p. 68
The First Step in Managing a Defect Is Defining It	p. 70
Test Case Effectiveness	p. 77
Case Study of the OpenSolaris Desktop Team	p. 79
Conclusions	p. 83
Acknowledgments	p. 83
References	p. 84
Beautiful XMMP Testing	p. 85
Introduction	p. 85
XMPP 101	p. 86
Testing XMPP Protocols	p. 88
Unit Testing Simple Request-Response Protocols	p. 89
Unit Testing Multistage Protocols	p. 94
Testing Session initialization	p. 97
Automated Interoperability Testing	p. 99
Diamond in the Rough: Testing XML Validity	p. 101
Conclusions	p. 101
References	p. 102
Beautiful Large-Scale Test Automation	p. 103
Before We Start	p. 104
What Is Large-Scale Test Automation?	p. 104
The First Steps	p. 106
Automated Tests and Test Case Management	p. 107
The Automated Test Lab	p. 111
Test Distribution	p. 112
Failure Analysis	p. 114
Reporting	p. 114
Putting It All Together	p. 116
Beautiful Is Better Then Ugly	p. 119
The Value of Stability	p. 120
Ensuring Correctness	p. 121
Conclusions	p. 127
Testing A Random Number Generator	p. 125
What Makes Random Number Generators Subtle to Test?	p. 130
Uniform Random Number Generators	p. 131
Nonuniform Random Number Generators	p. 132
A Progression of Tests	p. 134
Conclusions	p. 141
Change-Centric Testing	p. 143
How to set Up the Document-Driven, Change-Centric Testing Framework?	p. 145
Change-Centric Testing for Complex Code Development Models	p. 146
What Have We Learned So Far?	p. 152
Conclusions	p. 154
Software In Use	p. 155
A Connection to My Work	p. 156
From the Inside	p. 157
Adding Different Perspectives	p. 159
Exploratory, Ad-Hoc, and Scripted Testing	p. 161
Multiuser Testing	p. 163
The Science Lab	p. 165
Simulating Real Use	p. 166
Testing in the Regulated World	p. 168
At the End	p. 169
Software Development is a Creative Process	p. 171
Agile Development As Performance	p. 172
Practice, Rehearse, Perform	p. 173
Evaluating the Ineffable	p. 174
Two Critical Tools	p. 174
Software Testing Movements	p. 176
The Beauty of Agile Testing	p. 177
QA Is Not Evil	p. 178
Beauty Is the Nature of This Work	p. 179
References	p. 179
Test-Driven Development: Driving New Standards of Beauty	p. 181
Beauty As Proportion and Balance	p. 181
Agile: A New Proportion and Balance	p. 182
Test-Driven Development	p. 182
Examples Versus Tests	p. 184
Readable Examples	p. 185
Permanent Requirement Artifacts	p. 186
Testable Designs	p. 187
Tool Support	p. 189
Team Collaboration	p. 192
Experience the Beauty of TDD	p. 193
References	p. 194
Beautiful Testing as the Cornerstone of Business Success	p. 195
The Whole-Team Approach	p. 197
Automating Tests	p. 199
Driving Development with Tests	p. 202
Delivering Value	p. 206
A Success Story	p. 208
PostScript	p. 208
Pealing The Glass Onion At Socialtext	p. 209
It's Not Business...It's Personal	p. 209
Tester Remains On-Stage; Enter Beauty, Stage Right	p. 210
Come Walk with Me, The Best Is Yet to Be	p. 213
Automated Testing Isn't	p. 211
Into Socialtext	p. 215
A Balanced Breakfast Approach	p. 227
Regression and Process Improvement	p. 231
The Last Pieces of the Puzzle	p. 231
Acknowledgments	p. 233
Beautiful Testing Is Efficient Testing	p. 235
SLIME	p. 235
Scripting	p. 239
Discovering Developer Notes	p. 210
Oracles and Test Data Generation	p. 241
Mindmaps	p. 242
Efficiency Achieved	p. 244
Beautiful Tools
Seeding Bugs to Find Bugs: Beautiful Mutation Testing	p. 247
Assessing Test Suite Quality	p. 247
Watching the Watchmen	p. 249
An AspectJ Example	p. 252
Equivalent Mutants	p. 253
Focusing on Impact	p. 254
The Javalanche Framework	p. 255
Odds and Ends	p. 255
Acknowledgments	p. 256
References	p. 256
Reference Testing as Beautiful Testing	p. 257
Reference Test Structure	p. 258
Reference Test Extensibility	p. 261
Building Community	p. 266
Clam Anti-Virus: Testing Open Source with Open Tools	p. 269
The Clam Anti-Virus Project	p. 270
Testing Methods	p. 270
Summary	p. 283
Credits	p. 283
Web Application Testing With Windmill	p. 285
Introduction	p. 285
Overview	p. 286
Writing Tests	p. 286
The Project	p. 292
Comparison	p. 293
Conclusions	p. 293
References	p. 291
Testing One Million Web Pages	p. 295
In the Beginning...	p. 296
The Tools Merge and Evolve	p. 297
The Nitty-Gritty	p. 299
Summary	p. 301
Acknowledgments	p. 301
Testing Network Services in Multimachine Scenarios	p. 303
The Need for an Advanced Testing Tool in eBox	p. 303
Development of ANSTE lo Improve the eBox QA Process	p. 304
How eBox Uses ANSTE	p. 307
How Other Projects Can Benefit from ANSTE	p. 315
Contributors	p. 317
Index	p. 323
Table of Contents provided by Ingram. All Rights Reserved.

Supplemental Materials

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

Rewards Program

Currently unavailable