xUnit Test Patterns Refactoring Test Code

by Meszaros, Gerard

• ISBN13: 9780131495050
• ISBN10: 0131495054
• Format: Hardcover
• Copyright: 2007-05-21
• Publisher: Addison-Wesley Professional

Summary

Improves software return on investment by teaching the reader how to refactor test code and reduce or prevent crippling test maintenance.

Author Biography

Gerard Meszaros is Chief Scientist and Senior Consultant at ClearStream Consulting

Table of Contents

Visual Summary of the Pattern Language	p. xvii
Foreword	p. xix
Preface	p. xxi
Acknowledgments	p. xxvii
Introduction	p. xxix
Refactoring a Test	p. xlv
The Narratives	p. 1
A Brief Tour	p. 3
About This Chapter	p. 3
The Simplest Test Automation Strategy That Could Possibly Work	p. 3
Development Process	p. 4
Customer Tests	p. 5
Unit Tests	p. 6
Design for Testability	p. 7
Test Organization	p. 7
What's Next?	p. 8
Test Smells	p. 9
About This Chapter	p. 9
An Introduction to Test Smells	p. 9
What's a Test Smell?	p. 10
Kinds of Test Smells	p. 10
What to Do about Smells?	p. 11
A Catalog of Smells	p. 12
The Project Smells	p. 12
The Behavior Smells	p. 13
The Code Smells	p. 16
What's Next?	p. 17
Goals of Test Automation	p. 19
About This Chapter	p. 19
Why Test?	p. 19
Economics of Test Automation	p. 20
Goals of Test Automation	p. 21
Tests Should Help Us Improve Quality	p. 22
Tests Should Help Us Understand the SUT	p. 23
Tests Should Reduce (and Not Introduce) Risk	p. 23
Tests Should Be Easy to Run	p. 25
Tests Should Be Easy to Write and Maintain	p. 27
Tests Should Require Minimal Maintenance as the System Evolves Around Them	p. 29
What's Next?	p. 29
Philosophy of Test Automation	p. 31
About This Chapter	p. 31
Why Is Philosophy Important?	p. 31
Some Philosophical Differences	p. 32
Test First or Last?	p. 32
Tests or Examples?	p. 33
Test-by-Test or Test All-at-Once?	p. 33
Outside-In or Inside-Out?	p. 34
State or Behavior Verification?	p. 36
Fixture Design Upfront or Test-by-Test?	p. 36
When Philosophies Differ	p. 37
My Philosophy	p. 37
What's Next?	p. 37
Principles of Test Automation	p. 39
About This Chapter	p. 39
The Principles	p. 39
What's Next?	p. 48
Test Automation Strategy	p. 49
About This Chapter	p. 49
What's Strategic?	p. 49
Which Kinds of Tests Should We Automate?	p. 50
Per-Functionality Tests	p. 50
Cross-Functional Tests	p. 52
Which Tools Do We Use to Automate Which Tests?	p. 53
Test Automation Ways and Means	p. 54
Introducing xUnit	p. 56
The xUnit Sweet Spot	p. 58
Which Test Fixture Strategy Do We Use?	p. 58
What Is a Fixture?	p. 59
Major Fixture Strategies	p. 60
Transient Fresh Fixtures	p. 61
Persistent Fresh Fixtures	p. 62
Shared Fixture Strategies	p. 63
How Do We Ensure Testability?	p. 65
Test Last-at Your Peril	p. 65
Design for Testability-Upfront	p. 65
Test-Driven Testability	p. 66
Control Points and Observation Points	p. 66
Interaction Styles and Testability Patterns	p. 67
Divide and Test	p. 71
What's Next?	p. 73
xUnit Basics	p. 75
About This Chapter	p. 75
An Introduction to xUnit	p. 75
Common Features	p. 76
The Bare Minimum	p. 76
Defining Tests	p. 76
What's a Fixture?	p. 78
Defining States of Tests	p. 78
Running Tests	p. 79
Test Results	p. 79
Under the xUnit Covers	p. 81
Test Commands	p. 82
Test Suite Objects	p. 82
xUnit in the Procedural World	p. 82
What's Next?	p. 83
Transient Fixture Management	p. 85
About This Chapter	p. 85
Test Fixture Terminology	p. 86
What Is a Fixture?	p. 86
What Is a Fresh Fixture?	p. 87
What Is a Transient Fresh Fixture?	p. 87
Building Fresh Fixtures	p. 88
In-line Fixture Setup	p. 88
Delegated Fixture Setup	p. 89
Implicit Fixture Setup	p. 91
Hybrid Fixture Setup	p. 93
Tearing Down Transient Fresh Fixtures	p. 93
What's Next?	p. 94
Persistent Fixture Management	p. 95
About This Chapter	p. 95
Managing Persistent Fresh Fixtures	p. 95
What Makes Fixtures Persistent?	p. 95
Issues Caused by Persistent Fresh Fixtures	p. 96
Tearing Down Persistent Fresh Fixtures	p. 97
Avoiding the Need for Teardown	p. 100
Dealing with Slow Tests	p. 102
Managing Shared Fixtures	p. 103
Accessing Shared Fixtures	p. 103
Triggering Shared Fixture Construction	p. 104
What's Next?	p. 106
Result Verification	p. 107
About This Chapter	p. 107
Making Tests Self-Checking	p. 107
Verify State or Behavior?	p. 108
State Verification	p. 109
Using Built-in Assertions	p. 110
Delta Assertions	p. 111
External Result Verification	p. 111
Verifying Behavior	p. 112
Procedural Behavior Verification	p. 113
Expected Behavior Specification	p. 113
Reducing Test Code Duplication	p. 114
Expected Objects	p. 115
Custom Assertions	p. 116
Outcome-Describing Verification Method	p. 117
Parameterized and Data-Driven Tests	p. 118
Avoiding Conditional Test Logic	p. 119
Eliminating "if" Statements	p. 120
Eliminating Loops	p. 121
Other Techniques	p. 121
Working Backward, Outside-In	p. 121
Using Test-Driven Development to Write Test Utility Methods	p. 122
Where to Put Reusable Verification Logic?	p. 122
What's Next?	p. 123
Using Test Doubles	p. 125
About This Chapter	p. 125
What Are Indirect Inputs and Outputs?	p. 125
Why Do We Care about Indirect Inputs?	p. 126
Why Do We Care about Indirect Outputs?	p. 126
How Do We Control Indirect Inputs?	p. 128
How Do We Verify Indirect Outputs?	p. 130
Testing with Doubles	p. 133
Types of Test Doubles	p. 133
Providing the Test Double	p. 140
Configuring the Test Double	p. 141
Installing the Test Double	p. 143
Other Uses of Test Doubles	p. 148
Endoscopic Testing	p. 149
Need-Driven Development	p. 149
Speeding Up Fixture Setup	p. 149
Speeding Up Test Execution	p. 150
Other Considerations	p. 150
What's Next?	p. 151
Organizing Our Tests	p. 153
About This Chapter	p. 153
Basic xUnit Mechanisms	p. 153
Right-Sizing Test Methods	p. 154
Test Methods and Testcase Classes	p. 155
Testcase Class per Class	p. 155
Testcase Class per Feature	p. 156
Testcase Class per Fixture	p. 156
Choosing a Test Method Organization Strategy	p. 158
Test Naming Conventions	p. 158
Organizing Test Suites	p. 160
Running Groups of Tests	p. 160
Running a Single Test	p. 161
Test Code Reuse	p. 162
Test Utility Method Locations	p. 163
TestCase Inheritance and Reuse	p. 163
Test File Organization	p. 164
Built-in Self-Test	p. 164
Test Packages	p. 164
Test Dependencies	p. 165
What's Next?	p. 165
Testing with Databases	p. 167
About This Chapter	p. 167
Testing with Databases	p. 167
Why Test with Databases?	p. 168
Issues with Databases	p. 168
Testing without Databases	p. 169
Testing the Database	p. 171
Testing Stored Procedures	p. 172
Testing the Data Access Layer	p. 172
Ensuring Developer Independence	p. 173
Testing with Databases (Again!)	p. 173
What's Next?	p. 174
A Roadmap to Effective Test Automation	p. 175
About This Chapter	p. 175
Test Automation Difficulty	p. 175
Roadmap to Highly Maintainable Automated Tests	p. 176
Exercise the Happy Path Code	p. 177
Verify Direct Outputs of the Happy Path	p. 178
Verify Alternative Paths	p. 178
Verify Indirect Output Behavior	p. 179
Optimize Test Execution and Maintenance	p. 180
What's Next?	p. 181
The Test Smells	p. 183
Code Smells	p. 185
Obscure Test	p. 186
Conditional Test Logic	p. 200
Hard-to-Test Code	p. 209
Test Code Duplication	p. 213
Test Logic in Production	p. 217
Behavior Smells	p. 223
Assertion Roulette	p. 224
Erratic Test	p. 228
Fragile Test	p. 239
Frequent Debugging	p. 248
Manual Intervention	p. 250
Slow Tests	p. 253
Project Smells	p. 259
Buggy Tests	p. 260
Developers Not Writing Tests	p. 263
High Test Maintenance Cost	p. 265
Production Bugs	p. 268
The Patterns	p. 275
Test Strategy Patterns	p. 277
Recorded Test	p. 278
Scripted Test	p. 285
Data-Driven Test	p. 288
Test Automation Framework	p. 298
Minimal Fixture	p. 302
Standard Fixture	p. 305
Fresh Fixture	p. 311
Shared Fixture	p. 317
Back Door Manipulation	p. 327
Layer Test	p. 337
xUnit Basics Patterns	p. 347
Test Method	p. 348
Four-Phase Test	p. 358
Assertion Method	p. 362
Assertion Message	p. 370
Testcase Class	p. 373
Test Runner	p. 377
Testcase Object	p. 382
Test Suite Object	p. 387
Test Discovery	p. 393
Test Enumeration	p. 399
Test Selection	p. 403
Fixture Setup Patterns	p. 407
In-line Setup	p. 408
Delegated Setup	p. 411
Creation Method	p. 415
Implicit Setup	p. 424
Prebuilt Fixture	p. 429
Suite Fixture Setup	p. 441
Setup Decorator	p. 447
Chained Tests	p. 454
Result Verification Patterns	p. 461
State Verification	p. 462
Behavior Verification	p. 468
Custom Assertion	p. 474
Delta Assertion	p. 485
Guard Assertion	p. 490
Unfinished Test Assertion	p. 494
Fixture Teardown Patterns	p. 499
Garbage-Collected Teardown	p. 500
Automated Teardown	p. 503
In-line Teardown	p. 509
Implicit Teardown	p. 516
Test Double Patterns	p. 521
Test Double	p. 522
Test Stub	p. 529
Test Spy	p. 538
Mock Object	p. 544
Fake Object	p. 551
Configurable Test Double	p. 558
Hard-Coded Test Double	p. 568
Test-Specific Subclass	p. 579
Test Organization Patterns	p. 591
Named Test Suite	p. 592
Test Utility Method	p. 599
Parameterized Test	p. 607
Testcase Class per Class	p. 617
Testcase Class per Feature	p. 624
Testcase Class per Fixture	p. 631
Testcase Superclass	p. 638
Test Helper	p. 643
Database Patterns	p. 649
Database Sandbox	p. 650
Stored Procedure Test	p. 654
Table Truncation Teardown	p. 661
Transaction Rollback Teardown	p. 668
Design-for-Testability Patterns	p. 677
Dependency Injection	p. 678
Dependency Lookup	p. 686
Humble Object	p. 695
Test Hook	p. 709
Value Patterns	p. 713
Literal Value	p. 714
Derived Value	p. 718
Generated Value	p. 723
Dummy Object	p. 728
Appendixes	p. 733
Test Refactorings	p. 735
xUnit Terminology	p. 741
xUnit Family Members	p. 747
Tools	p. 753
Goals and Principles	p. 757
Smells, Aliases, and Causes	p. 761
Patterns, Aliases, and Variations	p. 767
Glossary	p. 785
References	p. 819
Index	p. 835
Table of Contents provided by Ingram. All Rights Reserved.

Excerpts

The Value of Self-Testing Code In Chapter 4 ofRefactoringRef, Martin Fowler writes: If you look at how most programmers spend their time, you'll find that writing code is actually a small fraction. Some time is spent figuring out what ought to be going on, some time is spent designing, but most time is spent debugging. I'm sure every reader can remember long hours of debugging, often long into the night. Every programmer can tell a story of a bug that took a whole day (or more) to find. Fixing the bug is usually pretty quick, but finding it is a nightmare. And then when you do fix a bug, there's always a chance that anther one will appear and that you might not even notice it until much later. Then you spend ages finding that bug. Some software is very difficult to test manually. In these cases, we are often forced into writing test programs. I recall a project I was working on in 1996. My task was to build an event framework that would let client software register for an event and be notified when some other software raised that event (the Observer GOF pattern). I could not think of a way to test this framework without writing some sample client software. I had about 20 different scenarios I needed to test, so I coded up each scenario with the requisite number of observers, events, and event raisers. At first, I logged what was occurring in the console and scanned it manually. This scanning became very tedious very quickly. Being quite lazy, I naturally looked for an easier way to perform this testing. For each test I populated a Dictionary indexed by the expected event and the expected receiver of it with the name of the receiver as the value. When a particular receiver was notified of the event, it looked in the Dictionary for the entry indexed by itself and the event it had just received. If this entry existed, the receiver removed the entry. If it didn't, the receiver added the entry with an error message saying it was an unexpected event notification. After running all the tests, the test program merely looked in the Dictionary and printed out its contents if it was not empty. As a result, running all of my tests had a nearly zero cost. The tests either passed quietly or spewed a list of test failures. I had unwittingly discovered the concept of a Mock Object (page 544) and a Test Automation Framework (page 298) out of necessity! My First XP Project In late 1999, I attended the OOPSLA conference, where I picked up a copy of Kent Beck's new book,eXtreme Programming ExplainedXPE. I was used to doing iterative and incremental development and already believed in the value of automated unit testing, although I had not tried to apply it universally. I had a lot of respect for Kent, whom I had known since the first PLoP 1 conference in 1994. For all these reasons, I decided that it was worth trying to apply eXtreme Programming on a ClearStream Consulting project. Shortly after OOPSLA, I was fortunate to come across a suitable project for trying out this development approach--namely, an add-on application that interacted with an existing database but had no user interface. The client was open to developing software in a different way. We started doing eXtreme Programming "by the book" using pretty much all of the practices it recommended, including pair programming, collective ownership, and test-driven development. Of course, we encountered a few challenges in figuring out how to test some aspects of the behavior of the application, but we still managed to write tests for most of the code. Then, as the project progressed, I started to notice a disturbing trend: It was taking longer and longer to implement seemingly similar tasks. I explained the problem to the developers and asked them to record on each task card how much time had been spent writing new tests, modifying existing tests, and writing the production code. Very quickly,