Software Testing with Visual Studio 2010

by Levinson, Jeff

• ISBN13: 9780321734488
• ISBN10: 0321734483
• Edition: 1st
• Format: Paperback
• Copyright: 2011-02-21
• Publisher: Addison-Wesley Professional

Summary

Use Visual Studio 2010rs"s breakthrough testing tools to improve quality throughout the entire software lifecycle Together, Visual Studio 2010 Ultimate, Visual Studio Test Professional 2010, Lab Management 2010, and Team Foundation Server offer Microsoft developers the most sophisticated, well-integrated testing solution theyrs"ve ever had. Now, Microsoft MVP and VS testing guru Jeff Levinson shows exactly how to use Microsoftrs"s new tools to save time, reduce costs, and improve quality throughout the entire development lifecycle. Levinson shows how Microsoftrs"s new tools can help you finally overcome long-standing communication, coordination, and management problems. Yours"ll discover how to perform first-rate functional testing; quickly create and execute tests and record the results with log files and video; and create bugs directly from tests, ensuring reproducibility and eliminating wasted time. Levinson offers in-depth coverage of Microsoftrs"s powerful new testing metrics, helping you ensure traceability all the way from requirements through finished software. Coverage includes: Planning your tests using Microsoft Test Manager (MTM) Creating test settings, structuring test cases, and managing the testing process Executing manual tests with Microsoft Test Manager and Test Runner Filing and resolving bugs, and customizing your bug reporting process Automating test cases and linking automated tests with requirements Executing automated test cases through both Visual Studio and Microsoft Test Manager Integrating automated testing into the build process Using Microsoftrs"s Lab Management virtualization platform to test applications, snapshot environments, and reproduce bugs Implementing detailed metrics for evaluating quality and identifying improvements Whether yours"re a developer, tester, manager, or analyst, this book can help you significantly improve the way you work and the results you deliver - both as an individualright now, and as a team memberthroughout your entire project.

Author Biography

Jeff Levinson has 16 years of experience in software development in many different roles–developer, tester, architect, project manager, and scrum master at several large companies. Jeff is currently a Senior Application Lifecycle Management (ALM) Consultant for Northwest Cadence, which is a company that specializes in Team Foundation Server, Visual Studio, methodologies, and process improvement. In his day-to-day work, Jeff helps teams, organizations, and companies adopt more efficient processes, improve quality, and reduce costs associated with software development. Jeff is a frequent speaker at industry events and writes a twice-monthly column for Visual Studio Magazine Online. This is his fourth book on software development. His other books are Building Client/Server Applications with VB.NET (Apress 2003), Pro Visual Studio Team System 2005 (Apress 2006), and Pro Visual Studio Team System with Team Edition for Database Professionals (Apress 2007). Jeff has a master’s degree in software engineering from Carnegie Mellon University and is an MCP, MCAD, MCSD, MCDBA, and MCT.

 

Jeff currently lives in Washington State with his wife and two children.

 

Table of Contents

Foreword    xvii

Preface    xxi

Acknowledgments    xxix

About the Author    xxxi

1  State of Testing    1

Software Testing Challenges    1

The Need for Testers    3

A Different Approach    5

Fixing Communication    5

Increasing Project Visibility    6

What Are the Tools Designed to Do?    7

Metrics    10

Citations    12

2  Software Quality and Testing Overview    13

Software Quality    13

Requirements    14

Business Value    14

Expectations    15

Nonfunctional Requirements    15

Where Do You Build Quality?    17

Process and Quality    19

Software Testing    19

The Testing Mindset    20

Software Testing Strategies    21

Types of Software Testing    22

Test Management    27

After the Product Is Shipped or Deployed    27

3  Planning Your Testing    29

Microsoft Test Manager    30

Test Plans    36

Properties    38

Contents    43

Adding Suites and Test Cases to Your Plan    46

Testing Configurations    48

Managing Test Configurations    49

Assigning Test Configurations    51

Assigning Testers    53

Test Case Planning Workflow    55

Analysis and Initial Design    56

Construction    61

User Acceptance Testing    62

Common Scenarios    64

Scheduling and Tracking Test Case Creation and Execution    64

Feature Driven Development    65

Moving from One Iteration to Another    67

Handling Different Test Configurations    68

4  Executing Manual Tests    71

Using the Test Case Work Item Type    72

Shared Steps    75

Data Driven Test Cases (Test Parameters)    77

Running Your First Tests    79

Test Runner    80

Examining Test Results    92

Test Run Results    93

Detailed Test Results    95

Exploratory Testing with MTM    101

5  Resolving Bugs    107

A Bug’s Life    107

Customer Reported Bug    110

Test Team Reported Bug    110

Triaging the Bug    110

Reactivations    111

Bug Differences and Modifications    111

The Generated Bug    116

How a Developer Uses IntelliTrace    120

Fixing the Bug    122

Associated Changesets    124

Associated Work Items    124

Impacted Tests    125

Setting the Build Quality    125

Assigning a New Build    127

Verifying That the Bug Is Fixed    129

Dealing with Impacted Tests    131

6  Automating Test Cases    135

To Automate or Not to Automate    136

The Automated Testing Framework    139

Creating an Automated Test from a Manual Test    141

Examining a Generated Web Application Coded UI Test    142

Adding Validations    157

Adding Additional Recorded Steps    164

Parameterized Coded UI Tests    166

Handling Issues Due to Inconsistency    168

Resolving the Data Inconsistency    169

Handling Dynamic Values    172

Other Tips    177

Combining Multiple Tests    178

Associating Coded UI Tests and Test Cases    178

7  Executing Automated Test Cases    183

Executing Automated Tests Through Visual Studio    183

Local Execution    184

Local Execution with Remote Collection    184

Remote Execution    185

Executing Automated Tests from the Command Line    190

Executing Automated Tests in MTM    191

Creating an Automated Build    191

Setting Up the Physical Environment    193

Running a Coded UI Test Through MTM    196

Executing Automated Tests with Team Build    202

Automated Testing Gotchas    205

Custom Dialogs    205

Cleaning Up Your Tests    207

8  Lab Management    209

Managing Virtual Environments Through MTM    210

Finishing Virtual Environment Configuration    217

Automated Test Settings    221

Lab Management Workflow    222

Executing a Lab Build    231

Running Automated Tests Through MTM    233

Manual Tests in a Virtual Environment    234

9  Reporting and Metrics    239

Understanding the Reporting Structure    240

Built-In Reports    242

Bug Status    244

Bug Trends    245

Reactivations    246

Build Quality Indicators    246

Build Success over Time    248

Build Summary    249

Stories Overview    250

Test Case Readiness    251

Test Plan Progress    252

Excel Services Reports (Dashboards)    253

Reporting with Microsoft Excel    254

Creating a Generated Report    255

The Testing Measures    256

Metrics    268

What to Measure    271

First-Time Defect Rate    273

Bug Reactivations    276

General Bug Counts    277

Index       283