How Google Tests Software

by ; ;
  • ISBN13:


  • ISBN10:


  • Edition: 1st
  • Format: Paperback
  • Copyright: 3/23/2012
  • Publisher: Addison-Wesley Professional
  • Purchase Benefits
  • Free Shipping On Orders Over $59!
    Your order must be $59 or more to qualify for free economy shipping. Bulk sales, PO's, Marketplace items, eBooks and apparel do not qualify for this offer.
  • Get Rewarded for Ordering Your Textbooks! Enroll Now
List Price: $39.99 Save up to $6.00
  • Buy New


Supplemental Materials

What is included with this book?

  • The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.
  • The eBook copy of this book is not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.


Seamless delivery of cloud software at scale is a massive challenge, but one company has largely surmounted it: Google. In reaching this goal, a state-of-the-art testing process was of paramount importance. This book offers an unprecedented "under the hood" look at how Google tests its cloud software, introducing never-before-revealed best practices for testing and automation. Test pioneer James Whittaker and two Google testing experts bring together powerful lessons, place them in perspective, and codify them for use by any practitioner. They offer candid insights into what's working for Google - and an insider's skeptical look at processes that aren't yet working optimally. Along the way, they offer readers a solid foundation for making decisions about: * Risk analysis and test planning * Thinking like the user (and knowing which user!) * Exploratory, black box, white box, and acceptance testing * Incorporating community feedback * Tracking issues more effectively * Using off-the-shelf tools, custom tools, spreadsheets, and dashboards * Understanding the lives of software engineers and test engineering managers * Testing early "Docs and Mocks" * Testing interfaces, classes, modules, libraries, binaries, services, and infrastructure * Managing code reviews * Performing small, medium, and large-scale tests * Refactoring * Using test hooks, presubmit scripts and queues, and continuous builds * Deployment and production monitoring and more

Author Biography

James Whittaker: James is an Engineering Director at Google and has been responsible for testing Chrome, maps, and Google web apps. He used to work for Microsoft and was a professor before that. James is one of the best known names in testing. Jason Arbon: Jason is a Test Engineer at Google and has been test lead for Google Desktop, Chrome, and Chrome OS and development lead for an array of open source test tools. He worked at Microsoft prior to joining Google. Jeff Carollo: Jeff is a Software Engineer in Test and has focused on developer tools for code presubmit testing. He has consulted with dozens of internal Google development teams helping them improve initial code quality. He converted to Software Engineer in 2010 and leads development of Google+ APIs.

Table of Contents

Forewordp. xiii
Forewordp. xvii
Prefacep. xxiii
Introduction to Google Software Testingp. 1
Quality≠Testp. 5
Rolesp. 6
Organizational Structurep. 8
Crawl, Walk, Runp. 10
Types of Testsp. 12
The Software Engineer in Testp. 15
The Life of an SETp. 17
Development and Test Workflowp. 17
Who Are These SETs Anyway?p. 22
The Early Phase of a Projectp. 22
Team Structurep. 24
Design Docsp. 25
Interfaces and Protocolsp. 27
Automation Planningp. 28
Testabilityp. 29
SET Workflow: An Examplep. 32
Test Executionp. 40
Test Size Definitionsp. 41
Use of Test Sizes in Shared Infrastructurep. 44
Benefits of Test Sizesp. 46
Test Runtime Requirementsp. 48
Case 1: Change in Common Libraryp. 52
Test Certifiedp. 54
An Interview with the Founders of the Test Certified Programp. 57
Interviewing SETsp. 62
An Interview with Tool Developer Ted Maop. 68
An Interview with Web Driver Creator Simon Stewartp. 70
The Test Engineerp. 75
A User-Facing Test Rolep. 75
The Life of a TEp. 76
Test Planningp. 79
Riskp. 97
Life of a Test Casep. 108
Life of a Bugp. 113
Recruiting TEsp. 127
Test Leadership at Googlep. 134
Maintenance Mode Testingp. 137
Quality Bots Experimentp. 141
BITE Experimentp. 153
Google Test Analyticsp. 163
Free Testing Workflowp. 169
External Vendorsp. 173
An Interview with Google Docs TE Lindsay Websterp. 175
An Interview with YouTube TE Apple Chowp. 181
The Test Engineering Managerp. 187
The Life of a TEMp. 187
Getting Projects and Peoplep. 189
Impactp. 191
An Interview with Gmail TEM Ankit Mehtap. 193
An Interview with Android TEM Hung Dangp. 198
An Interview with Chrome TEM Joel Hynoskip. 202
The Test Engineering Directorp. 206
An Interview with Search and Geo Test Director Shelton Marp. 207
An Interview with Engineering Tools Director Ashish Kumarp. 211
An Interview with Google India Test Director Sujay Sahnip. 214
An Interview with Engineering Manager Brad Greenp. 219
An Interview with James Whittakerp. 222
Improving How Google Tests Softwarep. 229
Fatal Flaws in Google's Processp. 229
The Future of the SETp. 231
The Future of the TEp. 233
The Future of the Test Director and Managerp. 234
The Future of Test Infrastructurep. 234
In Conclusionp. 235
Chrome OS Test Planp. 237
Overview of Themesp. 237
Risk Analysisp. 238
Per-Build Baseline Testingp. 239
Per-LKG Day Testingp. 239
Per-Release Testingp. 239
Manual Versus Automationp. 240
Dev Versus Test Quality Focusp. 240
Release Channelsp. 240
User Inputp. 241
Test Case Repositoriesp. 24l
Test Dashboardingp. 241
Virtualizationp. 241
Performancep. 242
Stress, Long-Running, and Stabilityp. 242
Test Execution Framework (Autotest)p. 242
OEMsp. 242
Hardware Labp. 242
E2E Farm Automationp. 243
Testing the Browser AppManagerp. 243
Browser Testabilityp. 243
Hardwarep. 244
Timelinep. 244
Primary Test Driversp. 246
Relevant Documentsp. 246
Test Tours for Chromep. 247
The Shopping Tourp. 247
The Student Tourp. 248
Suggested Areas to Testp. 248
The International Calling Tourp. 249
Suggested Areas to Testp. 249
The Landmark Tourp. 249
Suggested Landmarks in Chromep. 249
The All Nighter Tourp. 250
Suggested Areas to Testp. 250
The Artisan's Tourp. 251
Tools in Chromep. 251
The Bad Neighborhood Tourp. 251
Bad Neighborhoods in Chrome OSp. 251
The Personalization Tourp. 252
Ways to Customize Chromep. 252
Blog Posts on Tools and Codep. 253
Take a BITE out of Bugs and Redundant Laborp. 253
Unleash the QualityBotsp. 255
RPF: Google's Record Playback Frameworkp. 257
Google Test Analytics-Now in Open Sourcep. 260
Comprehensivep. 260
Quickp. 260
Actionablep. 260
Sustained Valuep. 260
Indexp. 265
Table of Contents provided by Ingram. All Rights Reserved.

Rewards Program

Write a Review