The practical guide to implementing configuration management (CM) best practices that support consistently successful technology development bull; bull;Shows how to use CM to meet business objectives, contractual requirements, and compliance rules. bull;Covers all six CM pillars: source code management, build engineering, environment configuration, change control, release engineering, and deployment. bull;Helps IT pros avoid costly pitfalls, and implement pragmatic, lean processes. bull;For both agile and waterfall organizations. As IT systems become ever more complex and mission-critical, IT professionals must focus more and more on the discipline of Configuration Management (CM). By implementing CM best practices, IT organizations can systematically manage change, avoiding unexpected or unwanted problems introduced by changes to hardware, software, or networks. While increasingly popular frameworks such as ITIL offer guidance on CM, they can be very difficult to understand and implement. Moreover, few IT practitioners have the knowledge they need to succeed with CM, whether they use formal frameworks or not. In this concise, realistic book, two leading experts introduce today's best CM practices, and show how to implement them in any development organization. Bob Aiello and Leslie Sachs cover all six CM 'pillars': source code management, build engineering, environment configuration, change control, release engineering, and deployment. They show how to implement CM in ways that: bull; bull;Support software and systems development. bull;Meet compliance rules such as SOX and SAS-70. bull;Anticipate emerging standards such as IEEE/ISO 12207. bull;Improve both quality and productivity through 'just in time' process improvement. bull;Truly reflect the real-world constraints faced in large development projects.

Bob Aiello is the editor-in-chief for CM Crossroads and a consultant specializing in software process improvement, including software configuration and release management. Mr. Aiello has more than 25 years of experience as a technical manager in several top NYC financial services firms where he had companywide responsibility for CM, often providing hands-on technical support for enterprise source code management tools, SOX/Cobit compliance, build engineering, continuous integration, and automated application deployment. Mr. Aiello is the vice chair of the IEEE 828 Standards working group (CM Planning) and is a member of the IEEE Software and Systems Engineering Standards Committee (S2ESC) management board. He is a longstanding member of the steering committee of the NYC Software Process Improvement Network (CitySPIN), where he has served as the chair of the CM SIG. Mr. Aiello holds a master’s degree in industrial psychology from NYU and a bachelor’s degree in computer science and math from Hofstra University.

Leslie Sachs is the COO of Yellow Spider, Inc., which specializes in providing CM-related consulting services that are aligned with the practices described in this book. Ms. Sachs also writes about applying personality to technology endeavors in her column titled Personality Matters. A New York State Certified School Psychologist with more than 20 years of experience, Ms. Sachs has worked in a variety of clinical and business settings where she has provided many effective interventions designed to improve the social and educational functioning of both individuals and groups. Ms. Sachs has a Masters of Science degree in school and community psychology from Pace University and interned in Bellevue Hospital’s famed Psychiatric Center in NYC. A firm believer in the uniqueness of every individual, she has recently done advanced training with Mel Levine’s All Kinds of Minds Institute.

Preface xxi

Introduction xxxiii

PART I THE CORE CM BEST PRACTICES FRAMEWORK 1

Chapter 1 Source Code Management 3

Terminology and Source Code Management 5

Goals of Source Code Management 5

Principles of Source Code Management 6

1.1 Why Is Source Code Management Important? 6

1.2 Where Do I Start? 7

1.3 Source Code Management Core Concepts 9

1.3.1 Creating Baselines and Time Machines 9

1.3.2 Reserved Versus Unreserved Checkouts 10

1.3.3 Sandboxes and Workspaces 11

1.3.4 Variant Management (Branching) 11

1.3.5 Copybranches Versus Deltas 12

1.3.6 How to Handle Bugfixes 12

1.3.7 Streams 14

1.3.8 Merging 15

1.3.9 Changesets 16

1.4 Defect and Requirements Tracking 16

1.5 Managing the Globally Distributed Development Team 17

1.6 Tools Selection 19

1.6.1 Open Source Versus Commercial 21

1.6.2 Product Maturity and Vendor Commitment 21

1.6.3 Extensibility and Open API 22

1.6.4 Don’t Overengineer Your Source Code Management 22

1.7 Recognizing the Cost of Quality (and Total Cost of Ownership) 23

1.7.1 Building Your Source Code Management Budget 24

1.8 Training 24

1.8.1 The “Bob Method” for Training 24

1.9 Defining the Usage Model 25

1.10 Time to Implement and Risks to Success 26

1.11 Establishing Your Support Process 26

1.12 Advanced Features and Empowering Users 27

Conclusion 27

Chapter 2 Build Engineering 29

Goals of Build Engineering 30

Principles of Build Engineering 30

2.1 Why Is Build Engineering Important? 31

2.2 Where Do I Start? 32

2.3 Build Engineering Core Concepts 32

2.3.1 Version IDs or Branding Your Executables 32

2.3.2 Immutable Version IDs 33

2.3.3 Stamping In a Version Label or Tag 33

2.3.4 Managing Compile Dependencies 33

2.3.5 The Independent Build 34

2.4 Core Considerations for Scaling the Build Function 34

2.4.1 Selling the Independent Build 35

2.4.2 Overengineering the Build 35

2.4.3 Testing Your Own Integrity 36

2.4.4 Reporting to Development Can Be a Conflict of Interest 37

2.4.5 Organizational Choices 37

2.5 Build Tools Evaluation and Selection 38

2.5.1 Apache Ant Enters the Build Scene 38

2.5.2 Of Mavens and Other Experts 38

2.5.3 Maven Versus Ant 39

2.5.4 Using Ant for Complex Builds 39

2.5.5 Continuous Integration 40

2.5.6 CI Servers 40

2.5.7 Integrated Development Environments 40

2.5.8 Static Code Analysis 41

2.5.9 Build Frameworks 41

2.5.10 Selecting Your Build Tools 41

2.5.11 Conducting the Bakeoff and Reaching Consensus 42

2.6 Cost of Quality and Training 42

2.7 Making a Good Build Better 42

2.7.1 “Bob-Proofing” Your Build 43

2.7.2 Test-Driven Builds 43

2.7.3 Trust, But Verify 43

2.7.4 The Cockpit of a Plane 44

2.8 The Role of the Build Engineer 44

2.8.1 Know What You Build 45

2.8.2 Partner with Developers 46

2.8.3 Drafting a Rookie 46

2.9 Architecture Is Fundamental 46

2.10 Establishing a Build Process 47

2.10.1 Establishing Organizational Standards 47

2.11 Continuous Integration Versus the Nightly Build 47

2.12 The Future of Build Engineering 48

Conclusion 48

Chapter 3 Environment Configuration 49

Goals of Environment Configuration Control 50

Principles of Environment Configuration Control 51

3.1 Why Is Environment Configuration Important? 51

3.2 Where Do I Start? 51

3.3 Supporting Code Promotion 52

3.4 Managing the Configuration 52

3.4.1 Which Database Are You Using? 53

3.4.2 Did That Trade Go Through? 53

3.4.3 How About a Few Tokens? 54

3.4.4 Centralizing the Environment Variable Assignment 55

3.5 Practical Approaches to Establishing a CMDB 55

3.5.1 Identify and Then Control 56

3.5.2 Understanding the Environment Configuration 56

3.6 Change Control Depends on Environment Configuration 56

3.7 Minimize the Number of Controls Required 57

3.8 Managing Environments 57

3.9 The Future of Environment Configuration 57

Conclusion 58

Chapter 4 Change Control 59

Goals of Change Control 60

Principles of Change Control 60

4.1 Why Is Change Control Important? 61

4.2 Where Do I Start? 61

4.3 The Seven Types of Change Control 61

4.3.1 A Priori 62

4.3.2 Gatekeeping 62

4.3.3 Configuration Control 62

4.3.4 Change Advisory Board 63

4.3.5 Emergency Change Control 64

4.3.6 Process Engineering 64

4.3.7 Senior Management Oversight 64

4.4 Creating a Change Control Function 65

4.5 Examples of Change Control in Action 65

4.5.1 The 29-Minute Change Control Meeting 66

4.5.2 Change Control at the Investment Bank 66

4.5.3 Change Control at the Trading Firm 67

4.5.4 Forging Approvals 69

4.6 Don’t Forget the Risk 69

4.7 Driving the CM Process Through Change Control 69

4.8 Entry/Exit Criteria 70

4.9 After-Action Review 71

4.10 Make Sure That You Evaluate Yourself 71

Conclusion 71

Chapter 5 Release Management 73

Goals of Release Management 74

Principles of Release Management 74

5.1 Why Is Release Management Important? 75

5.2 Where Do I Start? 75

5.3 Release Management Concepts and Practices 76

5.3.1 Packaging Strategies That Work 76

5.3.2 Package Version Identification 76

5.3.3 Sending a Release Map with the Release 77

5.3.4 What Does Immutable Mean? 77

5.4 The Ergonomics of Release Management 77

5.4.1 Avoiding Human Error 78

5.4.2 Understanding the Technology 78

5.4.3 Tools from Build Engineering 79

5.4.4 Avoiding Human Error 79

5.4.5 My Own Three-Step Process 79

5.4.6 Too Many Moving Parts 80

5.5 Release Management as Coordination 80

5.5.1 Communicating the Status of a Release 80

5.5.2 Don’t Forget the Release Calendar 80

5.5.3 RM and Configuration Control 81

5.6 Requirements Tracking 81

5.7 Taking Release Management to the Next Level 81

5.7.1 Using Cryptography to Sign Your Code 82

5.7.2 Operating Systems Support for Release Management 82

5.7.3 Improving Your RM Process 2

Conclusion 83

Chapter 6 Deployment 85

Goals of Deployment 86

Principles of Deployment 86

6.1 Why Is Deployment Important? 87

6.2 Where Do I Start? 87

6.3 Practices and Examples 87

6.3.1 Staging Is Key 87

6.3.2 Scripting the Release Process Itself 89

6.3.3 Frameworks for Deployment 89

6.3.4 What If Bob Makes a Mistake? 89

6.3.5 More on the Depot 90

6.3.6 Auditing Your Release 90

6.4 Conducting a Configuration Audit 91

6.5 Don’t Forget the Smoke Test 92

6.6 Little Things Matter a Lot 92

6.7 Communications Planning 92

6.7.1 Announcing Outages and Completed Deployments 93

6.8 Deployment Should Be Delegated 93

6.9 Trust But Verify 93

6.10 Improving the Deployment Process 93

Conclusion 94

PART II ARCHITECTURE AND HARDWARE CM 95

Chapter 7 Architecting Your Application for CM 97

Goals of Architecting Your Application for CM 98

7.1 Why Is Architecture Important? 99

7.2 Where Do I Start? 99

7.3 How CM Facilitates Good Architecture 99

7.4 What Architects Can Learn From Testers 99

7.4.1 Testing as a Service to the Developers 100

7.5 Configuration Management—Driven Development (CMDD) 101

7.6 Coping with the Changing Architecture 101

7.7 Using Source Code Management to Facilitate Architecture 102

7.8 Training Is Essential 102

7.9 Source Code Management as a Service 103

7.10 Build Engineering as a Service 103

Conclusion 103

Chapter 8 Hardware Configuration Management 105

Goals of Hardware CM 106

8.1 Why Is Hardware CM Important? 106

8.2 Where Do I Start? 107

8.3 When You Can’t Version Control a Circuit Chip 107

8.3.1 A Configuration Item by Any Other Name 107

8.3.2 Version Control for Design Specifications 108

8.4 Don’t Forget the Interfaces 108

8.5 Understanding Dependencies 108

8.6 Traceability 108

8.7 Deploying Changes to the Firmware 109

8.8 The Future of Hardware CM 109

Conclusion 109

PART III THE PEOPLE SIDE OF CM 111

Chapter 9 Rightsizing Your Processes 113

Goals of Rightsizing Your CM Processes 114

9.1 Why Is Rightsizing Your Processes Important? 115

9.2 Where Do I Start? 115

9.3 Verbose Processes Just Get in the Way 116

9.4 SPINs and Promoting the CMM 117

9.5 Disappearing Verbose Processes 117

9.5.1 Agile Processes Just Work 118

9.5.2 Open Unified Process 118

9.5.3 Getting Lean 119

9.5.4 An Extremely Brief Description That I Hope Motivates You to Take a Closer Look at Lean Software Development 119

9.6 The Danger of Having Too Little Process 120

9.7 Just-in-Time Process Improvement 120

9.8 Don’t Overengineer Your CM 120

9.9 Don’t Forget the Technology 121

9.10 Testing Your Own Processes 121

9.11 Process Consultation 122

9.11.1 Transparency That Is Genuine 122

9.12 Create a Structure for Sustainability 122

Conclusion 123

Chapter 10 Overcoming Resistance to Change 125

Goals of Overcoming Resistance to Change 126

10.1 Why Is Overcoming Resistance to Change Important? 127

10.2 Where Do I Start? 127

10.3 Matching Process to Culture 127

10.4 Mixing Psychology and Computer Programming 129

10.5 Process Improvement from Within 129

10.6 Picking Your Battles 131

10.7 Fostering Teamwork 131

10.8 Why Good Developers Oppose Process Improvement 132

10.9 Procedural Justice 132

10.10 Input from Everyone 132

10.11 Showing Leadership 133

10.12 Process Improvement People May Be the Problem 133

10.13 Combining Process and Technology Training 134

10.14 Listening to the Rhythm 135

10.15 Processes Need to Be Tested 136

10.16 Baby Steps and Process Improvement 136

10.17 Selling Process Improvement 137

10.18 What’s in It for Me? 137

10.19 Process Improvement as a Service 137

10.20 Guerrilla Tactics for Process Improvement 138

Conclusion 139

Chapter 11 Personality and CM: A Psychologist Looks at the Workplace 141

Goals of Understanding Personality: What’s in It for Me? 142

11.1 Personality Primer for CM Professionals 144

11.2 What Do CM Experts Need to Consider in Terms of Personality? 146

11.2.1 Communication Styles 147

11.2.2 Do Men and Women Use and Interpret Language Differently? 147

11.2.3 Effective Consultation 148

11.2.4 Verifying the Message 148

11.2.5 Information Processing Preferences 149

11.2.6 Birth Order at Work 150

11.2.7 Firstborns as Leaders 150

11.2.8 The Middle-Born Compromiser 151

11.2.9 The Youngest as Initiator 151

11.2.10 The Only Child 151

11.2.11 Being Yourself 152

11.3 Applying Psychology to the Workplace 152

11.3.1 Effective Teamwork Begins at Home 153

11.3.2 Volleyball or Effective Collaboration 153

11.3.3 Embedding Build Engineers and Testers in the Development Team 153

11.3.4 Blackbox Versus Whitebox Versus Graybox 154

11.3.5 Group Dynamics That Can Damage the Organization 154

11.3.6 Where CM and QA Fit In 154

11.4 Family Dynamics! 155

11.4.1 Indecisiveness 155

11.5 Workplace Culture and Personality 156

11.5.1 Personality and Structure 156

11.5.2 We Already Invented All the Good Ideas 157

11.5.3 Loose Cannons Who Don’t Want to Comply 157

11.5.4 Enforcing Process, While Still Keeping the Train Moving 158

11.5.5 Formulas for Success 158

11.5.6 Caveats 159

Conclusion 159

Chapter 12 Learning From Mistakes That I Have Made 161

Goals of Learning from Mistakes 162

12.1 Why Is It Important to Learn from Our Mistakes? 162

12.2 Where Do I Get Started? 162

12.3 Understanding Our Mistakes 163

12.4 The Mistakes I Have Made 163

12.4.1 Missing the Big Picture 163

12.4.2 Writing Release Automation Can Be Challenging . 164

12.4.3 Thinking That a Good Process Will Carry Itself 165

12.4.4 Failing to Gain Consensus 165

12.4.5 Failing to Show Leadership for CM 165

12.4.6 Becoming Part of the Problem 165

12.4.7 Forgetting to Ask for Help 166

12.5 Turning a Mistake into a Lesson Learned 166

12.5.1 Clarifying What I Need to Get the Job Done 166

12.5.2 Getting the Training That I Need 167

12.6 Common Mistakes That I Have Seen Others Make 167

12.6.1 Ivory Tower 167

12.6.2 Failing to Get Technical and Hands-On 167

12.6.3 Not Being Honest and Open 168

Conclusion 168

PART IV COMPLIANCE, STANDARDS, AND FRAMEWORKS 169

Chapter 13 Establishing IT Controls and Compliance 171

Goals of Establishing IT Controls and Compliance 172

13.1 Why Are IT Controls and Compliance Important? 173

13.2 How Do I Get Started? 173

13.3 Understanding IT Controls and Compliance 174

13.3.1 Sarbanes-Oxley Act of 2002 174

13.3.2 Management Assessment of Internal Controls 174

13.3.3 Committee of Sponsoring Organizations 175

13.3.4 Cobit as a Framework for IT Controls 176

13.3.5 What Does It Mean to Attest to And Report on the Assessment Made by the Management? 176

13.3.6 Health Insurance Portability and Accountability Act of 1996 177

13.3.7 When the GAO Comes Knocking 177

13.3.8 Results of the Audit 178

13.3.9 GAO Reports on NARA’s Configuration Management Practices 179

13.3.10 ERA Configuration Management Plan 179

13.3.11 Areas for Improvement 180

13.3.12 Understanding the Results of the Audit 180

13.3.13 Office of the Comptroller of the Currency 181

13.4 Essential Compliance Requirements 181

13.4.1 Providing Traceability of Requirements to Releases 182

13.4.2 Production Separation of Controls 182

13.5 The Moral Argument for Supporting CM Best Practices 182

13.6 Improving Quality and Productivity Through Compliance 183

13.7 Conducting a CM Assessment 183

13.7.1 Assessment First Steps 184

13.7.2 Listen First Regardless of How Bad the Situation Appears 184

Conclusion 185

Chapter 14 Industry Standards and Frameworks 187

Goals of Using Industry Standards and Frameworks 188

14.1 Why Are Standards and Frameworks Important? 188

14.2 How Do I Get Started? 189

14.3 Terminology Required 189

14.3.1 Configuration Item 189

14.3.2 Configuration Identification 190

14.3.3 Configuration Control 190

14.3.4 Interface Control 190

14.3.5 Configuration Status Accounting 191

14.3.6 Configuration Audit 191

14.3.7 Subcontractor/Vendor Control 192

14.3.8 Conformance Versus Noncompliance 192

14.4 Applying These Terms to the Standards and Frameworks 193

14.5 Industry Standards 193

14.5.1 IEEE 828–Standard for Software Configuration Management Plans 193

14.5.2 ISO 10007–Quality Management Systems–Guidelines for Configuration Management 195

14.5.3 ANSI/ITAA EIA-649-A–National Consensus Standard for Configuration Management 196

14.5.4 ISO/IEC/IEEE 12207 and 15288 196

14.6 Industry Frameworks 196

14.6.1 ISACA Cobit 197

14.6.2 CMM/CMMI 207

14.6.3 itSMF’s ITIL Framework 208

14.6.4 SWEBOK 214

14.6.5 Open Unified Process (OpenUP) 215

14.6.6 Agile/SCRUM 216

Conclusion 217

Index 219

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

Configuration management (CM) plays a critical role in any technology development effort. I have been involved with implementing and supporting CM for over 25 years and much of what I am about to discuss comes directly from my own personal experience. I have implemented and supported each of these CM practices, often with the agreement that I could be woken in the middle of the night if my processes/automation did not work as expected. As an instructor, I have taught industry strength CM tools to over 900+ technology professionals (again with the offer that they got my home phone number upon successfully completing my class). My colleagues and students have consistently indicated that my passion and love for this discipline has always been abundantly clear. It is my view that configuration management consists of six functional areas: Source Code Management Build Engineering Environment Configuration Change Control Release Engineering DeploymentI have searched for, but never found, any single book (or even a series of books) that covered all of these functional areas. Most CM books are either too narrowly focused on one key area (e.g., building code with Ant) or so "ivory tower" that they did not give me enough information on how to really implement these functions in a practical real world environment. It's nice to point out the need to "maintain control of all configuration items", but unless you tell me exactly how to do that in a practical and realistic way, the advice is not truly utilizable. It is my intent both to cast a wide net on the CM practices that you need to understand and also to provide enough detail so that you know not only "what" each CM function entails, but, just as importantly " how" to implement each of the CM functions. I expect that my readers will hold me to that commitment (see the URL of the supporting website below). The Traditional Definition of Configuration ManagementConfiguration management or in this context, software configuration management (SCM) has a traditional definition of consisting of four specific functions. They are: Configuration identification Change control Status accounting Configuration auditThese functions have long been described in industry standards and frameworks and obviously viewed as essential to any valid configuration management effort. While I agree completely that these functions are correct and essential, I find their terminology to be difficult for many technology professionals to understand and appreciate. In this book, I will discuss the traditional CM functions, and I will also suggest a framework for understanding and implementing configuration management in a way that I believe will reflect current industry practices. Specifically, I will show the relationship between the four classic functions and the six functions of source code management, build engineering, environment configuration, change control, release engineering, and deployment that I believe more closely reflect the way that CM is actually done on a day to day basis. This is an important focus of my efforts to make configuration management best practices more approachable and practical for technology professionals to enjoy as part of their own process improvement efforts. Terminology and CMConfiguration Management, like many other disciplines, suffers from the use of confusing terminology. I am not going to solve that problem in this book, but I will endeavor to at least not make the situation worse. The acronym SCM has been used to refer to both Source Code Management and, more recently, Software Configuration Management. One of my most knowledgeable colleagues has prevailed upon me to not make the situation worse, so I will only use the S