Practical Go Building Scalable Network and Non-Network Applications

by Saha, Amit

• ISBN13: 9781119773818
• ISBN10: 1119773814
• eBook ISBN(s): 9781119773832
• Edition: 1st
• Format: Paperback
• Copyright: 2021-12-21
• Publisher: Wiley
• More Book Details
• Purchase Benefits

Summary

YOUR PRACTICAL, HANDS-ON GUIDE TO WRITING APPLICATIONS USING GO

Google announced the Go programming language to the public in 2009, with the version 1.0 release announced in 2012. Since its announcement to the community, and the compatibility promise of the 1.0 release, the Go language has been used to write scalable and high-impact software programs ranging from command-line applications and critical infrastructure tools to large-scale distributed systems. It’s speed, simplicity, and reliability make it a perfect choice for developers working in various domains.

In Practical Go - Building Scalable Network + Non-Network Applications, you will learn to use the Go programming language to build robust, production-ready software applications. You will learn just enough to building command line tools and applications communicating over HTTP and gRPC.

This practical guide will cover:

• Writing command line applications
• Writing a HTTP services and clients
• Writing RPC services and clients using gRPC
• Writing middleware for network clients and servers
• Storing data in cloud object stores and SQL databases
• Testing your applications using idiomatic techniques
• Adding observability to your applications
• Managing configuration data from your applications

You will learn to implement best practices using hands-on examples written with modern practices in mind. With its focus on using the standard library packages as far as possible, Practical Go will give you a solid foundation for developing large applications using Go leveraging the best of the language’s ecosystem.

Author Biography

AMIT SAHA is a software engineer at Atlassian, located in Sydney, Australia. He has written Doing Math with Python: Use Programming to Explore Algebra, Statistics, Calculus, and More! (No Starch Press, 2015) and Write Your First Program (PHI Learning, 2013). His other writings have been published in technical magazines, conference proceedings, and research journals. He can be found online at https://echorand.me.

Table of Contents

Introduction xvii

Getting Started xxi

Chapter 1 Writing Command-Line Applications 1

Your First Application 1

Writing Unit Tests 8

Using the Flag Package 14

Testing the Parsing Logic 20

Improving the User Interface 22

Removing Duplicate Error Messages 23

Customizing Usage Message 24

Accept Name via a Positional Argument 25

Updating the Unit Tests 28

Summary 32

Chapter 2 Advanced Command-Line Applications 33

Implementing Sub-commands 33

An Architecture for Sub-command-Driven Applications 37

Testing the Main Package 43

Testing the Cmd Package 45

Making Your Applications Robust 47

User Input with Deadlines 48

Handling User Signals 52

Summary 56

Chapter 3 Writing HTTP Clients 57

Downloading Data 57

Testing the Data Downloader 59

Deserializing Received Data 61

Sending Data 66

Working with Binary Data 72

Summary 80

Chapter 4 Advanced HTTP Clients 81

Using a Custom HTTP Client 81

Downloading from an Overloaded Server 81

Testing the Time-Out Behavior 85

Configuring the Redirect Behavior 88

Customizing Your Requests 91

Implementing Client Middleware 92

Understanding the RoundTripper Interface 93

A Logging Middleware 94

Add a Header to All Requests 96

Connection Pooling 99

Configuring the Connection Pool 103

Summary 104

Chapter 5 Building HTTP Servers 105

Your First HTTP Server 105

Setting Up Request Handlers 108

Handler Functions 109

Testing Your Server 112

The Request Struct 114

Method 115

URL 115

Proto, ProtoMajor, and ProtoMinor 116

Header 116

Host 116

Body 116

Form, PostForm 116

MultipartForm 117

Attaching Metadata to a Request 118

Processing Streaming Requests 121

Streaming Data as Responses 126

Summary 132

Chapter 6 Advanced HTTP Server Applications 133

The Handler Type 133

Sharing Data across Handler Functions 134

Writing Server Middleware 139

Custom HTTP Handler Technique 139

The HandlerFunc Technique 140

Chaining Middleware 142

Writing Tests for Complex Server Applications 147

Code Organization 147

Testing the Handler Functions 153

Testing the Middleware 155

Testing the Server Startup 157

Summary 159

Chapter 7 Production-Ready HTTP Servers 161

Aborting Request Handling 161

Strategies to Abort Request Processing 165

Handling Client Disconnects 169

Server-Wide Time-Outs 173

Implement a Time-Out for All Handler Functions 173

Implementing Server Time-Out 174

Implementing Graceful Shutdown 179

Securing Communication with TLS 184

Configuring TLS and HTTP/2 184

Testing TLS Servers 188

Summary 192

Chapter 8 Building RPC Applications with gRPC 193

gRPC and Protocol Buffers 193

Writing Your First Service 197

Writing the Server 198

Writing a Client 203

Testing the Server 207

Testing the Client 211

A Detour into Protobuf Messages 214

Marshalling and Unmarshalling 214

Forward and Backward Compatibility 219

Multiple Services 220

Error Handling 226

Summary 228

Chapter 9 Advanced gRPC Applications 229

Streaming Communication 229

Server-Side

Streaming 230

Client-Side

Streaming 237

Bidirectional Streaming 239

Receiving and Sending Arbitrary Bytes 247

Implementing Middleware Using Interceptors 256

Client-Side Interceptors 257

Server-Side Interceptors 263

Wrapping Streams 269

Chaining Interceptors 271

Summary 272

Chapter 10 Production-Ready gRPC Applications 275

Securing Communication with TLS 275

Robustness in Servers 278

Implementing Health Checks 278

Handling Runtime Errors 286

Aborting Request Processing 289

Robustness in Clients 297

Improving Connection Setup 298

Handling Transient Failures 300

Setting Time-Outs for Method Calls 305

Connection Management 306

Summary 309

Chapter 11 Working with Data Stores 311

Working with Object Stores 312

Integration with Package Server 313

Testing Package Uploads 323

Accessing Underlying Driver Types 325

Working with Relational Databases 327

Integration with Package Server 328

Testing Data Storage 339

Data Type Conversions 343

Using Database Transactions 346

Summary 348

Appendix A Making Your Applications Observable 349

Logs, Metrics, and Traces 349

Emitting Telemetry Data 352

Command-Line Applications 352

HTTP Applications 360

gRPC Applications 364

Summary 366

Appendix B Deploying Applications 367

Managing Configuration 367

Distributing Your Application 370

Deploying Server Applications 372

Summary 373

Index 375

 xiii

Contents

Introduction xvii

Getting Started xxi

Chapter 1 Writing Command-Line Applications 1

Your First Application 1

Writing Unit Tests 8

Using the Flag Package 14

Testing the Parsing Logic 20

Improving the User Interface 22

Removing Duplicate Error Messages 23

Customizing Usage Message 24

Accept Name via a Positional Argument 25

Updating the Unit Tests 28

Summary 32

Chapter 2 Advanced Command-Line Applications 33

Implementing Sub-commands 33

An Architecture for Sub-command-Driven Applications 37

Testing the Main Package 43

Testing the Cmd Package 45

Making Your Applications Robust 47

User Input with Deadlines 48

Handling User Signals 52

Summary 56

Chapter 3 Writing HTTP Clients 57

Downloading Data 57

Testing the Data Downloader 59

Deserializing Received Data 61

Sending Data 66

COPYRIGHTED MATERIAL

xiv Contents

Working with Binary Data 72

Summary 80

Chapter 4 Advanced HTTP Clients 81

Using a Custom HTTP Client 81

Downloading from an Overloaded Server 81

Testing the Time-Out Behavior 85

Configuring the Redirect Behavior 88

Customizing Your Requests 91

Implementing Client Middleware 92

Understanding the RoundTripper Interface 93

A Logging Middleware 94

Add a Header to All Requests 96

Connection Pooling 99

Configuring the Connection Pool 103

Summary 104

Chapter 5 Building HTTP Servers 105

Your First HTTP Server 105

Setting Up Request Handlers 108

Handler Functions 109

Testing Your Server 112

The Request Struct 114

Method 115

URL 115

Proto, ProtoMajor, and ProtoMinor 116

Header 116

Host 116

Body 116

Form, PostForm 116

MultipartForm 117

Attaching Metadata to a Request 118

Processing Streaming Requests 121

Streaming Data as Responses 126

Summary 132

Chapter 6 Advanced HTTP Server Applications 133

The Handler Type 133

Sharing Data across Handler Functions 134

Writing Server Middleware 139

Custom HTTP Handler Technique 139

The HandlerFunc Technique 140

Chaining Middleware 142

Writing Tests for Complex Server Applications 147

Code Organization 147

Testing the Handler Functions 153

Testing the Middleware 155

Testing the Server Startup 157

Summary 159

Contents xv

Chapter 7 Production-Ready

HTTP Servers 161

Aborting Request Handling 161

Strategies to Abort Request Processing 165

Handling Client Disconnects 169

Server-Wide Time-Outs

173

Implement a Time-Out

for All Handler Functions 173

Implementing Server Time-Out

174

Implementing Graceful Shutdown 179

Securing Communication with TLS 184

Configuring TLS and HTTP/2 184

Testing TLS Servers 188

Summary 192

Chapter 8 Building RPC Applications with gRPC 193

gRPC and Protocol Buffers 193

Writing Your First Service 197

Writing the Server 198

Writing a Client 203

Testing the Server 207

Testing the Client 211

A Detour into Protobuf Messages 214

Marshalling and Unmarshalling 214

Forward and Backward Compatibility 219

Multiple Services 220

Error Handling 226

Summary 228

Chapter 9 Advanced gRPC Applications 229

Streaming Communication 229

Server-Side

Streaming 230

Client-Side

Streaming 237

Bidirectional Streaming 239

Receiving and Sending Arbitrary Bytes 247

Implementing Middleware Using Interceptors 256

Client-Side

Interceptors 257

Server-Side

Interceptors 263

Wrapping Streams 269

Chaining Interceptors 271

Summary 272

Chapter 10 Production-Ready

gRPC Applications 275

Securing Communication with TLS 275

Robustness in Servers 278

Implementing Health Checks 278

Handling Runtime Errors 286

Aborting Request Processing 289

xvi Contents

Robustness in Clients 297

Improving Connection Setup 298

Handling Transient Failures 300

Setting Time-Outs

for Method Calls 305

Connection Management 306

Summary 309

Chapter 11 Working with Data Stores 311

Working with Object Stores 312

Integration with Package Server 313

Testing Package Uploads 323

Accessing Underlying Driver Types 325

Working with Relational Databases 327

Integration with Package Server 328

Testing Data Storage 339

Data Type Conversions 343

Using Database Transactions 346

Summary 348

Appendix A Making Your Applications Observable 349

Logs, Metrics, and Traces 349

Emitting Telemetry Data 352

Command-Line

Applications 352

HTTP Applications 360

gRPC Applications 364

Summary 366

Appendix B Deploying Applications 367

Managing Configuration 367

Distributing Your Application 370

Deploying Server Applications 372

Summary 373

Index 375

NOTE A glossary of relevant terms is available for free download from the book’s

web page: https://www.wiley.com/go/practicalgo.xiii

Contents

Introduction xvii

Getting Started xxi

Chapter 1 Writing Command-Line Applications 1

Your First Application 1

Writing Unit Tests 8

Using the Flag Package 14

Testing the Parsing Logic 20

Improving the User Interface 22

Removing Duplicate Error Messages 23

Customizing Usage Message 24

Accept Name via a Positional Argument 25

Updating the Unit Tests 28

Summary 32

Chapter 2 Advanced Command-Line Applications 33

Implementing Sub-commands 33

An Architecture for Sub-command-Driven Applications 37

Testing the Main Package 43

Testing the Cmd Package 45

Making Your Applications Robust 47

User Input with Deadlines 48

Handling User Signals 52

Summary 56

Chapter 3 Writing HTTP Clients 57

Downloading Data 57

Testing the Data Downloader 59

Deserializing Received Data 61

Sending Data 66

COPYRIGHTED MATERIAL

xiv Contents

Working with Binary Data 72

Summary 80

Chapter 4 Advanced HTTP Clients 81

Using a Custom HTTP Client 81

Downloading from an Overloaded Server 81

Testing the Time-Out Behavior 85

Configuring the Redirect Behavior 88

Customizing Your Requests 91

Implementing Client Middleware 92

Understanding the RoundTripper Interface 93

A Logging Middleware 94

Add a Header to All Requests 96

Connection Pooling 99

Configuring the Connection Pool 103

Summary 104

Chapter 5 Building HTTP Servers 105

Your First HTTP Server 105

Setting Up Request Handlers 108

Handler Functions 109

Testing Your Server 112

The Request Struct 114

Method 115

URL 115

Proto, ProtoMajor, and ProtoMinor 116

Header 116

Host 116

Body 116

Form, PostForm 116

MultipartForm 117

Attaching Metadata to a Request 118

Processing Streaming Requests 121

Streaming Data as Responses 126

Summary 132

Chapter 6 Advanced HTTP Server Applications 133

The Handler Type 133

Sharing Data across Handler Functions 134

Writing Server Middleware 139

Custom HTTP Handler Technique 139

The HandlerFunc Technique 140

Chaining Middleware 142

Writing Tests for Complex Server Applications 147

Code Organization 147

Testing the Handler Functions 153

Testing the Middleware 155

Testing the Server Startup 157

Summary 159

Contents xv

Chapter 7 Production-Ready

HTTP Servers 161

Aborting Request Handling 161

Strategies to Abort Request Processing 165

Handling Client Disconnects 169

Server-Wide Time-Outs

173

Implement a Time-Out

for All Handler Functions 173

Implementing Server Time-Out

174

Implementing Graceful Shutdown 179

Securing Communication with TLS 184

Configuring TLS and HTTP/2 184

Testing TLS Servers 188

Summary 192

Chapter 8 Building RPC Applications with gRPC 193

gRPC and Protocol Buffers 193

Writing Your First Service 197

Writing the Server 198

Writing a Client 203

Testing the Server 207

Testing the Client 211

A Detour into Protobuf Messages 214

Marshalling and Unmarshalling 214

Forward and Backward Compatibility 219

Multiple Services 220

Error Handling 226

Summary 228

Chapter 9 Advanced gRPC Applications 229

Streaming Communication 229

Server-Side

Streaming 230

Client-Side

Streaming 237

Bidirectional Streaming 239

Receiving and Sending Arbitrary Bytes 247

Implementing Middleware Using Interceptors 256

Client-Side

Interceptors 257

Server-Side

Interceptors 263

Wrapping Streams 269

Chaining Interceptors 271

Summary 272

Chapter 10 Production-Ready

gRPC Applications 275

Securing Communication with TLS 275

Robustness in Servers 278

Implementing Health Checks 278

Handling Runtime Errors 286

Aborting Request Processing 289

xvi Contents

Robustness in Clients 297

Improving Connection Setup 298

Handling Transient Failures 300

Setting Time-Outs

for Method Calls 305

Connection Management 306

Summary 309

Chapter 11 Working with Data Stores 311

Working with Object Stores 312

Integration with Package Server 313

Testing Package Uploads 323

Accessing Underlying Driver Types 325

Working with Relational Databases 327

Integration with Package Server 328

Testing Data Storage 339

Data Type Conversions 343

Using Database Transactions 346

Summary 348

Appendix A Making Your Applications Observable 349

Logs, Metrics, and Traces 349

Emitting Telemetry Data 352

Command-Line

Applications 352

HTTP Applications 360

gRPC Applications 364

Summary 366

Appendix B Deploying Applications 367

Managing Configuration 367

Distributing Your Application 370

Deploying Server Applications 372

Summary 373

Index 375

NOTE A glossary of relevant terms is available for free download from the book’s

web page: https://www.wiley.com/go/practicalgo.