Here is a programmer's guide to using and programming POSIX threads, commonly known as Pthreads. A "coder's book", this title tells how to use Pthreads in the real world, making efficient and portable applications. Pthreads are an important set of current tools programmers need to have in today's network-intensive climate.

David R. Butenhof, a recognized Pthreads authority, was deeply involved in the creation of the IEEE POSIX standard as well as the X/Open threading extensions, which were fast-tracked into X/Open XSH5 (UNIX98). An engineer at Digital Equipment Corporation, he was the lead architect and developer of Digital's own threading architecture and designed and implemented much of the Pthreads interfaces on Digital UNIX 4.0.

0201633922AB04062001

List of Example Programs

xii

(3)

Preface

xv

Intended audience

xvi

(1)

About the author

xvi

(1)

Acknowledgments

xvii

1 Introduction

1

(34)

1.1 The "bailing programmers"

3

(1)

1.2 Definitions and terminology

4

(4)

1.2.1 Asynchronous

4

(1)

1.2.2 Concurrency

4

(1)

1.2.3 Uniprocessor and multiprocessor

5

(1)

1.2.4 Parallelism

5

(1)

1.2.5 Thread safety and reentrancy

6

(1)

1.2.6 Concurrency control functions

7

(1)

1.3 Asynchronous programming is intuitive

8

(4)

1.3.1 ...because UNIX is asynchronous

9

(2)

1.3.2 ...because the world is asynchronous

11

(1)

1.4 About the examples in this book

12

(1)

1.5 Asynchronous programming, by example

13

(7)

1.5.1 The baseline, synchronous version

14

(1)

1.5.2 A version using multiple processes

15

(2)

1.5.3 A version using multiple threads

17

(2)

1.5.4 Summary

19

(1)

1.6 Benefits of threading

20

(5)

1.6.1 Parallelism

20

(2)

1.6.2 Concurrency

22

(2)

1.6.3 Programming model

24

(1)

1.7 Costs of threading

25

(3)

1.7.1 Computing overhead

26

(1)

1.7.2 Programming discipline

26

(1)

1.7.3 Harder to debug

27

(1)

1.8 To thread or not to thread?

28

(1)

1.9 POSIX thread concepts

29

(6)

1.9.1 Architectural overview

30

(1)

1.9.2 Types and interfaces

30

(1)

1.9.3 Checking for errors

31

(4)

2 Threads

35

(10)

2.1 Creating and using threads

35

(4)

2.2 The life of a thread

39

(6)

2.2.1 Creation

40

(1)

2.2.2 Startup

41

(1)

2.2.3 Running and blocking

42

(1)

2.2.4 Termination

43

(1)

2.2.5 Recycling

44

(1)

3 Synchronization

45

(52)

3.1 Invariants, critical sections, and predicates

45

(2)

3.2 Mutexes

47

(23)

3.2.1 Creating and destroying a mutex

49

(3)

3.2.2 Locking and unlocking a mutex

52

(9)

3.2.2.1 Nonblocking mutex locks

58

(3)

3.2.3 Using mutexes for atomicity

61

(1)

3.2.4 Sizing a mutex to fit the job

62

(1)

3.2.5 Using more than one mutex

63

(7)

3.2.5.1 Lock hierarchy

63

(7)

3.2.5.2 Lock chaining

70

(1)

3.3 Condition variables

70

(18)

3.3.1 Creating and destroying a condition variable

74

(3)

3.3.2 Waiting on a condition variable

77

(4)

3.3.3 Waking condition variable waiters

81

(1)

3.3.4 One final alarm program

82

(6)

3.4 Memory visibility between threads

88

(9)

4 A few ways to use threads

97

(34)

4.1 Pipeline

98

(8)

4.2 Work Crew

106

(14)

4.3 Client/Server

120

(11)

5 Advanced threaded programming

131

(66)

5.1 One-time initialization

131

(3)

5.2 Attributes objects

134

(8)

5.2.1 Mutex attributes

135

(2)

5.2.2 Condition variable attributes

137

(1)

5.2.3 Thread attributes

138

(4)

5.3 Cancellation

142

(19)

5.3.1 Deferred cancelability

147

(3)

5.3.2 Asynchronous cancelability

150

(4)

5.3.3 Cleaning up

154

(7)

5.4 Thread-specific data

161

(11)

5.4.1 Creating thread-specific data

163

(3)

5.4.2 Using thread-specific data

166

(1)

5.4.3 Using destructor functions

167

(5)

5.5 Realtime scheduling

172

(17)

5.5.1 POSIX realtime options

173

(1)

5.5.2 Scheduling policies and priorities

174

(7)

5.5.3 Contention scope and allocation domain

181

(2)

5.5.4 Problems with realtime scheduling

183

(2)

5.5.5 Priority-aware mutexes

185

(4)

5.5.5.1 Priority ceiling mutexes

186

(2)

5.5.5.2 Priority inheritance mutexes

188

(1)

5.6 Threads and kernel entities

189

(8)

5.6.1 Many-to-one (user level)

190

(1)

5.6.2 One-to-one (kernel level)

191

(2)

5.6.3 Many-to-few (two level)

193

(4)

6 POSIX adjusts to threads

197

(44)

6.1 fork

197

(7)

6.1.1 Fork handlers

199

(5)

6.2 exec

204

(1)

6.3 Process exit

204

(1)

6.4 Stdio

204

(5)

6.4.1 flockfile and funlockfile

205

(2)

6.4.2 getchar_unlocked and putchar_unlocked

207

(2)

6.5 Thread-safe functions

209

(5)

6.5.1 User and terminal identification

210

(2)

6.5.2 Directory searching

212

(1)

6.5.3 String token

212

(1)

6.5.4 Time representation

212

(1)

6.5.5 Random number generation

213

(1)

6.5.6 Group and user database

213

(1)

6.6 Signals

214

(27)

6.6.1 Signal actions

215

(1)

6.6.2 Signal masks

216

(1)

6.6.3 pthread_kill

217

(10)

6.6.4 sigwait and sigwaitinfo

227

(3)

6.6.5 SIGEV_THREAD

230

(4)

6.6.6 Semaphores: synchronizing with a signal-catching function

234

(7)

7 "Real code"

241

(48)

7.1 Extended synchronization

241

(29)

7.1.1 Barriers

242

(11)

7.1.2 Read-write locks

253

(17)

7.2 Work queue manager

270

(13)

7.3 But what about existing libraries?

283

(6)

7.3.1 Modifying libraries to be thread-safe

284

(1)

7.3.2 Living with legacy libraries

285

(4)

8 Hints to avoid debugging

289

(18)

8.1 Avoiding incorrect code

290

(12)

8.1.1 Avoid relying on "thread inertia"

291

(2)

8.1.2 Never bet your mortgage on a thread race

293

(4)

8.1.3 Cooperate to avoid deadlocks

297

(2)

8.1.4 Beware of priority inversion

299

(1)

8.1.5 Never share condition variables between predicates

300

(1)

8.1.6 Sharing stacks and related memory corrupters

301

(1)

8.2 Avoiding performance problems

302

(5)

8.2.1 Beware of concurrent serialization

302

(1)

8.2.2 Use the right number of mutexes

303

(1)

8.2.2.1 Too many mutexes will not help

304

(1)

8.2.3 Never fight over cache lines

304

(3)

9 POSIX threads mini-reference

307

(40)

9.1 POSIX 1003.1c-1995 options

307

(1)

9.2 POSIX 1003.1c-1995 limits

308

(1)

9.3 POSIX 1003.1c-1995 interfaces

309

(38)

9.3.1 Error detection and reporting

310

(1)

9.3.2 Use of void* type

311

(1)

9.3.3 Threads

311

(5)

9.3.4 Mutexes

316

(3)

9.3.5 Condition variables

319

(4)

9.3.6 Cancellation

323

(2)

9.3.7 Thread-specific data

325

(1)

9.3.8 Realtime scheduling

326

(10)

9.3.9 Fork handlers

336

(1)

9.3.10 Stdio

336

(2)

9.3.11 Thread-safe functions

338

(4)

9.3.12 Signals

342

(3)

9.3.13 Semaphores

345

(2)

10 Future standardization

347

(16)

10.1 X/Open XSH5 [UNIX98]

347

(9)

10.1.1 POSIX options for XSH5

348

(1)

10.1.2 Mutex type

349

(2)

10.1.3 Set concurrency level

351

(2)

10.1.4 Stack guard size

353

(1)

10.1.5 Parallel I/O

354

(1)

10.1.6 Cancellation points

355

(1)

10.2 POSIX 1003.1j

356

(5)

10.2.1 Barriers

358

(1)

10.2.2 Read-write locks

358

(1)

10.2.3 Spinlocks

359

(1)

10.2.4 Condition variable wait clock

359

(2)

10.2.5 Thread abort

361

(1)

10.3 POSIX 1003.14

361

(2)

Bibliography

363

(4)

Thread resources on the Internet

367

(2)

Index

369

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

The White Rabbit put on his spectacles, "Where shall I begin, please your Majesty?" he asked. "Begin at the beginning," the King said, very gravely, "and go on till you come to the end: then stop." *** - Lewis Carroll, Alice's Adventures in WonderlandThis book is about "threads" and how to use them. "Thread" is just a name for a basic software "thing" that can do work on a computer. A thread is smaller, faster, and more maneuverable than a traditional process. In fact, once threads have been added to an operating system, a "process" becomes just data--address space, files, and so forth--plus one or more threads that do something with all that data.With threads, you can build applications that utilize system resources more efficiently, that are more friendly to users, that run blazingly fast on multiprocessors, and that may even be easier to maintain. To accomplish all this, you need only add some relatively simple function calls to your code, adjust to a new way of thinking about programming, and leap over a few yawning chasms. Reading this book carefully will, I hope, help you to accomplish all that without losing your sense of humor.The threads model used in this book is commonly called "Pthreads," or "POSIX threads." Or, more formally (since you haven't yet been properly introduced), the POSIX 1003.1cn1995 standard. I'll give you a few other names later - but for now, "Pthreads" is all you need to worry about.As I write this, Sun's Solaris, Digital's Digital UNIX, and SGI's IRIX already support Pthreads. The other major commercial UNIX operating systems will soon have Pthreads as well, maybe even by the time you read this, including IBM's AIX and Hewlett-Packard's HP-UX. Pthreads implementations are also available for Linux and other UNIX operating systems.In the personal computer market, Microsoft's Win32 API (the primary programming interface to both Windows NT and Windows 95) supports threaded programming, as does IBM's OS/2. These threaded programming models are quite different from Pthreads, but the important first step toward using them productively is understanding concurrency, synchronization, and scheduling. The rest is (more or less) a matter of syntax and style, and an experienced thread programmer can adapt to any of these models.The threaded model can be (and has been) applied with great success to a wide range of programming problems. Here are just a few: Large scale, computationally intensive programs High-performance application programs and library code that can take advantage of multiprocessor systems Library code that can be used by threaded application programs Realtime application programs and library code Application programs and library code that perform I/O to slow external devices (such as networks and human beings). Intended audienceThis book assumes that you are an experienced programmer, familiar with developing code for an operating system in "the UNIX family" using the ANSI C language. I have tried not to assume that you have any experience with threads or other forms of asynchronous programming. The Introduction chapter provides a general overview of the terms and concepts you'll need for the rest of the book. If you don't want to read the Introduction first, that's fine, but if you ever feel like you're "missing something" you might try skipping back to get introduced.Along the way you'll find examples and simple analogies for everything. In the end I hope that you'll be able to continue comfortably threading along on your own. Have fun, and "happy threading." About the authorI have been involved in the Pthreads standard since it began, although I stayed at home for the first few meetings. I was finally forced to spend a grueling week in the avalanche-proof concrete bunker at the base of Snowbird