9780201633467

Introduction

This book describes the TCP/IP protocol suite, but from a different perspective than other texts on TCP/IP. Instead of just describing the protocols and what they do, we'll use a popular diagnostic tool to watch the protocols in action. Seeing how the protocols operate in varying circumstances provides a greater understanding of how they work and why certain design decisions were made. It also provides a look into the implementation of the protocols, without having to wade through thousands of lines of source code.

When networking protocols were being developed in the 1960s through the 1980s, expensive, dedicated hardware was required to see the packets going "across the wire." Extreme familiarity with the protocols was also required to comprehend the packets displayed by the hardware. Functionality of the hardware analyzers was limited to that built in by the hardware designers.

Today this has changed dramatically with the ability of the ubiquitous workstation to monitor a local area network Mogul 1990. Just attach a workstation to your network, run some publicly available software (described in Appendix A), and watch what goes by on the wire. While many people consider this a tool to be used for diagnosing network problems, it is also a powerful tool for understanding how the network protocols operate, which is the goal of this book.

This book is intended for anyone wishing to understand how the TCP/IP protocols operate: programmers writing network applications, system administrators responsible for maintaining computer systems and networks utilizing TCP/IP, and users who deal with TCP/IP applications on a daily basis.

Organization of the Book

We take a bottom-up approach to the TCP/IP protocol suite. After providing a basic introduction to TCP/IP in Chapter 1, we will start at the link layer in Chapter 2 and work our way up the protocol stack. This provides the required background for later chapters for readers who aren't familiar with TCP/IP or networking in general.

This book also uses a functional approach instead of following a strict bottom-to-top order. For example, Chapter 3 describes the IP layer and the IP header. But there are numerous fields in the IP header that are best described in the context of an application that uses or is affected by a particular field. Fragmentation, for example, is best understood in terms of UDP (Chapter 11), the protocol often affected by it. The time-to-live field is fully described when we look at the Traceroute program in Chapter 8, because this field is the basis for the operation of the program. Similarly, many features of ICMP are described in the later chapters, in terms of how a particular ICMP message is used by a protocol or an application.

We also don't want to save all the good stuff until the end, so we describe TCP/IP applications as soon as we have the foundation to understand them. Ping and Traceroute are described after IP and ICMP have been discussed. The applications built on UDP (multicasting, the DNS, TFTP, and BOOTP) are described after UDP has been examined. The TCP applications, however, along with network management, must be saved until the end, after we've thoroughly described TCP. This text focuses on how these applications use the TCP/IP protocols. We do not provide all the details on running these applications.

Readers

This book is self-contained and assumes no specific knowledge of networking or TCP/IP. Numerous references are provided for readers interested in additional details on specific topics.

This book can be used in many ways. It can be used as a self-study reference and covered from start to finish by someone interested in all the details on the TCP/IP protocol suite. Readers with some TCP/IP background might want to skip ahead and start with Chapter 7, and then focus on the specific chapters in which they're interested. Exercises are provided at the end of the chapters, and most solutions are in Appendix D. This is to maximize the usefulness of the text as a self-study reference.

When used as part of a one- or two-semester course in computer networking, the focus should be on IP (Chapters 3 and 9), UDP (Chapter 11), and TCP (Chapters 17-24), along with some of the application chapters.

Many forward and backward references are provided throughout the text, along with a thorough index, to allow individual chapters to be studied by themselves. A list of all the acronyms used throughout the text, along with the compound term for the acronym, appears on the inside back covers.

If you have access to a network you are encouraged to obtain the software used in this book (Appendix F) and experiment on your own. Hands-on experimentation with the protocols will provide the greatest knowledge (and make it more fun).

Systems Used for Testing

Every example in the book was run on an actual network and the resulting output saved in a file for inclusion in the text. Figure 1.11 (p. 18) shows a diagram of the different hosts, routers, and networks that are used. (This figure is also duplicated on the inside front cover for easy reference while reading the book.) This collection of networks is simple enough that the topology doesn't confuse the examples, and with four systems acting as routers, we can see the error messages generated by routers.

Most of the systems have a name that indicates the type of software being used: bsdi, svr4, sun, solaris, aix, slip, and so on. In this way we can identify the type of software that we're dealing with by looking at the system name in the printed output.

A wide range of different operating systems and TCP/IP implementations are used:

BSD/386 Version 1.0 from Berkeley Software Design, Inc., on the hosts named bsdi and slip. This system is derived from the BSD Networking Software, Release 2.0. (We show the lineage of the various BSD releases in Figure 1.10 on p. 17.)
Unix System V/386 Release 4.0 Version 2.0 from U.H. Corporation, on the host named svr4. This is vanilla SVR4 and contains the standard implementation of TCP/IP from Lachman Associates used with most versions of SVR4.
SunOS 4.1.3 from Sun Microsystems, on the host named sun. The SunOS 4.1.x systems are probably the most widely used TCP/IP implementations. The TCP/IP code is derived from 4.2BSD and 4.3BSD.
Solaris 2.2 from Sun Microsystems, on the host named solaris. The Solaris 2.x systems have a different implementation of TCP/IP from the earlier SunOS 4.1.x systems, and from SVR4. (This operating system is really SunOS 5.2, but is commonly called Solaris 2.2.)
AIX 3.2.2 from IBM on the host named aix. The TCP/IP implementation is based on the 4.3BSD Reno release.
4.4BSD from the Computer Systems Research Group at the University of California at Berkeley, on the host vangogh.cs.berkeley.edu. This system has the latest release of TCP/IP from Berkeley. (This system isn't shown in the figure on

We are deeply saddened to learn of the death of noted author W. Richard Stevens. His passing is obviously a tremendous loss for the technical community, but it is a personal one for us as well. Rich was both a gifted colleague and a valued friend who will be greatly missed. We extend our sympathies to his family.

Obituary from the Arizona Daily Star:

STEVENS, W. Richard, noted author of computer books died on September 1. He is best known for his "UNIX Network Programming" series (1990, 1998, 1999), "Advanced Programming in the UNIX Environment" (1992), and "TCP/IP Illustrated" series (1994, 1995, 1996). Richard was born in 1951 in Luanshya, Northern Rhodesia (now Zambia), where his father worked for the copper industry. The family moved to Salt Lake City, Hurley, New Mexico, Washington, DC and Phalaborwa, South Africa. Richard attended Fishburne Military School in Waynesboro, Virginia. He received a B.SC. in Aerospace Engineering from the University of Michigan in 1973, and an M.S. (1978) and Ph.D. (1982) in Systems Engineering from the University of Arizona. He moved to Tucson in 1975 and from then until 1982 he was employed at Kitt Peak National Observatory as a computer programmer. From 1982 until 1990 he was Vice President of Computing Services at Health Systems International in New Haven, CT, moving back to Tucson in 1990. Here he pursued his career as an author and consultant. He was also an avid pilot and a part-time flight instructor during the 1970's.
He is survived by his loving wife of 20 years, Sally Hodges Stevens; three wonderful children, Bill, Ellen and David; sister, Claire Stevens of Las Vegas, NV; brother, Bob and wife Linda Stevens of Dallas, TX; nieces, Laura, Sarah, Collette, Christy; and nephew, Brad. He is predeceased by his parents, Royale J. Stevens (1915-1984); and Helen Patterson Stevens (1916-1997). Helen lived in Tucson from 1991-1997, and Royale lived here in the early 1930's attending Tucson High School while his father was treated for TB at the Desert Sanitorium (now TMC). The family asks that in lieu of flowers, donations be made in Richard's name to Habitat for Humanity, 2950 E. 22nd Street, Tucson, AZ 85713. A memorial service for Richard will be held at St. Phillip's in the Hills Episcopal Church on Tuesday, September 7th at 12:00 noon. Following the service there will be a reception in the Murphy Gallery of the Church. Please wear colorful clothing to the service; Richard loved colors.

W. Richard Stevens was an acknowledged UNIX and networking expert and the highly-respected author of several books. He was also a sought-after instructor and consultant.

Preface

xv

Introduction

1

(20)

Introduction

1

(1)

Layering

1

(5)

TCP/IP Layering

6

(1)

Internet Addresses

7

(2)

The Domain Name System

9

(1)

Encapsulation

9

(2)

Demultiplexing

11

(1)

Client-Server Model

12

(1)

Port Numbers

12

(2)

Standardization Process

14

(1)

RFCs

14

(1)

Standard, Simple Services

15

(1)

The Internet

16

(1)

Implementations

16

(1)

Application Programming Interfaces

17

(1)

Test Network

18

(1)

Summary

19

(2)

Link Layer

21

(12)

Introduction

21

(1)

Ethernet and IEEE 802 Encapsulation

21

(2)

Trailer Encapsulation

23

(1)

SLIP: Serial Line IP

24

(1)

Compressed SLIP

25

(1)

PPP: Point-to-Point Protocol

26

(2)

Loopback Interface

28

(1)

MTU

29

(1)

Path MTU

30

(1)

Serial. Line Throughput Calculations

30

(1)

Summary

31

(2)

IP: Internet Protocol

33

(20)

Introduction

33

(1)

IP Header

34

(3)

IP Routing

37

(5)

Subnet Addressing

42

(1)

Subnet Mask

43

(2)

Special Case IP Addresses

45

(1)

A Subnet Example

46

(1)

ifconfig Command

47

(2)

netstat Command

49

(1)

IP Futures

49

(1)

Summary

50

(3)

ARP: Address Resolution Protocol

53

(12)

Introduction

53

(1)

An Example

54

(2)

ARP Cache

56

(1)

ARP Packet Format

56

(1)

ARP Examples

57

(3)

Proxy ARP

60

(2)

Gratuitous ARP

62

(1)

arp Command

63

(1)

Summary

63

(2)

RARP: Reverse Address Resolution Protocol

65

(4)

Introduction

65

(1)

RARP Packet Format

65

(1)

RARP Examples

66

(1)

RARP Server Design

67

(1)

Summary

68

(1)

ICMP: Internet Control Message Protocol

69

(16)

Introduction

69

(1)

ICMP Message Types

70

(2)

ICMP Address Mask Request and Reply

72

(2)

ICMP Timestamp Request and Reply

74

(3)

ICMP Port Unreachable Error

77

(4)

4.4BSD Processing of ICMP Messages

81

(2)

Summary

83

(2)

Ping Program

85

(12)

Introduction

85

(1)

Ping Program

85

(6)

IP Record Route Option

91

(4)

IP Timestamp Option

95

(1)

Summary

96

(1)

Traceroute Program

97

(14)

Introduction

97

(1)

Traceroute Program Operation

97

(2)

LAN Output

99

(3)

WAN Output

102

(2)

IP Source Routing Option

104

(5)

Summary

109

(2)

IP Routing

111

(16)

Introduction

111

(1)

Routing Principles

112

(5)

ICMP Host and Network Unreachable Errors

117

(2)

To Forward or Not to Forward

119

(1)

ICMP Redirect Errors

119

(4)

ICMP Router Discovery Messages

123

(2)

Summary

125

(2)

Dynamic Routing Protocols

127

(16)

Introduction

127

(1)

Dynamic Routing

127

(1)

Unix Routing Daemons

128

(1)

RIP: Routing Information Protocol

129

(7)

RIP Version 2

136

(1)

OSPF: Open Shortest Path First

137

(1)

BGP: Border Gateway Protocol

138

(2)

CIDR: Classless Interdomain Routing

140

(1)

Summary

141

(2)

UDP: User Datagram Protocol

143

(26)

Introduction

143

(1)

UDP Header

144

(1)

UDP Checksum

144

(3)

A Simple Example

147

(1)

IP Fragmentation

148

(3)

ICMP Unreachable Error (Fragmentation Required)

151

(2)

Determining the Path MTU Using Traceroute

153

(2)

Path MTU Discovery with UDP

155

(2)

Interaction Between UDP and ARP

157

(2)

Maximum UDP Datagram Size

159

(1)

ICMP Source Quench Error

160

(2)

UDP Server Design

162

(5)

Summary

167

(2)

Broadcasting and Multicasting

169

(10)

Introduction

169

(2)

Broadcasting

171

(1)

Broadcasting Examples

172

(3)

Multicasting

175

(3)

Summary

178

(1)

IGMP: Internet Group Management Protocol

179

(8)

Introduction

179

(1)

IGMP Message

180

(1)

IGMP Protocol

180

(3)

An Example

183

(3)

Summary

186

(1)

DNS: The Domain Name System

187

(22)

Introduction

187

(1)

DNS Basics

188

(3)

DNS Message Format

191

(3)

A Simple Example

194

(4)

Pointer Queries

198

(3)

Resource Records

201

(2)

Caching

203

(3)

UDP or TCP

206

(1)

Another Example

206

(2)

Summary

208

(1)

TFTP: Trivial File Transfer Protocol

209

(6)

Introduction

209

(1)

Protocol

209

(2)

An Example

211

(2)

Security

213

(1)

Summary

213

(2)

BOOTP: Bootstrap Protocol

215

(8)

Introduction

215

(1)

BOOTP Packet Format

215

(3)

An Example

218

(1)

BOOTP Server Design

219

(1)

BOOTP Through a Router

220

(1)

Vendor-Specific Information

221

(1)

Summary

222

(1)

TCP: Transmission Control Protocol

223

(6)

Introduction

223

(1)

TCP Services

223

(2)

TCP Header

225

(2)

Summary

227

(2)

TCP Connection Establishment and Termination

229

(34)

Introduction

229

(1)

Connection Establishment and Termination

229

(6)

Timeout of Connection Establishment

235

(1)

Maximum Segment Size

236

(2)

TCP Half-Close

238

(2)

TCP State Transition Diagram

240

(6)

Reset Segments

246

(4)

Simultaneous Open

250

(2)

Simultaneous Close

252

(1)

TCP Options

253

(1)

TCP Server Design

254

(6)

Summary

260

(3)

TCP Interactive Data Flow

263

(12)

Introduction

263

(1)

Interactive Input

263

(2)

Delayed Acknowledgments

265

(2)

Nagle Algorithm

267

(7)

Window Size Advertisements

274

(1)

Summary

274

(1)

TCP Bulk Data Flow

275

(22)

Introduction

275

(1)

Normal Data Flow

275

(5)

Sliding Windows

280

(2)

Window Size

282

(2)

PUSH Flag

284

(1)

Slow Start

285

(1)

Bulk Data Throughput

286

(6)

Urgent Mode

292

(4)

Summary

296

(1)

TCP Timeout and Retransmission

297

(26)

Introduction

297

(1)

Simple Timeout and Retransmission Example

298

(1)

Round-Trip Time Measurement

299

(2)

An RTT Example

301

(5)

Congestion Example

306

(4)

Congestion Avoidance Algorithm

310

(2)

Fast Retransmit and Fast Recovery Algorithms

312

(1)

Congestion Example (Continued)

313

(3)

Per-Route Metrics

316

(1)

ICMP Errors

317

(3)

Repacketization

320

(1)

Summary

321

(2)

TCP Persist Timer

323

(8)

Introduction

323

(1)

An Example

323

(2)

Silly Window Syndrome

325

(5)

Summary

330

(1)

TCP Keepalive Timer

331

(8)

Introduction

331

(1)

Description

332

(1)

Keepalive Examples

333

(4)

Summary

337

(2)

TCP Futures and Performance

339

(20)

Introduction

339

(1)

Path MTU Discovery

340

(4)

Long Fat Pipes

344

(3)

Window Scale Option

347

(2)

Timestamp Option

349

(2)

PAWS: Protection Against Wrapped Sequence Numbers

351

(1)

T/TCP: A TCP Extension for Transactions

351

(3)

TCP Performance

354

(2)

Summary

356

(3)

SNMP: Simple Network Management Protocol

359

(30)

Introduction

359

(1)

Protocol

360

(3)

Structure of Management Information

363

(1)

Object Identifiers

364

(1)

Introduction to the Management Information Base

365

(2)

Instance Identification

367

(3)

Simple Examples

370

(2)

Management Information Base (Continued)

372

(10)

Additional Examples

382

(3)

Traps

385

(1)

ASN.1 and BER

386

(1)

SNMP Version 2

387

(1)

Summary

388

(1)

Telnet and Rlogin: Remote Login

389

(30)

Introduction

389

(2)

Rlogin Protocol

391

(5)

Rlogin Examples

396

(5)

Telnet Protocol

401

(5)

Telnet Examples

406

(11)

Summary

417

(2)

FTP: File Transfer Protocol

419

(22)

Introduction

419

(1)

FTP Protocol

419

(7)

FTP Examples

426

(13)

Summary

439

(2)

SMTP: Simple Mail Transfer Protocol

441

(20)

Introduction

441

(1)

SMTP Protocol

442

(6)

SMTP Examples

448

(4)

SMTP Futures

452

(7)

Summary

459

(2)

NFS: Network File System

461

(20)

Introduction

461

(1)

Sun Remote Procedure Call

461

(4)

XDR: External Data Representation

465

(1)

Port Mapper

465

(2)

NFS Protocol

467

(7)

NFS Examples

474

(5)

NFS Version 3

479

(1)

Summary

480

(1)

Other TCP/IP Applications

481

(10)

Introduction

481

(1)

Finger Protocol

481

(2)

Whois Protocol

483

(1)

Archie, WAIS, Gopher, Veronica, and WWW

484

(2)

X Window System

486

(4)

Summary

490

(1)

Appendix A. The tcpdump Program

491

(8)

A.1 BSD Packet Filter

491

(2)

A.2 SunOS Network Interface Tap

493

(1)

A.3 SVR4 Data Link Provider Interface

494

(1)

A.4 tcpdump Output

495

(1)

A.5 Security Considerations

496

(1)

A.6 Socket Debug Option

496

(3)

Appendix B. Computer Clocks

499

(4)

Appendix C. The sock Program

503

(4)

Appendix D. Solutions to Selected Exercises

507

(18)

Appendix E. Configurable Options

525

(14)

E.1 BSD/386 Version 1.0

526

(1)

E.2 SunOS 4.1.3

527

(2)

E.3 System V Release 4

529

(1)

E.4 Solaris 2.2

529

(7)

E.5 AIX 3.2.2

536

(1)

E.6 4.4BSD

537

(2)

Appendix F. Source Code Availability

539

(4)

Bibliography

543

(12)

Index

555

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

Introduction This book describes the TCP/IP protocol suite, but from a different perspective than other texts on TCP/IP. Instead of just describing the protocols and what they do, we'll use a popular diagnostic tool to watch the protocols in action. Seeing how the protocols operate in varying circumstances provides a greater understanding of how they work and why certain design decisions were made. It also provides a look into the implementation of the protocols, without having to wade through thousands of lines of source code.When networking protocols were being developed in the 1960s through the 1980s, expensive, dedicated hardware was required to see the packets going "across the wire." Extreme familiarity with the protocols was also required to comprehend the packets displayed by the hardware. Functionality of the hardware analyzers was limited to that built in by the hardware designers.Today this has changed dramatically with the ability of the ubiquitous workstation to monitor a local area network Mogul 1990. Just attach a workstation to your network, run some publicly available software (described in Appendix A), and watch what goes by on the wire. While many people consider this a tool to be used fordiagnosingnetwork problems, it is also a powerful tool forunderstandinghow the network protocols operate, which is the goal of this book.This book is intended for anyone wishing to understand how the TCP/IP protocols operate: programmers writing network applications, system administrators responsible for maintaining computer systems and networks utilizing TCP/IP, and users who deal with TCP/IP applications on a daily basis. Organization of the Book We take a bottom-up approach to the TCP/IP protocol suite. After providing a basic introduction to TCP/IP in Chapter 1, we will start at the link layer in Chapter 2 and work our way up the protocol stack. This provides the required background for later chapters for readers who aren't familiar with TCP/IP or networking in general.This book also uses a functional approach instead of following a strict bottom-to-top order. For example, Chapter 3 describes the IP layer and the IP header. But there are numerous fields in the IP header that are best described in the context of an application that uses or is affected by a particular field. Fragmentation, for example, is best understood in terms of UDP (Chapter 11), the protocol often affected by it. The time-to-live field is fully described when we look at the Traceroute program in Chapter 8, because this field is the basis for the operation of the program. Similarly, many features of ICMP are described in the later chapters, in terms of how a particular ICMP message is used by a protocol or an application.We also don't want to save all the good stuff until the end, so we describe TCP/IP applications as soon as we have the foundation to understand them. Ping and Traceroute are described after IP and ICMP have been discussed. The applications built on UDP (multicasting, the DNS, TFTP, and BOOTP) are described after UDP has been examined. The TCP applications, however, along with network management, must be saved until the end, after we've thoroughly described TCP. This text focuses on how these applications use the TCP/IP protocols. We do not provide all the details on running these applications. Readers This book is self-contained and assumes no specific knowledge of networking or TCP/IP. Numerous references are provided for readers intereste