TCP/IP Illustrated, Volume 3 TCP for Transactions, HTTP, NNTP, and the UNIX Domain Protocols

by Stevens, W. Richard

• ISBN13: 9780201634952
• ISBN10: 0201634953
• Edition: 1st
• Format: Hardcover
• Copyright: 1996-01-19
• Publisher: Addison-Wesley Professional

Summary

TCP/IP Illustrated, Volume 3 covers four major topics of great importance to anyone working TCP/IP. It contains the first thorough treatment of TCP for transactions, commonly known as T/TCP, an extension to TCP that makes client-server transactions faster and more efficient. Next, the book covers two popular applications of T/TCP, the very hot topic of HTTP (the Hypertext Transfer Protocol), the foundation for the World Wide Web, and NNTP (the Network News Transfer Protocol), the basis for the Usenet news system. Both of these topics have increased in significance as the Internet has exploded in size and usage. Finally, the book covers UNIX Domain Protocols, protocols that are used heavily in UNIX implementations.

Author Biography

We are deeply saddened to learn of the death of noted author W. Richard Stevens. His passing is obviously a tremendous loss for the technical community, but it is a personal one for us as well. Rich was both a gifted colleague and a valued friend who will be greatly missed. We extend our sympathies to his family.

Obituary from the Arizona Daily Star:

STEVENS, W. Richard, noted author of computer books died on September 1. He is best known for his "UNIX Network Programming" series (1990, 1998, 1999), "Advanced Programming in the UNIX Environment" (1992), and "TCP/IP Illustrated" series (1994, 1995, 1996). Richard was born in 1951 in Luanshya, Northern Rhodesia (now Zambia), where his father worked for the copper industry. The family moved to Salt Lake City, Hurley, New Mexico, Washington, DC and Phalaborwa, South Africa. Richard attended Fishburne Military School in Waynesboro, Virginia. He received a B.SC. in Aerospace Engineering from the University of Michigan in 1973, and an M.S. (1978) and Ph.D. (1982) in Systems Engineering from the University of Arizona. He moved to Tucson in 1975 and from then until 1982 he was employed at Kitt Peak National Observatory as a computer programmer. From 1982 until 1990 he was Vice President of Computing Services at Health Systems International in New Haven, CT, moving back to Tucson in 1990. Here he pursued his career as an author and consultant. He was also an avid pilot and a part-time flight instructor during the 1970's.

He is survived by his loving wife of 20 years, Sally Hodges Stevens; three wonderful children, Bill, Ellen and David; sister, Claire Stevens of Las Vegas, NV; brother, Bob and wife Linda Stevens of Dallas, TX; nieces, Laura, Sarah, Collette, Christy; and nephew, Brad. He is predeceased by his parents, Royale J. Stevens (1915-1984); and Helen Patterson Stevens (1916-1997). Helen lived in Tucson from 1991-1997, and Royale lived here in the early 1930's attending Tucson High School while his father was treated for TB at the Desert Sanitorium (now TMC). The family asks that in lieu of flowers, donations be made in Richard's name to Habitat for Humanity, 2950 E. 22nd Street, Tucson, AZ 85713. A memorial service for Richard will be held at St. Phillip's in the Hills Episcopal Church on Tuesday, September 7th at 12:00 noon. Following the service there will be a reception in the Murphy Gallery of the Church. Please wear colorful clothing to the service; Richard loved colors.

W. Richard Stevens was an acknowledged UNIX and networking expert and the highly-respected author of several books. He was also a sought-after instructor and consultant.

0201634953AB04062001

Table of Contents

Preface	xv
Part 1. TCP for Transactions	1	(158)
T/TCP Introduction	3	(26)
Introduction	3	(1)
UDP Client--Server	3	(6)
TCP Client--Server	9	(8)
T/TCP Client--Server	17	(3)
Test Network	20	(1)
Timing Example	21	(1)
Applications	22	(2)
History	24	(2)
Implementations	26	(2)
Summary	28	(1)
T/TCP Protocol	29	(10)
Introduction	29	(1)
New TCP Options for T/TCP	30	(3)
T/TCP Implementation Variables	33	(1)
State Transition Diagram	34	(2)
T/TCP Extended States	36	(2)
Summary	38	(1)
T/TCP Examples	39	(14)
Introduction	39	(1)
Client Reboot	40	(2)
Normal T/TCP Transaction	42	(1)
Server Receives Old Duplicate SYN	43	(1)
Server Reboot	44	(1)
Request or Reply Exceeds MSS	45	(4)
Backward Compatibility	49	(2)
Summary	51	(2)
T/TCP Protocol (Continued)	53	(16)
Introduction	53	(1)
Client Port Numbers and TIME_WAIT State	53	(3)
Purpose of the TIME_WAIT State	56	(3)
TIME_WAIT State Truncation	59	(3)
Avoiding the Three_Way Handshake with TAO	62	(6)
Summary	68	(1)
T/TCP Implementation: Socket Layer	69	(4)
Introduction	69	(1)
Constants	70	(1)
sosend Function	70	(2)
Summary	72	(1)
T/TCP Implementation: Routing Table	73	(14)
Introduction	73	(1)
Code Introduction	74	(1)
radix_node_head Structure	75	(1)
rtentry Structure	75	(1)
rt_metrics Structure	76	(1)
in_inithead Function	76	(1)
in_addroute Function	77	(1)
in_matroute Function	78	(1)
in_clsroute Function	78	(1)
in_rtqtimo Function	79	(3)
in_rtqkill Function	82	(3)
Summary	85	(2)
T/TCP Implementation: Protocol Control Blocks	87	(4)
Introduction	87	(1)
in_pcbladdr Function	88	(1)
in_pcbconnect Function	89	(1)
Summary	90	(1)
T/TCP Implementation: TCP Overview	91	(6)
Introduction	91	(1)
Code Introduction	91	(1)
TCP protosw Structure	92	(1)
TCP Control Block	93	(1)
tcp_init Function	94	(1)
tcp_slowtimo Function	94	(1)
Summary	95	(2)
T/TCP Implementation: TCP Output	97	(8)
Introduction	97	(1)
tcp_output Function	97	(7)
Summary	104	(1)
T/TCP Implementation: TCP Functions	105	(20)
Introduction	105	(1)
tcp_newtcpcb Function	105	(1)
tcp_rtlookup Function	106	(2)
tcp_gettaocache Function	108	(1)
Retransmission Timeout Calculations	108	(4)
tcp_close Function	112	(1)
tcp_msssend Function	113	(1)
tcp_mssrcvd Function	114	(7)
tcp_dooptions Function	121	(1)
tcp_reass Function	122	(2)
Summary	124	(1)
T/TCP Implementation: TCP Input	125	(24)
Introduction	125	(1)
Preliminary Processing	125	(4)
Header Predication	129	(1)
Initiation of Passive Open	130	(4)
Initiation of Active Open	134	(7)
PAWS: Protection Against Wrapped Sequence Numbers	141	(1)
ACK Processing	142	(1)
Completion of Passive Opens and Simultaneous Opens	142	(1)
ACK Processing (Continued)	143	(2)
FIN Processing	145	(2)
Summary	147	(2)
T/TCP Implementation: TCP User Requests	149	(10)
Introduction	149	(1)
PRU_CONNECT Request	149	(1)
tcp_connect Function	150	(4)
PRU_SEND and PRU_SEND_EOF Requests	154	(1)
tcp_usrclosed Function	155	(1)
tcp_sysctl Function	155	(1)
T/TCP Futures	156	(2)
Summary	158	(1)
Part 2. Additional TCP Applications	159	(60)
HTTP: Hypertext Transfer Protocol	161	(16)
Introduction	161	(1)
Introduction to HTTP and HTML	162	(3)
HTTP Protocol	165	(5)
An Example	170	(2)
HTTP Statistics	172	(1)
Performance Problems	173	(2)
Summary	175	(2)
Packets Found on an HTTP Server	177	(30)
Introduction	177	(3)
Multiple HTTP Servers	180	(1)
Client SYN Interarrival Time	181	(4)
RTT Measurements	185	(2)
listen Backlog Queue	187	(5)
Client SYN Options	192	(3)
Client SYN Retransmissions	195	(1)
Domain Names	196	(1)
Timing Out Persist Probes	196	(4)
Simulation of T/TCP Routing Table Size	200	(2)
Mbuf Interaction	202	(1)
TCP PCB Cache and Header Prediction	203	(2)
Summary	205	(2)
NNTP: Network News Transfer Protocol	207	(12)
Introduction	207	(2)
NNTP Protocol	209	(3)
A Simple News Client	212	(2)
A More Sophisticated News Client	214	(1)
NNTP Statistics	215	(1)
Summary	216	(3)
Part 3. The Unix Domain Protocols	219	(72)
Unix Domain Protocols: Introduction	221	(6)
Introduction	221	(1)
Usage	222	(1)
Performance	223	(1)
Coding Examples	224	(1)
Summary	225	(2)
Unix Domain Protocols: Implementations	227	(36)
Introduction	227	(1)
Code Introduction	227	(1)
Unix domain and protosw Structures	228	(2)
Unix Domain Socket Address Structures	230	(1)
Unix Domain Protocol Control Blocks	231	(2)
uipc_usrreq Function	233	(1)
PRU_ATTACH Request and unp_attach Function	233	(3)
PRU_DETACH Request and unp_detach Function	236	(1)
PRU_BIND Request and unp_bind Function	237	(3)
PRU_CONNECT Request and unp_connect Function	240	(5)
PRU_CONNECT2 Request and unp_connect2 Function	245	(4)
socketpair System Call	249	(4)
pipe System Call	253	(1)
PRU_ACCEPT Request	253	(2)
PRU_DISCONNECT Request and unp_disconnect Function	255	(2)
PRU_SHUTDOWN Request and unp_shutdown Function	257	(1)
PRU_ABORT Request and unp_drop Function	258	(1)
Miscellaneous Requests	259	(2)
Summary	261	(2)
Unix Domain Protocols: I/O and Descriptor Passing	263	(28)
Introduction	263	(1)
PRU_SEND and PRU_RCVD Requests	263	(6)
Descriptor Passing	269	(5)
unp_internalize Function	274	(2)
unp_externalize Function	276	(1)
unp_discard Function	277	(1)
unp_dispose Function	278	(1)
unp_scan Function	278	(2)
unp_gc Function	280	(8)
unp_mark Function	288	(1)
Performance (Revisited)	288	(1)
Summary	289	(2)
Appendix A. Measuring Network Times	291	(12)
A.1 RTT Measurements Using Ping	292	(2)
A.2 Protocol Stack Measurements	294	(6)
A.3 Latency and Bandwidth	300	(3)
Appendix B. Coding Applications for T/TCP	303	(6)
Bibliography	309	(6)
Index	315

Excerpts

Introduction and Organization of the Book This book is a logical continuation of theTCP/IP Illustratedseries: Stevens 1994, which we refer to asVolume 1, and Wright and Stevens 1995, which we refer to asVolume 2.This book is divided into three parts,each covering a different topic:TCP for transactions, commonly called T/TCP. This is an extension to TCP designed to make client-server transactions faster, more efficient, and reliable. This is done by omitting TCP's three-way handshake at the beginning of a connection and shortening the TIME_WAIT state at the end of a connection. We'll see that T/TCP can match UDP's performance for a client-server transaction and that T/TCP provides reliability and adaptability, both major improvements over UDP.A transaction is defined to be a client request to a server, followed by the server's reply. (The term transaction does not mean a database transaction, with locking, two-phase commit, and backout.)TCP/IP applications, specifically HTTP (the Hypertext Transfer Protocol, the foundation of the World Wide Web) and NNTP (the Network News Transfer Protocol, the basis for the Usenet news system).The Unix domain protocols. These protocols are provided by all Unix TCP/IP implementations and on many non-Unix implementations. They provide a form of interprocess communication (IPC) and use the same sockets interface used with TCP/IP. When the client and server are on the same host, the Unix domain protocols are often twice as fast as TCP/IP.Part 1, the presentation of T/TCP, is in two pieces. Chapters 1-4 describe the protocol and provide numerous examples of how it works. This material is a major expansion of the brief presentation of T/TCP in Section 24.7 ofVolume 1. The second piece, Chapters 5-12, describes the actual implementation of T/TCP within the 4.4BSD-Lite networking code (i.e., the code presented inVolume 2). Since the first T/TCP implementation was not released until September 1994, about one year afterVolume 1was published and right asVolume 2was being completed, the detailed presentation of T/TCP, with examples and all the implementation details, had to wait for another volume in the series.Part 2, the HTTP and NNTP applications, are a continuation of the TCP/IP applications presented in Chapters 25-30 ofVolume 1. In the two years sinceVolume 1was published, the popularity of HTTP has grown enormously, as the Internet has exploded, and the use of NNTP has been growing about 75% per year for more than 10 years. HTTP is also a wonderful candidate for T/TCP, given its typical use of TCP: short connections with small amounts of data transferred, where the total time is often dominated by the connection setup and teardown. The heavy use of HTTP (and therefore TCP) on a busy Web server by thousands of different and varied clients also provides a unique opportunity to examine the actual packets at the server (Chapter 14) and look at many features of TCP/IP that were presented inVolumes 1 and 2.The Unix domain protocols in Part 3 were originally considered forVolume 2but omitted when its size reached 1200 pages. While it may seem odd to cover protocols other than TCP/IP in a series titledTCP/IP Illustrated, the Unix domain protocols were implemented almost 15 years ago in 4.2BSD alongside the first implementation of BSD TCP/IP. They are used heavily today in any Berkeley-derived kernel, but their use is typically "under the covers," and most users are unaware of their presence. Besides being the foundation for Unix pipes on a Berkeley-derived kernel, another heavy user is the X Window System, when the client and server are on the same host (i.e., on typical workstations). Unix domain sockets are also used to pass descriptors between processes, a powerful technique for interprocess communication. Since the sockets API (application program interface) used with the Unix domain prot