Understanding Linux Network Internals

by Benvenuti, Christian

• ISBN13: 9780596002558
• ISBN10: 0596002556
• eBook ISBN(s): 9780596002558, 9780596002558
• Additional ISBN(s): 9780596002558, 9780596002558
• Format: Paperback
• Copyright: 2005-12-01
• Publisher: Oreilly & Associates Inc

Summary

As a perfect follow-up to O'Reilly's bestselling guide, Understanding the Linux Kernel, this new book takes Linux programmers and network administrators right to the heart of the operating system to explain how it performs critical networking functions. More than just a reference, it helps readers understand how best to modify Linux in different networking environments. This detailed book offers a guided tour of code in the Linux kernel (v 2.6) that forms the core of all networking functions, and shows how it works with the Internet's TCP/IP protocol stack. It covers core areas of networking in meticulous detail, from basics such as device registration, IP routing, and the evolution of TCP implementations to the routing, bridging, and neighbouring subsystems. Author Christian Benvenuti introduces each topic by explaining its importance and how it operates in the Linux networking galaxy.

Author Biography

Christian Benvenuti received his masters degree in Computer Science at the University of Bologna in Italy. He collaborated for a few years with the International Center for Theoretical Physics (ICTP) in Trieste, where he developed ad-hoc software based on the Linux kernel, was a scientific consultant for a project on remote collaboration, and served as an instructor for several training sessions on networking. The trainings, held mainly in Europe, Africa, and South America were all based on Linux systems and addressed to scientists from developing countries, where the ICTP has been promoting Linux for many years. He occasionally collaborates with a non-profit organization founded by ICTP members, Collaborium.org, to continue promoting Linux on developing countries.

In the past few years he worked as a software engineer for Cisco Systems in the Silicon Valley, where he focused on Layer two switching, high availability, and network security.

Table of Contents

Preface	xv
Part I. General Background
Introduction	3	(19)
Basic Terminology	3	(1)
Common Coding Patterns	4	(14)
User-Space Tools	18	(1)
Browsing the Source Code	19	(1)
When a Feature Is Offered as a Patch	20	(2)
Critical Data Structures	22	(36)
The Socket Buffer: sk_buff Structure	22	(21)
net_device Structure	43	(14)
Files Mentioned in This Chapter	57	(1)
User-Space-to-Kernel Interface	58	(17)
Overview	58	(2)
procfs Versus sysctl	60	(7)
ioctl	67	(3)
Netlink	70	(1)
Serializing Configuration Changes	71	(4)
Part II. System Initialization
Notification Chains	75	(9)
Reasons for Notification Chains	75	(2)
Overview	77	(1)
Defining a Chain	78	(1)
Registering with a Chain	78	(1)
Notifying Events on a Chain	79	(2)
Notification Chains for the Networking Subsystems	81	(1)
Tuning via /proc Filesystem	82	(1)
Functions and Variables Featured in This Chapter	83	(1)
Files and Directories Featured in This Chapter	83	(1)
Network Device Initialization	84	(22)
System Initialization Overview	84	(2)
Device Registration and Initialization	86	(1)
Basic Goals of NIC Initialization	86	(1)
Interaction Between Devices and Kernel	87	(6)
Initialization Options	93	(1)
Module Options	93	(1)
Initializing the Device Handling Layer: net_dev_init	94	(2)
User-Space Helpers	96	(4)
Virtual Devices	100	(3)
Tuning via /proc Filesystem	103	(1)
Functions and Variables Featured in This Chapter	104	(1)
Files and Directories Featured in This Chapter	105	(1)
The PCI Layer and Network Interface Cards	106	(10)
Data Structures Featured in This Chapter	106	(2)
Registering a PCI NIC Device Driver	108	(1)
Power Management and Wake-on-LAN	109	(1)
Example of PCI NIC Driver Registration	110	(2)
The Big Picture	112	(2)
Tuning via /proc Filesystem	114	(1)
Functions and Variables Featured in This Chapter	114	(1)
Files and Directories Featured in This Chapter	115	(1)
Kernel Infrastructure for Component Initialization	116	(20)
Boot-Time Kernel Options	116	(6)
Module Initialization Code	122	(3)
Optimized Macro-Based Tagging	125	(3)
Boot-Time Initialization Routines	128	(2)
Memory Optimizations	130	(4)
Tuning via /proc Filesystem	134	(1)
Functions and Variables Featured in This Chapter	134	(1)
Files and Directories Featured in This Chapter	135	(1)
Device Registration and Initialization	136	(41)
When a Device Is Registered	137	(1)
When a Device Is Unregistered	138	(1)
Allocating net_device Structures	138	(2)
Skeleton of NIC Registration and Unregistration	140	(1)
Device Initialization	141	(4)
Organization of net_device Structures	145	(2)
Device State	147	(2)
Registering and Unregistering Devices	149	(5)
Device Registration	154	(2)
Device Unregistration	156	(3)
Enabling and Disabling a Network Device	159	(2)
Updating the Device Queuing Discipline State	161	(5)
Configuring Device-Related Information from User Space	166	(3)
Virtual Devices	169	(2)
Locking	171	(1)
Tuning via /proc Filesystem	171	(1)
Functions and Variables Featured in This Chapter	172	(1)
Files and Directories Featured in This Chapter	173	(4)
Part III. Transmission and Reception
Interrupts and Network Drivers	177	(33)
Decisions and Traffic Direction	178	(1)
Notifying Drivers When Frames Are Received	178	(5)
Interrupt Handlers	183	(23)
softnet_data Structure	206	(4)
Frame Reception	210	(29)
Interactions with Other Features	211	(1)
Enabling and Disabling a Device	211	(1)
Queues	212	(1)
Notifying the Kernel of Frame Reception: NAPI and netif_rx	212	(7)
Old Interface Between Device Drivers and Kernel: First Part of netif_rx	219	(6)
Congestion Management	225	(3)
Processing the NET_RX_SOFTIRQ: net_rx_action	228	(11)
Frame Transmission	239	(22)
Enabling and Disabling Transmissions	241	(20)
General and Reference Material About Interrupts	261	(5)
Statistics	261	(1)
Tuning via /proc and sysfs Filesystems	262	(1)
Functions and Variables Featured in This Part of the Book	263	(2)
Files and Directories Featured in This Part of the Book	265	(1)
Protocol Handlers	266	(31)
Overview of Network Stack	266	(8)
Executing the Right Protocol Handler	274	(4)
Protocol Handler Organization	278	(1)
Protocol Handler Registration	279	(2)
Ethernet Versus IEEE 802.3 Frames	281	(12)
Tuning via /proc Filesystem	293	(1)
Functions and Variables Featured in This Chapter	293	(1)
Files and Directories Featured in This Chapter	294	(3)
Part IV. Bridging
Bridging: Concepts	297	(13)
Repeaters, Bridges, and Routers	297	(2)
Bridges Versus Switches	299	(1)
Hosts	300	(1)
Merging LANs with Bridges	300	(2)
Bridging Different LAN Technologies	302	(1)
Address Learning	302	(3)
Multiple Bridges	305	(5)
Bridging: The Spanning Tree Protocol	310	(45)
Basic Terminology	311	(1)
Example of Hierarchical Switched 1.2 Topology	311	(3)
Basic Elements of the Spanning Tree Protocol	314	(7)
Bridge and Port IDs	321	(2)
Bridge Protocol Data Units (BPDUs)	323	(5)
Defining the Active Topology	328	(7)
Timers	335	(5)
Topology Changes	340	(4)
BPDU Encapsulation	344	(2)
Transmitting Configuration BPDUs	346	(1)
Processing Ingress Frames	347	(2)
Convergence Time	349	(1)
Overview of Newer Spanning Tree Protocols	350	(5)
Bridging: Linux Implementation	355	(36)
Bridge Device Abstraction	355	(4)
Important Data Structures	359	(1)
Initialization of Bridging Code	360	(1)
Creating Bridge Devices and Bridge Ports	361	(1)
Creating a New Bridge Device	362	(1)
Bridge Device Setup Routine	362	(2)
Deleting a Bridge	364	(1)
Adding Ports to a Bridge	364	(3)
Enabling and Disabling a Bridge Device	367	(1)
Enabling and Disabling a Bridge Port	368	(2)
Changing State on a Bridge Port	370	(1)
The Big Picture	371	(2)
Forwarding Database	373	(2)
Handling Ingress Traffic	375	(5)
Transmitting on a Bridge Device	380	(1)
Spanning Tree Protocol (STP)	380	(9)
netdevice Notification Chain	389	(2)
Bridging: Miscellaneous Topics	391	(18)
User-Space Configuration Tools	391	(5)
Tuning via /proc Filesystem	396	(1)
Tuning via /sys Filesystem	396	(2)
Statistics	398	(1)
Data Structures Featured in This Part of the Book	398	(5)
Functions and Variables Featured in This Part of the Book	403	(2)
Files and Directories Featured in This Part of the Book	405	(4)
Part V. Internet Protocol Version 4 (IPv4)
Internet Protocol Version 4 (IPv4): Concepts	409	(30)
IP Protocol: The Big Picture	409	(2)
IP Header	411	(3)
IP Options	414	(6)
Packet Fragmentation/Defragmentation	420	(12)
Checksums	432	(7)
Internet Protocol Version 4 (IPv4): Linux Foundations and Features	439	(27)
Main IPv4 Data Structures	439	(4)
General Packet Handling	443	(10)
IP Options	453	(13)
Internet Protocol Version 4 (IPv4): Forwarding and Local Delivery	466	(7)
Forwarding	466	(6)
Local Delivery	472	(1)
Internet Protocol Version 4 (IPv4): Transmission	473	(38)
Key Functions That Perform Transmission	474	(36)
Interface to the Neighboring Subsystem	510	(1)
Internet Protocol Version 4 (IPv4): Handling Fragmentation	511	(25)
IP Fragmentation	512	(9)
IP Defragmentation	521	(15)
Internet Protocol Version 4 (IPv4): Miscellaneous Topics	536	(33)
Long-Living IP Peer Information	536	(4)
Selecting the IP Header's ID Field	540	(1)
IP Statistics	541	(4)
IP Configuration	545	(5)
IP-over-IP	550	(1)
IPv4: What's Wrong with It?	551	(2)
Tuning via /proc Filesystem	553	(2)
Data Structures Featured in This Part of the Book	555	(10)
Functions and Variables Featured in This Part of the Book	565	(3)
Files and Directories Featured in This Part of the Book	568	(1)
Layer Four Protocol and Raw IP Handling	569	(16)
Available L4 Protocols	569	(2)
L4 Protocol Registration	571	(3)
L3 to L4 Delivery: ip_local_deliver_finish	574	(8)
IPv4 Versus IPv6	582	(1)
Tuning via /proc Filesystem	583	(1)
Functions and Variables Featured in This Chapter	583	(1)
Files and Directories Featured in This Chapter	583	(2)
Internet Control Message Protocol (ICMPv4)	585	(40)
ICMP Header	586	(1)
ICMP Payload	587	(1)
ICMP Types	588	(7)
Applications of the ICMP Protocol	595	(3)
The Big Picture	598	(1)
Protocol Initialization	599	(1)
Data Structures Featured in This Chapter	600	(2)
Transmitting ICMP Messages	602	(9)
Receiving ICMP Messages	611	(6)
ICMP Statistics	617	(2)
Passing Error Notifications to the Transport Layer	619	(1)
Tuning via /proc Filesystem	620	(2)
Functions and Variables Featured in This Chapter	622	(1)
Files and Directories Featured in This Chapter	622	(3)
Part VI. Neighboring Subsystem
Neighboring Subsystem: Concepts	625	(26)
What Is a Neighbor?	625	(3)
Reasons That Neighboring Protocols Are Needed	628	(6)
Linux Implementation	634	(3)
Proxying the Neighboring Protocol	637	(3)
When Solicitation Requests Are Transmitted and Processed	640	(2)
Neighbor States and Network Unreachability Detection (NUD)	642	(9)
Neighboring Subsystem: Infrastructure	651	(48)
Main Data Structures	651	(4)
Common Interface Between L3 Protocols and Neighboring Protocols	655	(11)
General Tasks of the Neighboring Infrastructure	666	(4)
Reference Counts on neighbour Structures	670	(1)
Creating a neighbour Entry	671	(2)
Neighbor Deletion	673	(6)
Acting As a Proxy	679	(4)
L2 Header Caching	683	(4)
Protocol Initialization and Cleanup	687	(1)
Interaction with Other Subsystems	688	(4)
Interaction Between Neighboring Protocols and L3 Transmission Functions	692	(4)
Queuing	696	(3)
Neighboring Subsystem: Address Resolution Protocol (ARP)	699	(50)
ARP Packet Format	700	(2)
Example of an ARP Transaction	702	(1)
Gratuitous ARP	702	(5)
Responding from Multiple Interfaces	707	(1)
Tunable ARP Options	708	(6)
ARP Protocol Initialization	714	(2)
Initialization of a neighbour Structure	716	(6)
Transmitting and Receiving ARP Packets	722	(4)
Processing Ingress ARP Packets	726	(9)
Proxy ARP	735	(5)
Examples	740	(2)
External Events	742	(2)
ARPD	744	(2)
Reverse Address Resolution Protocol (RARP)	746	(2)
Improvements in ND (IPv6) over ARP (IPv4)	748	(1)
Neighboring Subsystem: Miscellaneous Topics	749	(28)
System Administration of Neighbors	749	(3)
Tuning via /proc Filesystem	752	(5)
Data Structures Featured in This Part of the Book	757	(17)
Files and Directories Featured in This Part of the Book	774	(3)
Part VII. Routing
Routing: Concepts	777	(25)
Routers, Routes, and Routing Tables	778	(3)
Essential Elements of Routing	781	(12)
Routing Table	793	(5)
Lookups	798	(2)
Packet Reception Versus Packet Transmission	800	(2)
Routing: Advanced	802	(28)
Concepts Behind Policy Routing	802	(6)
Concepts Behind Multipath Routing	808	(7)
Interactions with Other Kernel Subsystems	815	(4)
Routing Protocol Daemons	819	(2)
Verbose Monitoring	821	(1)
ICMP_Redirect Messages	822	(6)
Reverse Path Filtering	828	(2)
Routing: Linux Implementation	830	(31)
Kernel Options	830	(4)
Main Data Structures	834	(3)
Route and Address Scopes	837	(4)
Primary and Secondary IP Addresses	841	(1)
Generic Helper Routines and Macros	842	(1)
Global Locks	843	(1)
Routing Subsystem Initialization	844	(1)
External Events	845	(13)
Interactions with Other Subsystems	858	(3)
Routing: The Routing Cache	861	(37)
Routing Cache Initialization	861	(1)
Hash Table Organization	862	(2)
Major Cache Operations	864	(9)
Multipath Caching	873	(6)
Interface Between the DST and Calling Protocols	879	(6)
Flushing the Routing Cache	885	(1)
Garbage Collection	886	(10)
Egress ICMP Redirect Rate Limiting	896	(2)
Routing: Routing Tables	898	(14)
Organization of Routing Hash Tables	898	(6)
Routing Table Initialization	904	(1)
Adding and Removing Routes	905	(5)
Policy Routing and Its Effects on Routing Table Definitions	910	(2)
Routing: Lookups	912	(40)
High-Level View of Lookup Functions	912	(1)
Helper Routines	913	(1)
The Table Lookup: fn_hash_lookup	914	(5)
fib_lookup Function	919	(1)
Setting Functions for Reception and Transmission	920	(3)
General Structure of the Input and Output Routing Routines	923	(1)
Input Routing	924	(9)
Output Routing	933	(8)
Effects of Multipath on Next Hop Selection	941	(3)
Policy Routing	944	(2)
Source Routing	946	(2)
Policy Routing and Routing Table Based Classifier	948	(4)
Routing: Miscellaneous Topics	952	(39)
User-Space Configuration Tools	952	(6)
Statistics	958	(1)
Tuning via /proc Filesystem	958	(8)
Enabling and Disabling Forwarding	966	(2)
Data Structures Featured in This Part of the Book	968	(18)
Functions and Variables Featured in This Part of the Book	986	(3)
Files and Directories Featured in This Part of the Book	989	(2)
Index	991