Network Programming With Perl

by Stein, Lincoln D.

• ISBN13: 9780201615715
• ISBN10: 0201615711
• Edition: 1st
• Format: Paperback
• Copyright: 2000-12-27
• Publisher: Addison-Wesley Professional

Summary

"This is a superb book. Another splendid book from Lincoln, whose mastery and lucid exposition make this a must-have for the serious Perl programmer." --Jon Orwant, Chief Technology Officer, O'Reilly & Associates Founder of The Perl Journal, author of Mastering Algorithms with Perl, (O'Reilly & Associates) and co-author of Programming Perl, Third Edition (O'Reilly & Associates) Network Programming with Perl is a comprehensive, example-rich guide to creating network-based applications using the Perl programming language. Among its many capabilities, modern Perl provides a straightforward and powerful interface to TCP/IP, and this book shows you how to leverage these capabilities to create robust, maintainable, and efficient custom client/server applications. The book quickly moves beyond the basics to focus on high-level, application programming concepts, tools, and techniques. Readers will find a review of basic networking concepts and Perl fundamentals, including Perl's I/O functions, process model, and object-oriented extensions. In addition, the book examines a collection of the best third-party modules in the Comprehensive Perl Archive Network, including existing network protocols for e-mail, news, and the Web. The core of the book focuses on methods and alternatives for designing TCP-based client/server systems and more advanced techniques for specialized applications. Specific topics covered include: The Berkeley Sockets API The TCP protocol and the IO::Socket API FTP filesharing service The Net::Telnet module for adapting clients to interactive network services SMTP, including how to create and send e-mails with multimedia attachments POP, IMAP, and NNTP for receiving and processing e-mail HTTP and the LWP module for communicating with Web servers Forking servers and the UNIX and Windows inetd daemons Perl's experimental multithreaded API Multiplexed operations and nonblocking I/O Bulletproofing servers TCP urgent data UDP protocol and servers Broadcasting and multicasting Interprocess communication with UNIX-domain sockets Useful, working programs demonstrate ideas and techniques in action, including a real-time chat and messaging system, a program for processing e-mail containing MIME attachments, a program for mirroring an FTP site, and a Web robot. Network Programming with Perl focuses on TCP/IP rather than just the common Web protocols. Modeled after the critically acclaimed TCP/IP Illustrated by W. Richard Stevens, this book achieves a level of detail far superior to most. It is an essential resource for network administrators and Perl programmers who are creating network applications. 0201615711B04062001

Author Biography

Lincoln Stein has an M.D. and is a scientist at Cold Spring Harbor Laboratory. When the Web first emerged, he created and maintained one of the earliest Internet sites for distribution of Human Genome Project data and has since become an acknowledged expert in Web, network, and Perl programming. Known for his exceptional ability to synthesize and present complex information, he writes for The Perl Journal and Web Techniques magazines and is the author of four other books.

0201615711AB04062001

Table of Contents

Preface	xi
PART I BASICS	1	(130)
Input/Output Basics	3	(32)
Perl and Networking	3	(2)
Networking Made Easy	5	(2)
Filehandles	7	(19)
Using Object-Oriented Syntax with the IO::Handle and IO::File Modules	26	(7)
Summary	33	(2)
Processes, Pipes, and Signals	35	(24)
Processes	35	(4)
Pipes	39	(10)
Signals	49	(9)
Summary	58	(1)
Introduction to Berkeley Sockets	59	(28)
Clients, Servers, and Protocols	59	(3)
Berkeley Sockets	62	(6)
Scoket Addressing	68	(6)
A Simple Network Client	74	(2)
Network Names and Services	76	(5)
Network Analysis Tools	81	(4)
Summary	85	(2)
The TCP Protocol	87	(18)
A TCP Echo Client	87	(3)
Socket Functions Related to Outgoing Connections	90	(1)
A TCP Echo Server	91	(6)
Adjusting Socket Options	97	(3)
Other Socket-Related Functions	100	(2)
Exceptional Conditions during TCP Communications	102	(2)
Summary	104	(1)
The IO::Socket API	105	(26)
Using IO::Socket	105	(3)
IO::Socket Methods	108	(7)
More Practical Examples	115	(5)
Performance and Style	120	(1)
Concurrent Clients	121	(7)
Summary	128	(3)
PART II DEVELOPING CLIENTS FOR COMMON SERVICES	131	(166)
FTP and Telent	133	(30)
Net::FTP	133	(14)
Net::Telnet	147	(15)
Summary	162	(1)
SMTP: Sending Mail	163	(40)
Introduction to the Mail Modules	163	(1)
Net::SMTP	164	(5)
MailTools	169	(10)
MIME-Tools	179	(23)
Summary	202	(1)
POP, IMAP, and NNTP: Processing Mail and Netnews	203	(42)
The Post Office Protocol	203	(16)
The IMAP Protocol	219	(5)
Internet News Clients	224	(11)
A News-to-Mail Gateway	235	(8)
Summary	243	(2)
Web Clients	245	(52)
Installing LWP	245	(2)
LWP Basics	247	(13)
LWP Examples	260	(17)
Parsing HTML and XML	277	(19)
Summary	296	(1)
PART III DEVELOPING TCP CLIENT/SERVER SYSTEMS	297	(202)
Forking Servers and the inetd Daemon	299	(32)
Standard Techniques for Concurrency	299	(2)
Running Example: A Psychotherapist Server	301	(2)
The Psychotherapist as a Forking Server	303	(6)
A Client Script for the Psychotherapist Server	309	(3)
Daemonization on UNIX Systems	312	(6)
Starting Network Servers Automatically	318	(4)
Using the inetd Super Daemon	322	(6)
Summary	328	(3)
Multithreaded Applications	331	(14)
About Threads	331	(8)
A Multithreaded Psychiatrist Server	339	(3)
A Multithreaded Client	342	(1)
Summary	343	(2)
Multiplexed Applications	345	(14)
A Multiplexed Client	345	(2)
The IO::Select Module	347	(5)
A Multiplexed Psychiatrist Server	352	(6)
Summary	358	(1)
Nonblocking I/O	359	(50)
Creating Nonblocking I/O Handles	359	(3)
Using Nonblocking Handles	362	(2)
Using Nonblocking Handles with Line-Oriented I/O	364	(6)
A Generic Nonblocking I/O Module	370	(24)
Nonblocking Connects and Accepts	394	(14)
Summary	408	(1)
Bulletproofing Servers	409	(38)
Using the System Log	409	(16)
Setting User Privileges	425	(5)
Taint Mode	430	(3)
Using chroot()	433	(3)
Handling HUP and Other Signals	436	(10)
Summary	446	(1)
Preforking and Prethreading	447	(42)
Preforking	447	(30)
Prethreading	477	(9)
Performance Measures	486	(1)
Summary	487	(2)
IO::Poll	489	(10)
Using IO::Poll	489	(4)
IO::Poll Methods	493	(1)
A Nonblocking TCP Client Using IO::Poll	493	(4)
Summary	497	(2)
PART IV ADVANCED TOPICS	499	(152)
TCP Urgent Data	501	(26)
``Out-of-Band'' Data and the Urgent Pointer	501	(2)
Using TCP Urgent Data	503	(6)
The sockatmark() Function	509	(3)
A Traesty Server	512	(14)
Summary	526	(1)
The UDP Protocol	527	(24)
A Time of Day Client	527	(2)
Creating and Using UDP Sockets	529	(3)
UDP Errors	532	(1)
Using UDP Sockets with IO::Socket	533	(2)
Sending to Multiple Hosts	535	(3)
UDP Servers	538	(3)
Increasing the Robustness of UDP Applications	541	(9)
Summary	550	(1)
UDP Servers	551	(34)
An Internet Chat System	551	(4)
The Chat Client	555	(10)
The Chat Server	565	(11)
Detecting Dead Clients	576	(7)
Summary	583	(2)
Broadcasting	585	(22)
Unicasting versus Broadcasting	585	(1)
Broadcasting Explained	586	(2)
Sending and Receiving Broadcasts	588	(4)
Broadcasting Without the Broadcast Address	592	(11)
Enhancing the Chat Client to Support Resource Discovery	603	(3)
Summary	606	(1)
Multicasting	607	(30)
Multicast Basics	607	(7)
Using Multicast	614	(8)
Sample Multicast Applications	622	(14)
Summary	636	(1)
UNIX-Domain Sockets	637	(14)
Using UNIX-Domain Sockets	637	(5)
A ``Wrap'' Server	642	(4)
Using UNIX-Domain Sockets for Datagrams	646	(4)
Summary	650	(1)
Appendix A Additional Source Code	651	(24)
Net::NetmaskLite (Chapter 3)	651	(3)
PromptUtil.pm (Chapter 8 and 9)	654	(2)
IO::LineBufferedSet (Chapter 13)	656	(3)
IO::LineBufferedSessionData (Chapter 13)	659	(5)
DaemonDebug (Chapter 14)	664	(2)
Text::Travesty (Chapter 17)	666	(3)
mchat_client.pl (Chapter 21)	669	(6)
Appendix B Perl Error Codes and Special Variables	675	(8)
System Error Constants	675	(4)
Magic Variables Affecting I/O	679	(2)
Other Perl Globals	681	(2)
Appendix C Internet Reference Tables	683	(38)
Assigned Port Numbers	683	(20)
Registered Port Numbers	703	(16)
Internet Multicast Addresses	719	(2)
Appendix D Bibliography	721	(6)
Index	727

Excerpts

The network is everywhere. At the office, machines are wired together into local area networks, and the local networks are interconnected via the Internet. At home, personal computers are either intermittently connected to the Internet, or, increasingly, "always-on" cable and DSL modems. New wireless technologies, such as Bluetooth, promise to vastly expand the network realm, embracing everything from cell phones to kitchen appliances. Such an environment creates tremendous opportunities for innovation. Whole new classes of applications are now predicated on the availability of high-bandwidth, always-on connectivity. Interactive games allow players from around the globe to compete on virtual playing fields and the instant messaging protocols let them broadcast news of their triumphs to their friends. New peer-to-peer systems, such as Napster and Gnutella, allow people to directly exchange MP3 audio files and other types of digital content. The SETI@Home project takes advantage of idle time on the millions of personal computers around the world to search for signs of extraterrestrial life in a vast collection of cosmic noise. The ubiquity of the network allows for more earthbound applications as well. With the right knowledge, you can write a robot that will fetch and summarize prices from competitors'' Web sites; a script to page you when a certain stock drops below a specified level; a program to generate daily management reports and send them off via e-mail; a server that centralizes some number-crunching task on a single high-powered machine, or alternatively distributes that task among the multiple nodes of a computer cluster. Whether you are searching for the best price on a futon or for life in a distant galaxy, you''ll need to understand how network applications work in order to take full advantage of these opportunities. You''ll need a working understanding of the TCP/IP protocol--the common denominator for all Internet-based communications and the most common protocol in use in local area networks as well. You''ll need to know how to connect to a remote program, to exchange data with that program, and what to do when something goes wrong. To work with existing applications, such as Web servers, you''ll have to understand how the application-level protocols are built on top of TCP/IP, and how to deal with common data exchange formats such as XML and MIME. This book uses the Perl programming language to illustrate how to design and implement practical network applications. Perl is an ideal language for network programming for a number of reasons. First, like the rest of the language, Perl''s networking facilities were designed to make the easy things easy. It takes just two lines of code to open a network connection to a server somewhere on the Internet and send it a message. A fully capable Web server can be written in a few dozen lines of code. Second, Perl''s open architecture has encouraged many talented programmers to contribute to an ever-expanding library of useful third-party modules. Many of these modules provide powerful interfaces to common network applications. For example, after loading the LWP::Simple module, a single function call allows you to fetch the contents of a remote Web page and store it in a variable. Other third-party modules provide intuitive interfaces to e-mail, FTP, net news, and a variety of network databases. Perl also provides impressive portability. Most of the applications developed in this book will run without modification on UNIX machines, Windows boxes, Macintoshes, VMS systems, and OS/2. However, the most compelling reason to choose Perl for network application development is that it allows you to fully exploit the power of TCP/IP. Perl provides you with full access to the same low-level networking calls that are available to C programs and other natively compiled languages. You can create multicast applications, implement multiplexed servers, and design peer-to-peer systems. Using Perl, you can rapidly prototype new networking applications and develop interfaces to existing ones. Should you ever need to write a networking application in C or Java, you''ll be delighted to discover how much of the Perl API carries over into these languages. This Book''s Audience Network Programming with Perlis written for novice and intermediate Perl programmers. I assume you know the basics of Perl programming, including how to write loops, how to construct if-else statements, how to write regular expression pattern matches, the concept of the automatic$_variable, and the basics of arrays and hashes. You should have access to a Perl interpreter and some experience writing, running, and debugging scripts. Just as important, you should have access to a computer that is connected both to a local area network and to the Internet! Although the recipes in Chapter 10 on setting Perl-based network servers to start automatically when a machine is booted do require superuser (administrative) access, none of the other examples require privileged access to a machine. This book does take advantage of the object-oriented features in Perl version 5 and higher, but most chapters do not assume a deep knowledge of this system. Chapter 1 addresses all the details you will need as a casual user of Perl objects. This book is a thorough review of the TCP/IP protocol at the lowest level, or a guide to installing and configuring network hubs, routers, and name servers. Many good books on the mechanics of the TCP/IP protocol and network administration are listed in the references in Appendix D. Roadmap This book is organized into four main parts, Basics, Developing Cients for Common Services, Developing TCP Client/Server Systems, and Advanced Topics. Part I, Basics, introduces the fundamentals of TCP/IP network communications. Chapters 1 and 2, Networking Basicsand Processes, Pipes, and Signals, review Perl''s functions and variables for input and output, discuss the exceptions that can occur during I/O operations, and use the piped filehandle as the basis for introducing sockets. These chapters also review Perl''s process model, including signals and forking, and introduce Perl''s object-oriented extensions. Chapter 3, Introduction to Berkeley Sockets, discusses the basics of Internet networking and describes IP addresses, network ports, and the principles of client/server applications. It then turns to the Berkeley Socket API, which provides the programmer''s interface to TCP/IP. Chapters 4 and 5, T he TCP Protocoland The IO::Socket API and Simple TCP Applications, show the basics of TCP, the networking protocol that provides reliable stream-oriented communications. These chapters demonstrate how to create client and server applications and then introduce examples that show the power of technique as well as some common roadblocks. Part II, Developing Clients for Common Services, looks at a collection of the best third-party modules that developers have contributed to the Comprehensive Perl Archive Network (CPAN). Chapter 6, FTP and Telnet, introduces modules that provide access to the FTP file-sharing service, as well as to the flexible Net::Telnet module which allows you to create clients to access all sorts of network services. E-mail is still the dominant application on the Internet, and Chapter 7, SMTP: Sending Mail, introduces half of the equation. This chapter shows you how to create e-mail messages on the fly, including binary attachments, and send them to their destinations. Chapte