UNIX Systems Programming

Chapter Summaries

Chapter 1: Technology's Impact on Programs

This chapter introduces the ideas of communication, concurrency and asynchronous operation at the operating system level and at the application level. Handling such program constructs incorrectly can lead to failures with no apparent cause, even for input that previously seemed to work perfectly. Besides their added complexity, many of today's applications run for weeks or months, so they must properly release resources to avoid waste (so-called leaks of resources). Applications must also cope with outrageously malicious user input, and they must recover from errors and continue running. The Portable Operating System Interface (POSIX) standard is an important step toward producing reliable applications. Programmers who write for POSIX-compliant systems no longer need to contend with small but critical variations in the behavior of library functions across platforms. Most popular UNIX versions (including Linux and Mac OS X) are rapidly moving to support the base POSIX standard and various levels of its extensions.

Chapter 2: Programs, Processes and Threads

One popular definition of a process is an instance of a program whose execution has started but has not yet terminated. This chapter discusses the differences between programs and processes and the ways in which the former are transformed into the latter. The chapter addresses issues of program layout, command-line arguments, program environment and exit handlers.

Chapter 3: Processes in UNIX

A process is the basic active entity in most operating-system models. This chapter covers the UNIX process model, including process creation, process destruction and daemon processes. The chapter uses process fans and process chains to illustrate concepts of parentage, inheritance and other process relationships. The chapter also looks at the implications of critical sections in concurrent processes.

Chapter 4: UNIX I/O

UNIX uses a uniform device interface, through file descriptors, that allows the same I/O calls to be used for terminals, disks, tapes, audio and even network communication. This chapter explores the five functions that form the basis for UNIX device-independent I/O. The chapter also examines I/O from multiple sources, blocking I/O with timeouts, inheritance of file descriptors and redirection. The code carefully handles errors and interruption by signals.

Chapter 5: Files and Directories

Operating systems organize raw storage devices in file systems so that applications can use high-level operations rather than low-level device calls to access information. UNIX file systems are tree structured, with nodes representing files and arcs representing the contains relationship. UNIX directory entries associate filenames with file locations. These entries can either point directly to a structure containing the file location information (hard link) or point indirectly through a symbolic link. Symbolic links are files that associate one filename with another. This chapter also introduces functions for accessing file status information and directories from within programs.

Chapter 6: UNIX Special Files

This chapter discusses UNIX special files that represent devices. Two important examples of special files are pipes and FIFOs, interprocess communication mechanisms that allow processes running on the same system to share information and hence cooperate. The chapter introduces the client-server model and also discusses how to handle special files representing devices such as terminals.

Chapter 7: Project: The Token Ring

The projects of this chapter explore pipes, forks and redirection in the context of a ring of processes. Such a ring allows simple and interesting simulations of ring network topologies. The chapter also introduces fundamental ideas of distributed processing, including processor models, pipelining and parallel computation. Distributed algorithms such as leader election illustrate important implementation issues.

Chapter 8: Signals

Few people appreciate the insidious nature of asynchronous events until they encounter a problem that is difficult to reproduce. This chapter discusses signals and their effect on processes, emphasizing the concurrent aspects of signal handling. The chapter begins by defining basic signal concepts such as generation and delivery as well as explaining the difference between ignoring a signal and blocking a signal. Sample programs demonstrate how to use signals for notification and how to suspend a process while waiting for a signal. The chapter also covers error handling, signal safety and asynchronous I/O.

Chapter 9: Times and Timers

Operating systems use timers for purposes such as process scheduling, timeouts for network protocols, and periodic updates of system statistics. Applications access system time and timer functions to measure performance or to identify the time when events occur. Applications also use timers to implement protocols and to control interaction with users such as that needed for rate-limited presentations. This chapter discusses representations of time in the POSIX base standard as well as interval timers in the POSIX:XSI Extension and POSIX:TMR Extension. The chapter also explores concepts such as timer drift and timer overrun and demonstrates how to use POSIX realtime signals with timers.

Chapter 10: Project: Virtual Timers

Many systems create multiple ``virtual'' timers from a single hardware timer. This chapter develops application-level virtual timers based on a single operating system timer. The project explores timers, signals and the testing of asynchronous programs with timed input. Special care must be taken in blocking and unblocking the signals at the right times. The project emphasizes careful, modular design by specifying a well-defined interface between the user-implemented virtual timers and the underlying timer facility.

Chapter 11: Project: Cracking Shells

By developing a shell from the bottom up, this chapter explores the intricacies of process creation, termination, identification and the correct handling of signals. Example programs handle foreground and background processes, pipelines, process groups, sessions and controlling terminals. The chapter also looks at job control and terminal I/O. The closing project integrates these concepts by incorporating job control into a shell.

Chapter 12: POSIX Threads

One method of achieving parallelism is for multiple processes to cooperate and synchronize through shared memory or message passing. An alternative approach uses multiple threads of execution in a single address space. This chapter explains how threads are created, managed and used to solve simple problems. The chapter then presents an overview of basic thread management under the POSIX standard. The chapter discusses different thread models and explains how these models are accommodated under the standard.

Chapter 13: Thread Synchronization

POSIX supports mutex locks for short-term locking and condition variables for waiting on events of unbounded duration. Signal handling in threaded programs presents additional complications that can be reduced if signal handlers are replaced with dedicated threads. This chapter illustrates these thread synchronization concepts by implementing controlled access to shared objects, reader-writer synchronization and barriers.

Chapter 14: Critical Sections and Semaphores

Programs that manage shared resources must execute portions of code called critical sections in a mutually exclusive manner. This chapter discusses how critical sections arise and how to protect their execution by means of semaphores. After presenting an overview of the semaphore abstraction, the chapter describes POSIX named and unnamed semaphores. The closing section outlines a license manager project based on semaphores.

Chapter 15: POSIX IPC

UNIX interprocess communication (IPC) mechanisms of shared memory, message queues and semaphore sets are standardized in the POSIX:XSI Extension. These mechanisms, which allow unrelated processes to exchange information in a reasonably efficient way, use a key to identify, create or access the corresponding entity. The entities may persist in the system beyond the lifetime of the process that creates them, but conveniently, POSIX:XSI also provides shell commands to list and remove them.

Chapter 16: Project: Producer Consumer Synchronization

This chapter focuses on variations of producer-consumer synchronization using mutex locks, semaphores, condition variables and signals. Implementations for different types of stopping conditions are developed with careful attention to error handling and shutdown. The chapter describes two projects, a parallel file copy and a print server. The parallel file copy uses bounded buffers; the print server uses unbounded buffers.

Chapter 17: Project: The Not Too Parallel Virtual Machine

PVM (Parallel Virtual Machine) provides a high-level, but not transparent, system for a user to coordinate tasks spread across workstations on a network. This project describes a threaded implementation of the Not Too Parallel Virtual Machine (NTPVM) dispatcher, a simplified PVM system. The multithreaded implementation illustrates the interaction between threads and fork, providing a semirealistic application in which to explore complex thread interactions.

Chapter 18: Connection-Oriented Communication

Most local-area networks have file servers that manage common disk space, making it easier to share files and perform backups for user clients. Standard UNIX network services such as mail and file transfer also use the client-server paradigm. This chapter discusses several common client-server models for providing services over existing network infrastructure. The models are implemented with the Universal Internet Communication Interface (UICI), a simplified API for connection-oriented communication that is freely available from the book web site. The UICI interface is then implemented in terms of stream sockets and TCP.

Chapter 19: Project: WWW Redirection

The World Wide Web has a client-server architecture based on a resource identification scheme (URI), a communication protocol (HTTP) and a document format (HTML), which together allow easy access and exchange of information. The decentralized nature of the Web and its effectiveness in making information accessible have led to fundamental social and cultural change. Every product, from breakfast cereal to cars, has a presence on the Web. Businesses and other institutions have come to regard the Web as an interface, even the primary interface, with their customers. By providing ubiquitous access to information, the Web has reduced barriers erected by geographic and political borders in a profound way.

Chapter 20: Connectionless Communication and Multicast

In unreliable connectionless communication, single messages are transmitted between sender and receiver. A message may or may not arrive correctly at its destination. While such communication has low overhead, it requires that the application manage errors. This chapter expands the UICI library to include facilities for connectionless communication with timeouts and error checking. The chapter develops applications of the simple-request and request-reply protocols based on the connectionless interface. The UICI connectionless interface is then implemented with sockets, using UDP. UICI UDP also includes functions for multicast communication.

Chapter 21: Project: Internet Radio

Broadcast, telephone and network technologies are converging rapidly, blurring the distinction between telephone and television. Software for video and telephone conferencing on the Internet is widely available, and most cable companies now offer high-speed Internet connections. Telephone companies have entered the entertainment business with video-on-demand and content services. The final resolution of these competing forces will probably be determined by politics and regulatory decisions as well as by technical merit. Whatever the outcome, more computers will handle voice, audio and video streams in addition to data. This chapter explores how well network communication protocols such as UDP, multicast and TCP support streaming media applications. The chapter outlines a project to send audio streams over a network under various conditions. The project explores timing, buffering and packet loss, as well as synchronization and a dynamic number of receivers.

Chapter 22: Project: Server Performance

Large-scale client-server architecture is ubiquitous on the Internet. Web sites may service thousands of simultaneous clients, with individual servers processing hundreds of clients. Parallelism can be achieved by multiple processes, by multiple threads within a process, by asynchronous I/O and events within a single process thread or by combinations of these approaches. This chapter explores the interaction of threading, forking, network communication and disk I/O on the performance of servers.