CS644 week 4: Interprocess communication

Last week

Link: /week3

Review

Concepts: Programs and processes, user and group IDs, forking and executing
Syscalls: os.fork, os.execve, os.waitpid, os._exit

Solutions

Re-implementing subprocess.run: week3/run.py

Pipes

Last week we learned how to create new processes. But, aside from waitpid, the processes couldn't interact with each other. This week, we'll learn some techniques for communicating between different processes – interprocess communication, or IPC.

One of the simplest IPC primitives is the pipe. A pipe is a one-way data stream. The writer writes to one end and the reader reads from the other end (and there can be multiple writers and readers, though one of each is the typical case).

You may have heard of pipes in the context of shell scripting, such as ps aux | grep myprocess. The pipes we are talking about are a lower-level OS primitive. In the in-class exercise, we'll see how shell pipelines are implemented using OS pipes.

To create a pipe, call os.pipe:

# https://docs.python.org/3/library/os.html#os.pipe
os.pipe() -> Tuple[int, int]

os.pipe returns a pair of file descriptors, one for each end of the pipe: (read, write). (There is also an os.pipe2 function that supports a couple of extra flags.)

Exercise: Pipe edge cases

Exercise: Write example programs to answer the following questions:

What does read return when nothing has been written to the pipe?
What does read(fd, 100) return when less than 100 bytes have been written to the pipe?
What does read return when the write end of the pipe is closed?
What does write return when the read end of the pipe is closed?
Is there a limit to how much you can write to a pipe without reading anything?
Bonus: Can you show whether writes to a pipe are atomic in the presence of multiple writers?

Exercise: Pipes and `fork`

You can use both ends of the pipe in the same process, and occasionally that is useful. But normally you want one end of the pipe in one process and the other end in another. Solution: call pipe2, then fork. Because file descriptors are copied on fork, both the child process and the parent process will have references to the read and write ends of the pipe (each process can close the end it doesn't need).

Exercise: Use a pipe to read the standard output of a child process into a string in the parent process.

Follow-up

Spawn two child processes and use a pipe to direct the standard output of one into the standard input of the other (i.e., create the equivalent of a shell pipeline).

Exercise: FIFOs

Pipes work best between a parent and child process. If you want similar semantics but between unrelated processes (e.g., some daemon process and a shell program), you can use FIFOs (for "first-in, first-out"; also known as named pipes). FIFOs are functionally identical to pipes except that they have an entry in the filesystem so that unrelated processes can discover and open them.

Exercise: Use the os.mkfifo function (or os.mknod, which is the underlying syscall) to create a FIFO, and read the bytes written to. Run echo test > my.fifo in another shell to demonstrate.

Unix domain sockets

Pipes and FIFOs are sufficient for simple, one-way IPC, but are inconvenient for complex IPC between a server and many clients.

Enter Unix domain sockets. Unix domain sockets allow for processes on the same machine to pass data to each other using a path on the filesystem, similar to FIFOs. Unlike FIFOs, the communication is full-duplex: both ends can read and write separately (i.e., if process A writes to the sock)

Unix domain sockets use the sockets API, which is the same API that is used for networking (as we shall see in coming weeks). Unix domain sockets are not networking, though – they only pass data between processes on the same machine.

The server program follows a fixed sequence:

Create the socket: socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
- AF_UNIX creates the socket as a Unix domain socket.
- SOCK_STREAM creates the socket with a connection-oriented, stream interface. An alternative is SOCK_DGRAM for a connectionless, datagram interface.
Bind the socket to a path on the filesystem: sock.bind("/path/to/my.sock")
Start listening for new connections (this does not block): sock.listen()
Accept an incoming connection (this does block): sock.accept(), which returns a tuple (conn, addr) where conn is a socket.socket object unique to that connection. addr is blank for Unix domain sockets.

sock = socket.socket(...)
sock.bind(...)
sock.listen()
while True:
    conn, _ = sock.accept()
    handle_connection(conn)

The Python interface is object-oriented – sock is an instance of the socket.socket class – but the sequence of syscalls and arguments is near-identical to C.

If you've ever done web development, this may seem very similar to having a server bind to a port on localhost. The key differences are (a) Unix domain sockets use file paths instead of port numbers, and (b) the data transfer does not go through the networking stack at all, while localhost servers go through the kernel's loopback network interface.

Clients merely need to create the socket and call connect:

sock = socket.socket(...)
sock.connect(path)

Exercise: UDS client and server

Exercise: Write a simple UDS server that receives connections in a loop, reads the first 4,096 bytes of data, writes the data reversed (you can do this with data[::-1] in Python), and closes the connection. Write a client program that connects to the server, writes some data, and prints what it got back.

You can use the regular os.read and os.write calls to work with sockets (call fileno() on the socket object to get the underlying file descriptor), though there are also specialized socket.send and socket.recv functions that expose socket-specific functionality.

Exercise: UDS edge cases