Unix pipes

Recently, I was asked a question about how Unix pipes works. This post is meant to clarify some of the most important points on how pipes are set up between two processes. Traditional Unix pipes consist of two file descriptors, one representing the “read end” of the pipe and the other representing the “write end” of the pipe. A program writes to the pipe by writing to the file descriptor representing the write end of the pipe, and reads elements from the pipe by reading from the file descriptor representing the read end of a pipe.

A question now arises: how are these file descriptors shared between two processes to allow them to communicate with each other? The answer to this question lies in the way a Unix process is created. I describe this process in detail here and here. Essentially, a new process is created by first calling the fork() system call to make a copy of the existing process. The fork() call creates a new process that has a copy of the resources, including open file descriptors, of the existing process. Next, a call to exec is made to overlay the address space of the new process with the image of the binary file representing the new program. Although all of the data of the old process is replaced after the call to exec, file descriptors remain open unless they are marked as close-on-exec.

So, using these system call semantics, we can use pipes to communicate between two processes as follows; The first process creates a pipe, thus, creating two file descriptors representing the read and write ends of the pipe as mentioned above. Next, the process calls fork and exec to start the second process. The first process should not set close-on-exec on the file descriptors representing the pipe. Once the exec function completes, we have two processes that share the file descriptors representing the pipe. They can now communicate by reading and writing to the pipe.