Chapter 6 - Unix special files

Pipes, fifos, client/server models using special files, and terminals (6.5).

6.1-6.2 Pipes

Simple pipe

The first (oldest) method of IPC in Unix was the pipe (which you've all used via the pipe symbol). The pipe function creates an internal buffer in the OS that processes can access via file descriptors:

SYNOPSIS
       #include < unistd.h>

       int pipe(int filedes[2]);

DESCRIPTION
       pipe  creates a pair of file descriptors, pointing to a pipe inode, and
       places them in the array pointed to  by  filedes.   filedes[0]  is  for
       reading, filedes[1] is for writing.

RETURN VALUE
       On  success,  zero is returned.  On error, -1 is returned, and errno is
       set appropriately.

ERRORS
       EMFILE Too many file descriptors are in use by the process.

       ENFILE The system file table is full.

       EFAULT filedes is not valid.

A pipe is transient. Typically, a pipe is set up by a parent which then does a fork so that two (or more) processes are sharing a pipe. When a process reads from a pipe with something in it, it returns immediately. If the pipe is empty BUT someone has the pipe open for writing, a read blocks. If no process has the pipe open for writing, the read returns 0 indicating an EOF.

Note that on first impression processes on either end can both read and write the pipe. But in fact, while each end has their own fd[0] and fd[1] pair for reading and writing (respectively), these are copies pointing to the same system file table entries as discused in USP Chapter 4. Consequently, pipes can only be used in one direction. As we will see below, piping in the other direction needs to be closed out (please forgive grammar).

In parentwritepipe.c we have a simple pipe that just has the parent writing to a child via a pipe:

[06:21:06]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./parentwritepipe
[29006]:my bufin is {empty}, my bufout is {hello}
[06:21:09]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ [29007]:my bufin is {hello}, my bufout is {hello}
fg

(29007 is the child, it read the pipe and received bufin="hello". Note that
reads are not atomic and it's possible to return from a pipe with only some
of the bytes read.

A copyfile pipe

Suppose that we alter the code so that instead of writing a buffer to a pipe and reading from a pipe in the child we do:

if (childpid)
	copyfile(STDIN_FILENO, fd[1]); // parent reads stdin and writes to pipe
else
	copyfile(fd[0], STDOUT_FILENO); // child reads pipe and write to stdout

Here is the run parentwritepipe1.c:

[06:28:35]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./parentwritepipe1
hello
hello
my
my
name
name
is sherri
is sherri
[29025]: my bufin is {empty}, my bufout is {hello}
[06:28:47]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ps
  PID TTY          TIME CMD
28550 ttyp2    00:00:00 bash
28671 ttyp2    00:00:00 vi
28969 ttyp2    00:00:00 vi
29026 ttyp2    00:00:00 parentwritepipe
29027 ttyp2    00:00:00 ps
[06:28:50]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ fg

Notice that a process is still running. This is because the last child read blocks; why?. The child blocks because the child's fd[1] write descriptor is still open (even though the child was never using it). The parent has detected the EOF from stdin, displayed its fprintf message, and exited with no problem, closing its descriptors fd[0] and fd[1] to the pipe. But the child write descriptor is still open so the child read blocks. We can modify the code with the following standard practice: the parent and the child each close the fd[n] part of the pipe they are NOT using. See parentwritepipe2.c Now the child read will return with an EOF rather than block because the child has closed its write descriptor and the parent write pipe was closed after the parent detected the EOF on the stdin.

Question: can a pipe be used both directions? Try to write a test program in which the parent writes to the pipe, the child reads from the pipe and then the child turns around and writes to the pipe and the parent reads from the pipe.

A barier architecture using a pipe

Pipes can be used for synchronization. Suppose that as a parent we want to start n-1 processes and have them wait for some event before proceeding. This architecture is called a barrier. To illustrate this, we can have a parent open a pipe and then fork off n-1 child processes. Then we (parent) will write n characters to the pipe (one for each child plus the parent). Each process, including the parent, reads 1 character from the pipe before proceeding to output its information to stderr. Since each each child read (and the parent) blocks until the parent writes to the pipe, no process continues until all the processes have been created. The barrier 'event' in this case is the successful completion of the creation of all the children by the parent. synchronizefan.c

[06:41:58]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./synchronizefan 10
i:1  process ID:29063  parent ID:29062  child ID:0
i:2  process ID:29064  parent ID:29062  child ID:0
i:3  process ID:29065  parent ID:29062  child ID:0
i:4  process ID:29066  parent ID:29062  child ID:0
i:5  process ID:29067  parent ID:29062  child ID:0
i:6  process ID:29068  parent ID:29062  child ID:0
i:10  process ID:29062  parent ID:28550  child ID:29071
[06:42:02]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ i:7  process ID:29069  parent ID:1  child ID:0
i:9  process ID:29071  parent ID:1  child ID:0
i:8  process ID:29070  parent ID:1  child ID:0

Once the barrier is passed, the children all proceed independently. In this case, they do nothing (independently).

A Unix pipeline is achieved by setting up the appropriate pipes so that each process is writing to the stdin of the next process. simpleredirect.c sets up a pipe and then executes the equivalent of "ls -l | sort -n +4":

Note - The sort command as shown won't work on ada. You have to use the command "sort -n -k4". That means you have to change the C source code as well in the execl system call.

The first thing it does is creat a pipe and then a fork. So we have two processes and a pipe that can be shared.
The child will exectue the ls -l, the parent will execute the sort -n +4

The child dups the write port (fd[1]) and make STDOUT_FILENO be a copy of fd[1]. It closes STDOUT_FILENO first if necessay. The result of this is that the stdout for the child is now fd[1], so any writes that use the standard out automatically go to the pipe.

SYNOPSIS
       #include < unistd.h>

       int dup(int oldfd);
       int dup2(int oldfd, int newfd);

DESCRIPTION
       dup and dup2 create a copy of the file descriptor oldfd.

       After successful return of dup or dup2, the old and new descriptors may
       be used interchangeably. They share locks, file position  pointers  and
       flags;  for example, if the file position is modified by using lseek on
       one of the descriptors, the position is also changed for the other.

       The two descriptors do not share the close-on-exec flag, however.

       dup uses the lowest-numbered unused descriptor for the new  descriptor.

       dup2  makes  newfd  be the copy of oldfd, closing newfd first if neces-
       sary.

The child then closes fd[0] and fd[1], since we only need the dup'd copy, and then exec the "ls -l". Note that "ls -l" doesn't know it will be writing to a pipe. It just writes to the stdout as usual.

The parent does the same except it will dup the STDIN_FILENO since it will execute the "sort -n +4" and needs to alter where it reads from.
What happens if we don't close the extra file descriptors (fd[0], fd[1])? The sort process would not detect the EOF cause the file would still be open so would block indefinitely.

6.3 Fifos

Pipes are handy but they are not persistent. About a decade after pipes were introduced, pipes were 'generalized' so that they could exist persistently: the were called named pipes or FIFO's. To create a fifo, use the system call mkfifo:

SYNOPSIS
       #include < sys/types.h>
       #include < sys/stat.h>

       int mkfifo(const char *pathname, mode_t mode);

DESCRIPTION
       mkfifo  makes  a  FIFO special file with name pathname.  mode specifies
       the FIFO's permissions. It is modified by the process's  umask  in  the
       usual way: the permissions of the created file are (mode & ~umask).

       A  FIFO special file is similar to a pipe, except that it is created in
       a different way.  Instead of being an anonymous communications channel,
       a  FIFO special file is entered into the file system by calling mkfifo.

       Once you have created a FIFO special file in this way, any process  can
       open  it  for  reading or writing, in the same way as an ordinary file.
       However, it has to be open at both ends simultaneously before  you  can
       proceed to do any input or output operations on it.  Opening a FIFO for
       reading normally blocks until some other process opens  the  same  FIFO
       for  writing,  and vice versa. See fifo(4) for non-blocking handling of
       FIFO special files.

To remove a fifo, use the unlink system call:

SYNOPSIS
       #include < unistd.h>

       int unlink(const char *pathname);

DESCRIPTION
       unlink  deletes  a  name from the filesystem. If that name was the last
       link to a file and no processes have the file open the file is  deleted
       and the space it was using is made available for reuse.

       If  the  name  was the last link to a file but any processes still have
       the file open the file will remain in existence  until  the  last  file
       descriptor referring to it is closed.

       If the name referred to a symbolic link the link is removed.

       If  the  name  referred  to a socket, fifo or device the name for it is
       removed but processes which have the object open may  continue  to  use
       it.

So now we can make a fifo: parentchildfifo.c . We can then execute two processes (a child and a parent) to independently open and read/write the named pipe. The child dofifochild.c opens and writes to the fifo. The parent dofifoparent.c opens and reads the fifo.

Note - If you can't get this to work because it gives an error trying to create the pipe1 in your working directory, then use "/tmp/pipe1" for your pipe. (Now explain that one!). It has to do with using the Windows file system which doesn't understand pipes.

[07:39:25]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./parentchildfifo pipe1
[29244]:(child) about to open FIFO pipe1...
[29243]:(parent) about to open FIFO pipe1...
[29243]:about to read...
[29244]:about to write...
[29244]:finishing...
[29243]:read [29244]:this was written by the child

[07:39:29]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ls -l pipe1
prw-------  1 sherri faculty 0 Mar  8 07:39 pipe1|
[07:40:43]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$

We can run this program again ... it won't have to make the pipe since it already exists. If we want to get rid of the pipe, we need to either unlink in the process or remove it from the shell.

6.4 Client-Server architecture using Pipes

The client/server model is a standard (and common) design for process interaction. One process (the client) requests something from another (the server). The architecture doesn't specify the mechanism by which a client requests a service, nor the mechanism by which a server supplies the services. In this chapter we can introduce it with the pipe (or fifo) as the mechanism. The first case we'll look at can be seen as a support for logging. Logging from multiple sources has the property that we'd like each source write to be atomic. Not all IO is guaranteed to be atomic (eg fprintf is not atomic). There are a variety of complex protocols that can (and have been) designed to address how to use a common resource such as a common log. However, using FIFO lets us use the atomic property of FIFO writes: writes of up to PIPE_BUF size bytes are guaranteed to be atomic. So you can set up a server that reads from a named pipe and then writes to a file.

Now multiple clients can write their logging information to a named pipe and know that their writes are atomic (within the size restriction). Then the single server and read the pipe to write to the logfile. pipeserver.c. creates the pipe if it doesn't exist and then copies the pipe contents to the standard output. (To write it to a log either use redirection or change the program to designate a logfile). The server opens the fifo for reading and writing so that the open will not block (otherwise, if just open for reading, the open will block until a writer is present. ( Opening a FIFO for reading normally blocks until some other process opens the same FIFO for writing, and vice versa. See fifo(4) for non-blocking handling of FIFO special files.)

Now we can make a client: pipeclient.c . Now these two processes are independent. They both are main programs and are started independently. The ony connection they have is that the server reads 'requests' from the indicated pipe and the clien writes 'requests' to the indicated pipe. We can start it by:

[07:56:30]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeserver pipe1 > log1 &
[2] 29371
[07:56:54]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeclient pipe1
[07:57:35]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ls -l log1
-rw-r--r--  1 sherri faculty 32 Mar  8 07:57 log1
[07:57:38]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$

The server keeps running until killed (just like real servers). So we can start further clients (both sequentially and via a fan).

[08:00:32]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ more log1
29404: Tue Mar  8 08:00:31 2005
29403: Tue Mar  8 08:00:31 2005
29401: Tue Mar  8 08:00:31 2005
29400: Tue Mar  8 08:00:31 2005
29402: Tue Mar  8 08:00:31 2005
[08:00:38]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeclient pipe1 &
[8] 29409
[08:00:58]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeclient pipe1 &
[9] 29410
[8]   Done                    ./pipeclient pipe1
[08:00:59]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeclient pipe1 &
[10] 29411
[9]   Done                    ./pipeclient pipe1
[08:01:00]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ ./pipeclient pipe1 &
[11] 29412
[10]   Done                    ./pipeclient pipe1
[08:01:00]grace:/usr/users4/cnc/web/cnc/concurrency/examples/chap6$ more log1
29404: Tue Mar  8 08:00:31 2005
29403: Tue Mar  8 08:00:31 2005
29401: Tue Mar  8 08:00:31 2005
29400: Tue Mar  8 08:00:31 2005
29402: Tue Mar  8 08:00:31 2005
29409: Tue Mar  8 08:00:58 2005
29410: Tue Mar  8 08:00:59 2005
29411: Tue Mar  8 08:01:00 2005
29412: Tue Mar  8 08:01:00 2005
[11]-  Done                    ./pipeclient pipe1

(These clients don't have anything interesting to log!)

While writes are atomic to pipes (within the size limits), reads are not. We can try to create a different kind of client-server architecture where one pipe is a request pipe and one pipe is a response pipe. A client makes a request on the request pipe and then the server responds on the response\ pipe. In this case a sequence number is the response (you could think of this as a license server). However, since the sequence number is more than one bytes, the read is not guaranteed to be complete. The protocol chosen is that if a client does a partial read it will transmit an error request on the request pipe. The server can then terminate the application and unlink the pipes. (Also, they can't write to the request pipe if there is no one reading it: write will generate a SIGPIPE.) Future clients will then get an error on their read (the fifo no longer exists). (Need to do this since the byte that was 'lost' could have gone to anyone!)

(There are other alternatives here for the server: it could create a unique pipe for each client to receive its response.)

seqserverbad.c is the 'bad' server. seqclientbad.c (I could not get it to fail, but should be able to run this experiment with a variety of sleeps and number of processes to get it to fail.)