/* chardev.c * Neal Nelson 2010.02.20 * * Added blocking on reader fifo empty and writer fifo full using * kernel semaphores. * * Simple fifo character device driver supporting a single writer * and a single reader. The reader end of the fifo and the * writer end of the fifo are on separate devices. * * Keeps a circular buffer of characters that is initialized on module * load, queued on device write calls, and dequeued on device read calls. * * Maintains a synchronization lock on device open for both reader and * writer, so that only one reader and one writer have access to the * fifo, giving a single peer to peer connection. * * When you load this module you will get instructions for * creating the appropriate device files with the kernel-assigned * major device numbers (probably 254,255). For example, * mknod ./dev/mydev c 254 0 (do man on mknod). This feature * could be programmed into the init_module by doing a mknod * system call. Similarly, on cleanup_module, a system("rm") * call could be made to remove the device. Or we could just * leave the device if we keep our own experimental directory * of devices. * * To write to the device just do a cat myfile >./dev/mydev * or an echo 123456789abcde > ./dev/mydev * and do a cat ./dev/mydev to read from the device. Note * that writing to the device with cat appends characters and * reading from the device with cat empties the device. * * Notes * * 2010.02.24 * Somehow I got the program to overflow a system buffer and * drop the fifo. How? Ah! I did a cat > mydevW and wrote a long * line and got no space left on device. This seems to be because * I did not block in my driver waiting to put the rest of the * characters into the fifo. So the message back to cat was * unable to write the characters because the device buffer was * full. Perfect! That's what it should be! The solution is * to block in the Kernel device driver call and not bother cat * with the unwritten characters. * * 2010.02.24 * echo 123456789abcd > mydevW * The echo command continues to try to write the overflow characters * with no success for several tries before it finally blocks. Then a * cat mydevR clears the buffer with 123456789 and allows the writer * to fill the buffer with abce and then the reader clears that too. * It all seems to work, except the block didn't happen until after * several failed attempts by echo to put the extra characters into * the buffer. Why did the block take so long to show up? Is that * a time slice delay? In other words, you can't block until a * time slice boundary? I bet I won't have that problem if I block * in my driver which is in Kernel space! Finish this program and * build a Kernel blocking driver to fix this in experiment 7. * * 2010.02.24 * The reader does not seem to block when the queue is empty. Why? * Cat is acting like the read system call returns 0 bytes read * and cat says, OK, nothing to read. No problem. Why doesn't the * read device driver block on queue empty? * * 2010.02.24 * I need to put in the down test to make sure to act appropriately * if a process is interrupted while on a wait queue (like not * go on and act inside the critical region). The semaphores. * * 2010.02.22 * Upon command line testing the program seems to work on all cases * I've covered except I lose characters on buffer full. * * Kernel messages seem to show up always on the current virtual terminal * rather than the terminal the user program was initiated. User program * messages (characters going through the fifo) show up on the terminal * where the user program was initiated. * * Test cases. * 1. echo 12345 > mydevW; cat mydevR * Small write and total read. * * 2. echo 123456789abc > mydevW; head -n 1 mydevR * Pulls out characters but you won't always see them right away * because a newline is not in the buffer. Loses tail of long lines. * * 3. cat > mydevW; head -n 1 mydevR * line by line write and line by line read. * Ok on short lines, long lines lose characters and lines without * newlines (when buffer fills) don't show up right away. * * 4. cat > mydevW; cat mydevR * I think this one goes forever on the reader end because I don't * get the end of file through the fifo to the reader. * * 5. Some test I did managed to lock the mydevR up as busy. Did * I abort a read without unlocking the reader lock? Probably. * * 6. Somehow I got the program to overflow a system buffer and * drop the fifo. How? Ah! I did a cat > mydevW and wrote a long * line and got no space left on device. This seems to be because * I did not block in my driver waiting to put the rest of the * characters into the fifo. So the message back to cat was * unable to write the characters because the device buffer was * full. Perfect! That's what it should be! * * * 2010.02.16 * Added atomic locking on the dev_open variable. Still needs to * be stress tested to demonstrate the lock performs when needed. * Hmmmm. It seems to me that what this program really needs mutual * exclusion around the circular buffer read and write! That will * require some more locking. But for now, the lock on open will * suffice to insure mutual exclusion on buffer access since no * process can read or write unless they have opened the device. * * 2010.02.12 * I fixed two bugs. First, I accidently typed an index i in qbuf[i] * in the put_user instead of the proper index qbuf[rp]. * Second, I fixed an infinite write loop that occurred when the * buffer to write had a length greater than the circular buffer. * This caused the caller to persistently call back waiting to * hear that all of the buffer length of characters had been * written to the device. To fix this I just lied and returned * the caller's buffer length on write instead of the actual * number of characters put into the device circular buffer. * That fix is consistent with just throwing the characters * away, although I should probably have returned * a -EOVERFLOW error or something like that. * * The initialization of the circular buffer pointers has to * occur in init_module not open_device because each system * call may separately open the file and we want the state * of the file to live beyond the lifetime of any process * accessing the file. * * I believe you just have to quit thinking about using 0 EOS * terminated strings and always use buffer addresses and * lengths. The following types are relevant for that. I suspect the * signed size is used to return negative numbers as error codes. * You can always return an unsigned int as an int (but not the reverse). * * size_t is unsigned int * ssize_t (signed size) is int * * Static functions in C just mean private to the file, * which is what I think we want for these kernel device driver * functions. The driver functions should only be called through * the file_operations fops structure instance. * Note that init_module and cleanup_module are not private. * * This code is intended to allow only one process to be using * the device at a time. The locking in this program does not seem * to be SMP safe. Does the file system somewhere above do * some locking on device opens? * */ /* kernel module programming */ #ifndef MODULE #define MODULE #endif #ifndef LINUX #define LINUX #endif #ifndef __KERNEL__ #define __KERNEL__ #endif #include /* needed by all modules */ #include /* for KERN_ALERT */ #include /* for struct file_operations */ #include /* for put_user */ #include /* for all those handy error constants */ #include /* for atomic locking on device open */ #include /* for spin lock on circular buffer */ #include /* for buffer empty and full blocking */ /* prototypes - this would normally go in a header file */ int init_module(void); void cleanup_module(void); static int deviceW_open(struct inode *, struct file *); static int deviceW_release(struct inode *, struct file *); static int deviceR_open(struct inode *, struct file *); static int deviceR_release(struct inode *, struct file *); static ssize_t deviceR_read(struct file *, char *, size_t, loff_t *); static ssize_t deviceW_write(struct file *, const char *, size_t, loff_t *); static size_t enQ(const char *, size_t); static size_t deQ(char *, size_t); #define WRITE_DEVICE_NAME "mydevW" /* write device name in /proc/devices */ #define READ_DEVICE_NAME "mydevR" /* read device name in /proc/devices */ #define MAX_BUF 10 /* max msg length from device */ /* Device driver global variables */ static int majorW, majorR; static atomic_t devW_open = ATOMIC_INIT(1); /* Write device lock */ static atomic_t devR_open = ATOMIC_INIT(1); /* Read device lock */ static spinlock_t my_spinlock = SPIN_LOCK_UNLOCKED; /* Circular buffer lock */ static struct semaphore fullQ; static struct semaphore emptyQ; static char qbuf[MAX_BUF]; /* circular buffer of char for fifo */ static int rp, wp; /* Gcc assigns the rest of the structure fields to NULL */ static struct file_operations fopsW = { .write = deviceW_write, .open = deviceW_open, .release = deviceW_release }; static struct file_operations fopsR = { .read = deviceR_read, .open = deviceR_open, .release = deviceR_release }; /* new character write device file functions */ static int deviceW_open(struct inode *inodp, struct file *filp) { /* Lock the device for one writer * returns true if the result is zero. */ if (!atomic_dec_and_test(&devW_open) ) return -EBUSY; MOD_INC_USE_COUNT; return 0; } static int deviceW_release(struct inode *inodp, struct file *filp) { /* Unlock the device */ atomic_inc(&devW_open); MOD_DEC_USE_COUNT; return 0; } static ssize_t deviceW_write(struct file *filp, const char *buf, /* caller's buffer; const so it doesn't change */ size_t length, /* buffer size, size_t is unsigned int */ loff_t *offset) /* Our offset in the file */ { ssize_t n; unsigned long flags; printk(KERN_ALERT "device_write: Entering\n"); /* Write the entire caller's buffer into the qbuf. We * can't assume that the buffer has a 0 terminated * string in it. */ /* use enQ to copy the user buffer into the qbuf. If * the qbuf is not big enough, then just truncate the * user message. Return the number of characters * entered into the queue. */ printk(KERN_ALERT "device_write rp: %d, wp: %d\n",rp,wp); down_interruptible(&fullQ); spin_lock_irqsave( &my_spinlock, flags ); /* be safe and disable irqs */ n = enQ(buf, length); spin_unlock_irqrestore( &my_spinlock, flags ); up(&emptyQ); printk(KERN_ALERT "device_write rp: %d, wp: %d\n",rp,wp); printk(KERN_ALERT "device_write: %d chars enqueued\n", n); printk(KERN_ALERT "device_write: %d chars discarded\n", length-n); /* We have to lie about how many characters were written because * the caller may keep calling back until all the characters up * to the length of the buffer are reported to have been written. * Another way to do this is to go ahead and let the caller * call back, but throw the characters away up to length and * report back length-n the on the second call. That'll get the * caller off our back. Thirdly, we could just overwrite the * buffer, letting it go ahead and wrap around. Finally, I * really probably should just return -EOVERFLOW from * But for this experiment I'm just going to lie. */ return n; } /* new character read device file functions */ static int deviceR_open(struct inode *inodp, struct file *filp) { /* Lock device for one reader * Returns true if the result is zero. */ if (!atomic_dec_and_test(&devR_open) ) return -EBUSY; MOD_INC_USE_COUNT; return 0; } static int deviceR_release(struct inode *inodp, struct file *filp) { /* Unlock the device */ atomic_inc(&devR_open); MOD_DEC_USE_COUNT; return 0; } static ssize_t deviceR_read(struct file *filp, char *buf, /* caller's buffer and length */ size_t length, loff_t *offset) /* Our offset in the file */ { int n; unsigned int flags; printk(KERN_ALERT "device_read: Entering\n"); /* Use deQ to copy the qbuf into user buffer. If the * user buffer is smaller than the circular buffer, * then just leave it up to the caller * to keep asking for more until we send back a 0 byte * read meaning EOF. That's what the file read system calls do * when interacting with file drivers. * * I think this code simply solves this problem below by * returning 0 for EOF just when when 0 bytes are finally * read from the circular buffer. */ printk(KERN_ALERT "device_read: rp: %d, wp: %d\n",rp,wp); down_interruptible(&emptyQ); spin_lock_irqsave( &my_spinlock, flags ); /* be safe and disable irqs */ n = deQ(buf, length); spin_unlock_irqrestore( &my_spinlock, flags ); up(&fullQ); printk(KERN_ALERT "device_read: rp: %d, wp: %d\n",rp,wp); printk(KERN_ALERT "device_read: %d chars dequeued\n", n); return n; } /* Circular Queue functions */ static size_t enQ(const char *s, size_t length) { size_t i; /* Put characters from the user buffer into the * circular buffer. When the queue fills, ignore the * rest of the input string sent to the queue. * Return number of characters put in queue, so * return 0 if the queue is already full. */ /* put the argument string into the buffer, no 0 EOS */ for (i = 0; i < length && ((wp+1) % MAX_BUF != rp); i++) { qbuf[wp] = s[i]; wp = (wp+1) % MAX_BUF; } return i; } static size_t deQ(char *buf, size_t length) { size_t i; /* Fill the caller's buffer with characters taken from queue * until either the queue is empty or the caller buffer is full. * Return the number of bytes read from the queue, so 0 if * queue is empty. */ for (i = 0; i < length && rp != wp; i++) { /* buf[i] = qbuf[rp]; */ put_user(qbuf[rp], &buf[i] ); rp = (rp+1) % MAX_BUF; } return i; } /* Initialize the module at insmod */ int init_module(void) { printk(KERN_ALERT "chardev fifo module loaded\n"); /* the first argument as 0 directs kernel to supply a major device number */ majorW = register_chrdev(0, WRITE_DEVICE_NAME, &fopsW); if (majorW < 0) { printk (KERN_ALERT "init_module: register char write dev failed with %d\n", majorW); return majorW; } majorR = register_chrdev(0, READ_DEVICE_NAME, &fopsR); if (majorR < 0) { printk (KERN_ALERT "init_module: register char read dev failed with %d\n", majorR); return majorR; } /* init circular buffer pointers */ rp = 0; wp = 0; /* init semaphores - probably a macro to do this like spinlocks and atomic types */ sema_init(&emptyQ,0); sema_init(&fullQ,MAX_BUF-1); /* the last entry of the circular queue isn't used */ printk(KERN_ALERT "init_module: Major device number for write is: %d\n", majorW); printk(KERN_ALERT "init_module: Create write device with mknod ./dev/mydevW c %d 0\n", majorW); printk(KERN_ALERT "init_module: Major device number for read is: %d\n", majorR); printk(KERN_ALERT "init_module: Create read device with mknod ./dev/mydevR c %d 0\n", majorR); /* A non-zero return means init_module failed; module can't be loaded */ return 0; } /* Clean up the module at rmmod */ void cleanup_module(void) { int ret; ret = unregister_chrdev(majorW, WRITE_DEVICE_NAME); if (ret < 0) printk(KERN_ALERT "cleanup_module: unregister write dev failed with %d\n", ret); ret = unregister_chrdev(majorR, READ_DEVICE_NAME); if (ret < 0) printk(KERN_ALERT "cleanup_module: unregister read dev failed with %d\n", ret); printk(KERN_ALERT "chardev fifo module unloaded\n"); } MODULE_LICENSE("GPL");