Programming assignment #1 - Warm-up

Due by 11:59pm, Monday, September 17th

This assignment has four goals:

  1. To dust off your C/C++ programming skills;
  2. To observe the operating systems' (OS') activities at the user-level;
  3. To implement an OS command interpreter (shell);
  4. To use some system calls (such as fork() and exec()).
This is an individual assignment. However, you are encouraged to help (and seek help from) your peers (except sharing code, of course). This assignment contains the following two parts: 1) observing the OS through the /proc file system; 2) building a shell. Everything you do in this warm-up assignment is at the user-level (outside of the OS kernel).

Part I: Observing the OS through the /proc file system

The OS is a program that uses various data structures. Like all programs in execution, you can determine the performance and other behavior of the OS by inspecting its state - the values stored in its data structures. In this part of the assignment, we study some aspects of the organization and behavior of a Linux system by observing values of kernel data structures exposed through the /proc virtual file system.

The /proc virtual file system:
Linux uses the /proc file system to collect information from kernel data structures. The /proc implementation provided with Linux can read many different kernel data structures. If you cd to /proc on a Linux machine, you will see a number of files and directories at that location. Files in this directory subtree each corresponds to some kernel data structure. The subdirectories with numeric names contain virtual files with information about the process whose process ID is the same as the directory name. Files in /proc can be read like ordinary ASCII files. You can open each file and read it using library routines such as fgets() or fscanf(). The proc (5) manual page explains the virtual files and their content available through the /proc file system.

Requirements in detail:
In this part, you are asked to write a program to report the behavior of the Linux kernel. Your program should run in two different versions. The default version should print the following values on stdout:

A second version of the program should run continuously and print lists of the following dynamic values (each value in the lists is the average over a specified interval): If your program (compiled executable) is called proc_parse, running it without any parameter should print out information required for the first version. Running it with two parameters "proc_parse <read_rate> <printout_rate>" should print out information required for the second version. read_rate represents the time interval between two consecutive reads on the /proc file system. printout_rate indicates the time interval over which the average values should be calculated. Both read_rate and printout_rate are in seconds. For instance, proc_parse 2 60 should read kernel data structures once every two seconds. It should then print out averaged kernel statistics once a minute (average of 30 samples). The second version of your program doesn't need to terminate.

Unix Documentation: The Manual
All Unix systems come with an online manual. The ``man'' command is used to look up pages within the manual. For example, to look at the manual page for the ``chdir'' system call, type:
     man chdir
To look up information on the /proc virtual file system, type:
     man proc
Sometimes, the same item appears in separate sections of the manual pages. System calls are documented in Section 2. You can specify a section number before the name of the item for which you want documentation. For example, to lookup the {\tt chdir()} system call in Section 2, use the following command:
     man 2 chdir
Section 1 contains information on commands, Section 2 contains information on system calls, and Section 3 contains information on C and Unix library functions.

Part II: Building a shell

UNIX shells:
The OS command interpreter is the program that people interact with in order to launch and control programs. On UNIX systems, the command interpreter is often called shell: a user-level program that gives people a command-line interface to launching, suspending, and killing other programs. sh, ksh, csh, tcsh, bash, ... are all examples of UNIX shells. You use a shell like this every time you log into a Linux machine at a URCS computer lab and bring up a terminal. It might be useful to look at the manual pages of these shells, for example, type "man csh".

The most rudimentary shell is structured as the following loop:

  1. Print out a prompt (e.g., "CSC2/456Shell$ ");
  2. Read a line from the user;
  3. Parse the line into the program name and an array of parameters;
  4. Use the fork() system call to spawn a new child process;
  5. Once the child (the launched program) finishes, the shell repeats the loop by jumping to 1.

Although most commands people type on the shell prompt are the names of other UNIX programs (such as ps or cat), shells also recognize some special commands (called internal commands) that are not program names. For example, the exit command terminates the shell, and the cd command changes the current working directory. Shells directly make system calls to execute these commands, instead of forking a child process to handle them.

Requirements in detail:
Your job is to implement a very primitive shell that knows how to launch new programs in the foreground and the background. It should also recognize a few internal commands. More specifically, it should support the following features.

To allow users to pass arguments you need to parse the input line into words separated by whitespace (spaces and '\t' tab characters). You might try to use strtok_r() for parsing (check the manual page of strtok_r() and Google it for examples of using it). In case you wonder, strtok_r() is a user-level utility, not a system call. This means this function is fulfilled without the help of the operating system kernel. To make the parsing easy for you, you can assume the '&' token (when used) is separated from the last argument with one or more spaces or '\t' tab characters.

The shell runs programs using two core system calls: fork() and execvp(). Read the manual pages to see how to use them. In short, fork() creates an exact copy of the currently running process, and is used by the shell to spawn a new process. The execvp() call is used to overload the currently running program with a new program, which is how the shell turns a forked process into the program it wants to run. In addition, the shell must wait until the previously started program completes unless the user runs it in the background (with &). This is done with the wait() system call or one of its variants (such as waitpid()). All these system calls can fail due to unforeseen reasons (see their manual pages for details). You should check their return status and report errors if they occur.

You can get a working shell from here (executable on Linux x86 platforms) and play with it. Note that this is not intended to be a complete solution. It does not support some features that you are asked to implement.

In order to level the playing field between those who have and have not taken computer organization at UR, you may consult the skeleton provided at /u/cs256/src/sh-skeleton.c, which was the skeleton given to the CSC 252 students. There is no compulsion to use this skeleton, however. You can certainly create your own organization and reference this skeleton to determine the kinds of calls and structures you will need to create.

No input the user gives should cause the shell to exit (except when the user types exit or Ctrl+D). This means your shell should handle errors gracefully, no matter where they occur. Even if an error occurs in the middle of a long pipeline, it should be reported accurately and your shell should recover gracefully. In addition, your shell should not generate leaking open file descriptors. Hint: you can monitor the current open file descriptors of the shell process through the /proc file system.

Controlling Terminal:
When a terminal reads various control sequences from the keyboard, it sends appropriate signals to the processes associated with that terminal. For example, typing Ctrl-C sends a SIGINT signal while Ctrl-Z sends a SIGTSTP signal. The terminal is known as the controlling terminal for those processes. When a process is run in the background, it should not receive these signals. Likewise, when it is brought back into the foreground, it is attached to the controlling terminal and should receive these signals. You must add the bg command (which moves a foreground process into the background) and the fg command (which moves a background process into the foreground). Moving processes into/out of the foreground should change their association with the controlling terminal properly. Details on controlling terminals can be found in W. Richard Steven's book Advanced Programming in the Unix Envionment; this book is available online via the library.

Pipes:
Your shell needs to support pipes. Pipes allow the stdins and stdouts of a list of programs to be concatenated in a chain. More specifically, the first program's stdout is directed to the stdin of the second program; the second program's stdout is directed to the stdin of the third program; and so on so forth. Multiple piped programs in a command line are separated with the token "|". A command line will therefore have the following form:
     <program1> <arglist1> | <program2> <arglist2> | ... | <programN> <arglistN> [&]
Try an example like this: pick a text file with more than 10 lines (assume it is called textfile) and then type
     cat textfile | gzip -c | gunzip -c | tail -n 10
in a regular shell or in the working shell we provide. Pause a bit to think what it really does. Note that multiple processes need to be launched for piped commands and all of them should be waited on in a foreground execution. The pipe() and dup2() system calls will be useful.

Administrative policies

A note on the programming language:
C/C++ is the only choice for this assignment and all later programming assignments. We are not alone in this. Most existing operating system kernels (Linux and other UNIX variants) themselves are written in C; the remaining parts are written in assembly language. Higher-level languages (Java, Perl, ...), while possible, are less desirable because C allows more flexible and direct control of system resources.

Turn-in:
You are asked to electronically turn in your source files and a makefile. After compiling, we should see two executables (one for Part I and one for Part II). Attach a README file describing the names of the two executables, special compiling instructions, or anything else special you want to let us know. The README file should be in plain text format. Instructions for electronic turn-in can be found on the class web page.

Grading guideline: Please note that CSC 252 students may only use their own solutions as a starting point (if you choose to do so) and clearly indicate what changes were made. You will be penalized heavily for failure to pass tests. The tests will not be identical to those used in 252. You should improve your testing procedures and indicate clearly in your README what tests you conducted.

Late turn-in policy:
Late turn-ins will be accepted for up to three days, with 10% penalty for each late day. No turn-ins more than three days late will be accepted.