Assignment #1 - Performance Measurement

Due on Sunday, February 10. You should turn in a written report. You can either send an electronic copy to me, or you can drop a hard copy in my mailbox in the mailroom adjacent to the 7th floor software lab. I do not have an official late turn-in policy for this course. If you have some good reason to be late, please let me know and I will consider it on a case-by-case basis.

For comparison purposes, please do all measurements on machines in the 7th floor software lab. Examples include: brussels, hanoi, copenhagen, vaduz, caracas, moscow, bridgetown, vienna, nassau, havana, prague, kingston. Since these are desktop machines and sometimes people do use them, please choose an unused machine for your measurement so it will not bother others. Working on an unused machine also helps you to acquire stable measurement results.

For the purpose of timing measurement, you may find it helpful to use a high resolution timer (see a link on the left pane).

Part I: context switch cost
In the Operating Systems course, you have measured the context switch cost between two threads. Please repeat that measurement to report the thread context switch cost.

Now if my guess is right, your threads do not do anything meaningful between switches. In practice, when threads run certain tasks, the context switch cost will be higher due to cache warmup --- the switched-in thread must load the working set from the memory to cache. The exact cost depends on the data access pattern of the threads. In this part of the assignment, you are to measure the context switch cost when threads follow certain data access patterns:

  1. Each thread reads a block of data byte-by-byte sequentially (first byte, second byte, third byte, ...) between two consecutive context switches. Please measure the context switch cost at each of the following block sizes: 32KB, 256KB, and 2MB.
  2. Each thread reads a block of data in a seesaw fashion (first byte, last byte, second byte, last but one byte, third byte, ..., until all bytes are read) between two consecutive context switches. Please measure the context switch cost at each of the following block sizes: 32KB, 256KB, and 2MB.
One intuitive way to measure the context switch cost when the threads are doing some real work is as follows. First you measure the total cost of the execution, then you measure the cost of the execution when threads are doing their work without context switching. Finally you subtract the two costs. You may find that it is not straightforward to apply this idea in practice.

Part II: Concurrent server performance
In this assignment, you are asked to measure the performance of a concurrent server I wrote. Do the following to run the server on a machine of your choice: /u/kshen/www/csc573-spring2008/assignments/server. This server listens at port 5678. Each incoming request is served by a separate thread. The server accepts requests with a single character '1' or '0'. For '1', it performs some work (which I am not telling you now) and returns a response string "SUCCESS\n". For '0', the server simply terminates itself.

Here is one way to try out this server using telnet. Launch the server at machine hanoi in the graduate software lab. Do the following to send a request (from a terminal at a different machine):

Do the following to terminate the server (from a terminal at a different machine):

In this part of the assignment, you are asked to measure the response time of a large number (e.g., 1000) of requests and plot the distribution of the request response time. Fit the result with some known distribution if you can. Of course you should not do the measurements through manual telnet tests --- write a test program to send/receive TCP/IP messages and measure the timing. In this test, you should run one request at a time (without concurrency).

Now try to run multiple requests concurrently and then measure the maximum throughput of this server (in the number of requests per second). It is imperative to repeat your experiments until you have stable results.

You may notice that you only need the '1' requests in your tests. The '0' request is given to help you shut down the server after your tests are completed. Certainly you can also do so by killing the server process. In either case, you must remember to shut down the server after your tests. It may not automatically go away after you log out your session. Having hanging processes will make system administrators very unhappy!

Turn-in:
You are asked to turn in a written report. The report should describe the setup of your measurements. Of course it needs to present the measurement results (in graphical illustration if it helps). Describe important implications of your results and any further thoughts you developed through this process. Do me a favor not to turn in a hand-written report unless you are sure your handwriting is perfectly recognizable. You do not need to turn in any source code.