Due by 11:59pm, Monday, April 27.
In this assignment you are asked to build a concurrent Web proxy server that is capable of delivering Web content on behalf of a remote Web server. When a user's browser is configured to connect via a proxy server, her browser establishes a connection to the proxy server's machine and forwards its complete request to the proxy server rather than the "real" Web server. The proxy server accepts user's HTTP requests and forwards them to their final destination - essentially it introduces an extra "hop" between the user's browser (or the client) and the Web server. Concurrency (e.g., through multi-processing or multi-threading) is important to allow the server to process multiple simultaneous requests in parallel, a must-have feature for a practical server.
There are several reasons why people want to use a proxy server instead of connecting to the remote Web server directly: 1) users want to anonymize themselves from the Web server; 2) a corporation might want to monitor or restrict employees' Web surfing; 3) a proxy can be used to locally cache data in order to reduce the amount of global traffic. You do not need to implement these features in this assignment.
The managing TA for this assignment is Mohammad Hedayati (hedayati@cs.rochester.edu). Please first direct your questions about this assignment to the managing TA. The TA's office hours for this assignment are:
You can form a group of two for this assignment. You can also choose to work alone. You can choose to work in a group of two for four assignments (assignments #2/#3/#4/#5). It is our policy that you can NOT work with the same team partner for more than two assignments. If you do, we will apply 50% penalty on your third and fourth assignments with the same grouping.
The proxy server acts as an HTTP server to client browsers while it acts as an HTTP client to the "real" Web server. The protocol specification for version 1.1 of HTTP is defined in RFC 2616. This is not a long document among the protocol specifications, but it may well appear so for you. For your relief, you are asked to implement a very small part of the complete protocol, which will be explained below. We do try to be as specific as possible in this assignment description in order to reduce your need to refer to the lengthy protocol specification. But you should be prepared to do so occasionally.
A full HTTP-1.1 compliant Web server supports many methods. Your proxy server only needs to support the GET method. To serve each request, we first need to parse the request line and headers sent by the client. The request line for the proxy server typically looks like this:
GET http://www.foo.com/mumbo.html HTTP/1.1The requested Web page name contains the Web server name
www.foo.com
and the requested file on that server
/mumbo.html
. In this case, your proxy server should make a
TCP connection to Web server www.foo.com
at the default
port 80 and ask for file /mumbo.html
by send the following
request:
GET /mumbo.html HTTP/1.1\r\nhost: www.foo.com\r\n\r\n
After sending a request to the "real" Web server, an HTTP response
including the requested file will be received at the proxy server.
The proxy server should then forward the content to the client.
There are three parts in an HTTP response message: the status line,
the response headers, and the entity body. The status line and response headers
are terminated by an empty line (or an extra "\r\n"
).
In short, an HTTP response message may look like the following:
HTTP/1.1 200 OK Server: Joe's Web proxy server Date: Thu Sep 12 17:52:40 EDT 2002 Content-length: 3814 Last Modified: Thu Sep 12 01:37:39 EDT 2002 Content-type: text/html ... ... <content of the main.html>The status line and most of the header fields should be forwarded to the client without modification. Note that some responses use chunked transfer encoding that we discussed in class. You will need to handle that properly.
The server should be able to handle multiple simultaneous service requests in parallel. This means that the Web proxy server should use multiple processes or multiple threads. In the main process/thread, the server listens at a fixed port. When it receives a TCP connection request, it sets up a TCP connection socket and services the request in a separate process/thread.
Your server must also reap allocated resources for completed requests. In other words, your server should not leak resources.
Here we provide a hint on what the program skeleton may look like in the case of a multi-threading server. You are, however, not required to design your program this way.
Main routine { Parse the command line input (server port number); Create a server socket listening on the specified port; For each incoming client socket connection { Spawn a worker thread to handle the connection; Loop back to wait for/handle the next connection; } } Worker thread routine { Read the request line and header fields until two consecutive new lines; (Note that a new line can be a single "\n" or a character pair "\r\n".) Examine the first line (request line); If the request method is not "GET" { Return an error HTTP response with the status code "HTTP_BAD_METHOD"; } Make TCP connection to the "real" Web server; Send over an HTTP request; Receive the server's response; Close the TCP connection to the server; Send the server's response back to the client; Close the connection socket to the client. }
If you develop your program at your home computer, it is imperative to test it in school network before turn-in. Below we provide some information about how to test your program.
For testing, you can configure a Web browser to use your proxy server. For Firefox 37.0, you can configure it in the following way: 1) Open the Menu; 2) Click "Preferences"'; 3) Choose "Advanced" Tab; 4) Choose "Network" Sub-Tab; 5) Click "Settings" for Connection; 6) Click "Manual proxy configuration" and fill the server name as well as its port number. Make sure your proxy server is running and now all HTTP requests from your browser are served via the proxy server!
You can also test your server using telnet
, which
takes console input and sends to a server at a specific port number.
It then receives server responses and prints them on the console.
If your proxy server runs at port 8080 of cycle1.csug.rochester.edu
,
you can do
[yourself@foobar] telnet cycle1.csug.rochester.edu 8080<enter> GET http://www.google.com HTTP/1.1<enter> host: www.google.com<enter> <enter> ... ... <response from your server, including status line, header, and the requested file> [yourself@foobar]You can also try this with the Web server directly, then you will need to specify the default port number 80. For instance, you can do
[yourself@foobar] telnet www.google.com 80<enter> GET / HTTP/1.1<enter> host: www.google.com<enter> <enter> ... ... [yourself@foobar]
Here is a suggested way to test your proxy's ability to multiple concurrent connections:
(1) Open a connection to your proxy using telnet
, and then leave it open without
typing in any data. (2) Use a Web browser (pointed at your proxy) to
request content from some end server.
To test proper resource reaping (no resource leaking), you can write an automatic client that sends many requests to the proxy server and see if the server keeps functioning properly after serving thousands of requests.
Finally, we provide a tool that you can use to test the correctness of your program.
To run this tool (on a CSUG machine), if your proxy server runs at port 8080 of
cycle1.csug.rochester.edu
, do:
[yourself@foobar] ~cs252/labs15/proxy-test.py cycle1.csug.rochester.edu 8080Note that this tool DOES NOT cover all the cases that might be tested for grading.
You are asked to turn in your source files, a makefile if needed, and a
README file. No matter what programming language you choose to use,
your program should take a single parameter (its port number) on startup.
You should also name your executable to be ProxyServer
.
If your program is written in C/C++, you should be able to launch your
server using "ProxyServer <port-number>
".
If your program is written in Java, you should be able to launch your
server using "java ProxyServer <port-number>
".
The README file should be in plain text format. It should contain a description of your design, what is and what is not realized in your implementation. If your program requires any special compilation flag to build, you need to specify the full build command in the README file. You are strongly recommended to provide us a makefile.
You should electronically turn in the required files through Blackboard.
Late turn-ins will be accepted for up to three days, with 10% penalty for each late day. No turn-ins more than three-day late will be accepted.