Project for CSC 255/455, Spring 2008

Introduction

In the project, you will add your own optimization passes in the most recent Gnu C compiler, Gcc 4.2.2. Because Gcc is a real compiler, it is more difficult to use than the teaching compiler used in previous semesters. You have to know the basic framework of Gcc and how to manipulate the intermediate form, the gimple tree, for program analysis and transformation. This page includes many suggestions that will help you in the process. Please read the entire page and the page on source browsing and debugging before you start. Wish you a happy journey with Gcc. Below is a quote from Will Durant from his 1926 book The story of philosophy

"Happiness, says an old lesson, lies rather in achievement than in possession or satiation. The healthy man asks not so much for happiness as for an opportunity to exercise his capacities; and if he must pay the penalty of pain for this freedom and this power he makes the forfeit cheerfully; it is not too great a price. We need resistance to raise us, as it raises the airplane or the bird; we need obstacles against which to sharpen our strength and stimulate our growth. Life without tragedy would be unworthy of a man."

Installation, compilation, and turn-in

  • Check out code and install: See Gcc-4.0.1 notes on installation and change all 4.0.1 to 4.2.2.
  • For those who have no access to department research network, please download gcc-4.2.2.tar.bz2 and unzip it with the command tar -jxvf gcc-4.2.2.tar.bz2.
  • Use either our desktop or personal machine or one of the public use machines in the CS department. If you have to use the cluster (which is for mainly research use and for running 64-bit applications), use qsub -I at node64 to get a node to accelerate the make and install: see ClusterJobs
  • To prevent the problem of hardware incompatibility, you'd better build your compiler and use it and run the compiled executables on the same kind of machines.
  • Output gimple tree in c-like form: use option -fdump-tree-gimple
  • Invoke your pass: use -O2 -fcs255
  • Output gimple tree in c-like form after the pass for cs255: use option -fdump-cs255-tree-gimple
  • Put your own code into gcc-4.2.2/gcc/cs255.c
  • Use make for your testing: see An Example for Makefile
  • How to turn in your code: see Turn in for cs255

Browsing Gcc code and debugging

Other Gcc resources

Part 1: Trivial (5%)

In this part, you will add a pass that will insert code into a program to count and output the number of executed statements when the program executes. You need to first add a call at the beginning of the main function, which has a global counter initialized to 0; then add a call before each gimple statement, which increments the counter; and finally add a call at the exit, which reports the run-time instruction counts. The three functions are provided in cs255-lib.c and cs255-lib.h:
  • Initialization: you'll need to add it at the entry of the main function
  • Increment: to be added before each real gimple statements (all except for labels)
  • Report: to be added at all possible exits of the program
  • Remember to add #include "cs255-lib.h" in your own testing programs.

This part should take at most tens of lines of code and no more than a day (including half an hour compiling the compiler for the first time). The purpose is to get you familiar with Gcc and the basic Gcc functions that you'll need for adding your optimization passes. The code for counting the number of basic blocks and statements statically is already in the file. The code for getting the dynamic count of the basic blocks is also given.

Reading the relevant, existing code (in gcc-4.2.2/gcc/cs255.c) is the best way to help you writing your new code in an existing system. The next best way is to ask your instructor, TA, and your classmates to find out how to get Gcc to do something. You are encouraged to discuss the project and work out the algorithms on paper or a white or black board with others. The only limitation is that you should not copy other people's code---every line in your submitted code file must be yours.

For testing, there are some open test cases. You may use the results from them to check your code with your classmates. And they are very useful in following parts and final competition.

Deadline: 11:59pm Feb 11, 2008.

Part 2: Weird Control Flow (10%)

  • Reverse all printf, scanf, and __builtin_puts statements in a basic block. If there are k such statements in a basic block, the first statement will be switched with the last statement, the second statement will be switched with the (k-1)th statement, and so on. At a call to printf, Gcc copies the arguments to a temporary variable. For the project, you need to move these copy statements with the call. A recent question and answer discusses some ambiguity and its recommended resolution.
  • If the statement is a conditional branch in the form of if cond goto L1; else goto L2, change it to if (not cond) goto L1; else goto L2. You only need to deal with >, >=, <, <=, ==, =!.
  • If the statement is a self increment (or decrement) by 1, change it to the opposite, that is, to decrement (or increment) by 1. For example, change i=i+1 to i=i-1 and vice versa.
  • Try all our test cases to make sure that your compiler generates code with no error, even though the generated code may not run correctly.

When you are working on this transformation, use the following two tests to check your compiler. The counting instructions should be inserted after the control flow changes.

As part of the documentation, submit two programs that would look bizarre but when compiled by your compiler, will produce the same output as the following two normal looking programs when they are compiled by regular Gcc. (You may break for loop statement into a combination of several goto.)

Bonus point We offer additional 5% of the project grade as bonus to students who use their compiler, using both of the bizarre examples or their equivalents, to baffle someone outside the class and document his or her reactions in the submitted report. The deadline for the bonus is the same as the project deadline.

Program 1: ab.c

int main() 
{
   int a, b;
   printf("Input two integers for comparison\n");
   scanf("%d", &a);
   scanf("%d", &b);
   printf("a is %d\n", a);
   printf("b is %d\n", b);
   if (a > b)
     printf("a is greater than b\n");
   else
     printf("a is no greater than b\n");
}

Program 2: loop.c

int main() 
{
   int i, s, n;
   printf("Input an integer for sum\n");
   scanf("%d", &n);
   for (i=0, s=0; i<=n; i++)
     s += i;
   printf("The sum from 1 to %d is %d\n", n, s);
}

Deadline: 11:59pm Feb 18, 2008.

Part 3: Value Numbering (10%)

In this part you need to implement value numbering in your compiler. As you know, value numbering can be carried out in different scopes: basic block, extended basic block, or dominator block. In this assignment, you are required to implement basic-block value numbering. However, optimizing at a larger scope may help you in your final competition. It is up to you to decide which algorithm you implement.

The public test programs use only basic arithmetic and logical operations. Your compiler is required only to correctly optimize these programs. It does not need to support pointer operations except to allow the taking-address operation in scanf. The absence of dynamic data allocation and pointer dereference removes most of the aliasing concerns and gives you more room for optimization. In addition, if you are taking CS255 instead of CS455, your compiler does not need to support or optimize array operations.

Your compiler may fail if a program contains operations it does not support. It should fail gracefully: it should check for cases it does not support, and when it finds such cases, output a descriptive error and exit. Under no circumstances it may generate incorrect code (except in Part 2). Your compiler will be evaluated in hidden test cases, and an incorrectly optimized program would count against you more so than a failure in compilation.

In this and later projects, instrumentation should always be the last pass. Do not optimize on the instrumented program.

In this and later projects, do not use any Gcc optimization passes and make sure you have the update posted earlier. You can use Gcc routines for inspecting and changing gimple statements and their control flow, as you did in the first two projects. You need to write your own algorithm iterating through program statements, maintaining value tables, and removing redundant operations, as well as checking for unsupported operations. You may use vector, hash table, search tree, and other generic data structures in Gcc. You may also use the analysis in Gcc including the dominator information. For this project you may not use the SSA information or the Gcc code for value numbering.

You do not have to use Gcc code. If you do, you should document precisely the functions you use and the purpose of using them. Talk to the TA or the instructor if you are not sure.

If you find useful Gcc routines, please post them to the class discussion page. The project is a collaborative contest. You collaborate to understand and make the best use of existing code. You compete in the design and implementation of the optimization algorithms you add to the compiler.

Deadline: 11:59pm Feb 25, 2008.

Part 4: Data Flow (15%)

You are asked to implement and use data flow analysis in the last phase before the competition, in particular, implement the following parts in your cs255 pass.

  • Def-use graphs
  • Dead code elimination
  • Constant propagation (required for 455, Bonus for 255)

Deadline: 11:59pm Mar 17, 2008.

The Last Mile: Group Competition

By now you have a fairly complete grasp of the theories and techniques for redundancy elimination at the procedural level, working knowledge of the Gcc compiler, and a clear view of the optimization problem in terms of the type of the test programs. In the last phase of the project you have the freedom to develop creative and effective solutions and build your own super compiler. The results from the test programs and (later) hidden benchmarks will become part of the growing record of the annual compiler competition, so your compiler competes not only with the compiler from your classmates but also the best and brightest from the past years and future years. This is also the first time we can evaluate the Gcc optimization passes using the same metric. You can beat current and future versions of Gcc.

The competition is organized in small teams, usually three or four students per team. We will ask you to fill out a Team-Project Questionnaire and organize teams and assign team leads based on your preference and our discretion. Undergrads are generally not treated differently unless we form a team with predominantly 255 students, in which case we will grade the team differently. Each member will be taking on one or more of the following roles:

* Team lead is the embodiment of the team's competitive spirit (think of the upcoming Olympics), responsible for meeting intermediate project requirements and deadlines, arbitration and decision making in face of design and coding issues, and regular interaction with the TA and instructor during the competition.

* Optimizer architect/developer. We will ask you to use pair programming, where two students work together in implementation. One types in the code and the other inspects it. It benefits from the separation of concerns: the former concentrates on tactical issues including syntax, naming, and other immediate problems of coding, while the latter focuses on strategic aspects including possible problems and design issues. The greatest benefit of pair programming stems from the fact that it rarely happens that two people make the same mistake independently. You and your partner save each other from debugging for hours in a wild goose chase. Every team should have two architects/developers at any work session.

* Testing/application study. Instead direct compiler development, the person would ensure the correct results from the compiler and in doing so, identify additional opportunities in the test code and formulate and suggest new transformations or new variations of an existing transformation.

* Gcc guru/tool expert knows relatively more than others about the knobs and buttons of Gcc and useful data structures or knows how to find answers. The tool experts from different teams are encouraged to collaborate and share their knowledge, e.g. through the class discussion page.

* Documenting/reporting is important to internal team communication and it is necessary for grading purposes. Each team is required to maintain an internal wiki page (with access only to team members and the TA and the instructor) chronicling, in the reverse chronological order, the development effort including the meetings with a short summary of the discussion, coding sessions with a summary of the main outcomes, and the basic design document and the testing results. Each member can and should contribute but one person is responsible for the organization of the wiki page, starting from the creation of the page. Attribute ideas and efforts to particular team members when possible. The page is an important basis for evaluating the team effort and the individual contributions.

Grading of the competition is as follows. Basic participation---contribution to discussion, design and development and the clarity and completeness of documentation---counts for 50 points. This is the participation score. The compilers are ranked for each test program. A best score worths 10 points, 2nd best 8 points, and 3rd 5 points. If multiple compilers score the best, 2nd, or 3rd, they each receives 10, 8, and 5 points. The top three compilers for the hidden benchmarks receive proportional awards but the award points triple (30, 24, 15). The total is the performance score. Incorrect compilers will be penalized. Failing to produce correct code for any test program would lose 10% of the sum of the participation and performance scores. This is the penalty kick. The total score minus any penalty kick is the group score. The group score is not necessarily the individual score of all team members. The TA and the instructor will evaluate based on the group documents and adjust the individual score to reflect the relative contribution. We may conduct a confidential "exit poll" to ask each member to rank the contribution of other team members.

The project questionnaire will be sent out by email on Wednesday March 18 and should be returned by the end of Thursday March 19. The groups will be formed the next day. There will be a project meeting with the TA and the instructor at 11am on Tuesday March 25 and Fridays March 28 and April 4 in 703 (or, if 703 not available, at the software lab). The three meetings are required for team leads and optional for others.

Project milestones

  • Monday, March 24, groups formed, roles assigned.
  • Friday, March 28, design and implementation strategy reviewed. Gcc gurus met.
  • Friday, April 4, first feedback from the hidden benchmarks.
  • Wednesday, April 9, second feedback from the hidden benchmarks.
  • Monday, April 14, midnight, competition concludes.

Competition Rules

In gimple tree, there are some opportunities to decrease the executed statements by combining several into one. The following rules are for preventing these feasible but illegal optimizations.

  • Left hand side of an assignment statement should be a variable or an array access.
  • Right hand side of an assignment statement should be either a variable, a constant, an array access, a function call or an expression with no more than two operands (or one operator).
  • If there is an array access in an assignment statement, the statement has only two operands - one is the array access and the other is a variable.
  • Any operand of an expression can only be a variable or a constant.
  • The predicate of a conditional jump can be one comparison operation whose two operands are either a variable or a constant.
  • Parameters to a function call can only be a variable or a constant.
  • Index of an array can only be a variable or a constant.
  • Return parameter can only be a variable or a constant.

NOTE: you do NOT need to do any work to make the gimple tree to follow the above rules because gcc has already simplified it.

After the program transformation, new temporary variables could be introduced. An example program segment

    a = b + c[e] + d;
    if (a+d<b) goto L1;
         foo (m[a+d]);
         m[e+f] = m[a] + m[d];
    L1:

needs to be converted to something like the following:

  t1 = c[e];
  t2 = t1 + b;
  a = t2 + d;
  t3 = a + d;
  if (t3 < b) goto L1;

  t4 = a + d;
  t5 = m[t4];
  foo (t5);
  t6 = e + f;
  t7 = m[a];
  t8 = m[d];
  t9 = t7 + t8;
  m[t6] = t9;

L1:

For scanf, you do not need to make any temporary variable to take value of the pointer passed to the call. Remember that our language otherwise does not have pointers, so you do not deal with pointer induced aliasing. For printf, you still need to make all the arguments in variable or constant form except for the format string.

Edit | Attach | Watch | Print version | History: r43 < r42 < r41 < r40 < r39 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r43 - 2008-06-26 - IanChristopher
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding URCS? Send feedback