Semantic Analyais and Attribute Evaluation 9-16, 9-21, and 9-23, 2020 ======================================== Static Analysis Recall that static semantics are enforced at compile time, and dynamic semantics are enforced at run time. Some things have to be dynamic semantics because of LATE BINDING (discussed in Chap. 3): we lack the necessary info (e.g. input values) at compile time, or inferring what we want is uncomputable. A smart compiler may avoid run-time checks when it *is* able to verify compliance at compile time. This makes programs run faster. array bounds variant record tags dangling references Similarly, a conservative code improver will apply optimizations only when it knows they are safe alias analysis caching in registers computation out of order or in parallel escape analysis limited extent non-synchronized subtype analysis static dispatch of virtual methods An optimistic compiler may generate multiple versions with a dynamic check to dispatch always use the "optimized" version if it's speculative -- always safe and usually fast prefetching trace scheduling always start with the "optimized" version but check along the way to make sure it's safe, and be prepared to roll back transactional memory Alternatively, language designer may tighten rules type checking in ML v. Lisp (cons: 'a * 'a list -> 'a list) definite assignment in Java/C# v. C ---------------------------------------- As noted in Chap. 1, job of semantic analyzer is to (1) enforce rules (2) connect the syntax of the program (as discovered by the parser) to something else that has semantics (meaning) -- e.g. value for constant expressions code for subroutines This work can be interleaved with parsing in a variety of ways. - At one extreme: build an explicit parse tree, then call the semantic analyzer as a separate pass. - At the other extreme, perform all static dynamic checks and generate intermediate form _while_ parsing, using _action routines_ called from the parser. - The most common approach today is intermediate: use action routines to build an AST, then perform semantic analysis on each top-level AST fragment (class, function) as it is completed. We'll focus on this intermediate approach. But first, it's instructive to see how we _could_ build an explicit parse tree if we wanted. This will help motivate the code to build an AST. RD each routine returns its subtree TD-TD push markers at end-of-production each, when popped, pulls k subtrees off separate attribute stack and pushes new subtree, where k is length of RHS So how do we build a syntax tree instead? Start with RD requires passing some stuff into RD routines AST_node expr(): case input_token of id, literal, ( : T := term() return term_tail(T) else error AST_node term_tail(T1): case input_token of +, - : O := add_op() T2 := term() N := new node(O, T1, T2) return term_tail(N) | ), id, read, write, $$ : return T1 // epsilon else error It's standard practice to express the extra code as _action routines_ in the CFG: E => T { TT.st := T.n } TT { E.n := TT.n } TT => + T { TT2.st := new bin_op(+, TT1.st, T.n) } TT { TT1.n := TT2.n } TT => - T { TT2.st := new bin_op(-, TT1.st, T.n) } TT { TT1.n := TT2.n } TT => { TT.n := TT.st } T => F { FT.st := F.n } FT { T.n := FT.n } FT => * F { FT2.st := new bin_op(*, FT1.st, F.n) } FT { FT1.n := FT2.n } FT => / F { FT2.st := new bin_op(/, FT1.st, F.n) } FT { FT1.n := FT2.n } FT => { FT.n := FT.st } F => - F { F1.n := new un_op(-, F2.n) } F => ( E ) { F.n := E.n } F => const { F.n := new num(const.val) } F => id { F.n := new ident(is.name) } Here the numbers distinguish among instances of the same symbol in a given production. The .n and and .st suffixes are _attributes_ (fields) of symbols. << show how this handles, for example, (a + 1) * b >> A parser generator like ANTLR can turn this grammar into an RD parser that builds a syntax tree. It's also straightforward to turn the grammar w/ action routines into a table-driven TD parser. Give each action routine a number Push these into the stack along with other RHS symbols Execute them as they are encountered. That is: match terminals expand nonterminals by predicting productions execute action routines e.g. by calling a do_action(#) routine with a big switch statement inside requires space management for attributes in the TD-TD case: companion site (Sec. 4.5.2) explains how to maintain that space automatically ======================================== consider extended calculator grammar with types and declarations require declaration before use require type match on arithmetic ops program -> stmt_list $$ stmt_list -> decl stmt_list | stmt stmt_list | epsilon decl -> int id | real id stmt -> id := expr | read id | write expr expr -> term term_tail term_tail -> add_op term term_tail | epsilon term -> factor factor_tail factor_tail -> mult_op factor factor_tail | epsilon factor -> ( expr ) | id | int_const | real_const | float ( expr ) | trunc ( expr ) add_op -> + | - mult_op -> * | / could do some semantic checking while building the AST could even do so while building an explicit parse tree more common to implement checks once the AST is built eaiser -- tree has nicer structure more flexible -- can do non DFL2R traversals mutually recursive definitions e.g., methods of a class in most languages switch statement label checking etc. note that while building AST, parser definitely needs to tag nodes with source locations tagging of tree nodes is _annotation_ internally, tree nodes are structs annotations and pointers to children are fields (can also be done to an explicit parse tree) How do we express what we want the AST to look like? One appealing option is a _tree grammar_. Each "production" of tree grammar has parent on LHS and children on RHS. This is _not for parsing_; it's to describe the trees that - we want the parser to build - we need to annotate program -> item int_decl : item -> id item // item is next decl or stmt real_decl : item -> id item assign : item -> id expr item read : item -> id item write : item -> expr item null : item -> ε `+' : expr -> expr expr `-' : expr -> expr expr `*' : expr -> expr expr `/' : expr -> expr expr float : expr -> expr trunc : expr -> expr id : expr -> ε // no children int_const : expr -> ε real_const : expr -> ε The A:B syntax on the left means that A is one kind of a B, and may appear wherever a B is expected on a RHS. Note that "program -> item" does NOT mean that a program "is" an item (the way it does in a CFG), but merely that a program node in a syntax tree has one child, which is an item. Here's a syntax tree for int a read a real b read b write (float (a) + b) / 2.0 Tree grammars differ from CFGs. Language for a CFG is the possible *fringes* of parse trees. Language for a tree grammar is the possible *whole trees*. No comparable notion of parsing: structure of tree is self-evident. Our tree grammar helps guide us as we write (by hand) the action routines to build the AST. It can also help us write recursive tree-walking routines to perform semantic checks and (later) generate mid-level intermediate code (next lecture). - Esp. if we augment the tree grammar with _semantic functions_ that describe relationships among annotations of parent and children - Like action routines, but without clear sense of what is executed when. A CFG or tree grammar with semantic functions is an _attribute grammar_ (AG) Not used in most production compiers, but have been used in some (e.g., the first validated Ada compiler) and in some cool language-based tools - syntax-directed editing [Reps & Teitelbaum] - parallel CSS [Jones, Bodik, et al.] The book goes into AG theory just a little bit, talking about synthesized attributes (depend only on information below the current node in the tree) inherited attributes (depend at least in part on info from above or to the side) Our CFG w/ action routines to build the AST could be written as an AG by making each action routine a semantic function and then listing the functions for each production, w/out actually embedding them in the RHS. Note that AG doesn't actually specify order in which functions should be evaluated. Exist tools to figure that out, and rich theory of classes of grammars non-circular circular but converging When basing an AG on a CFG, it's desirable to have _attribute flow_ that's consistent with the order in which the parser builds the tree bottom-up parsers need S-attributed grammars -- all attributes are synthesized top-top parsers can use L-attributed grammars, which are a superset -- attributes are synthesized or depend on stuff to the left In practice, people do hand-written tree walk on ASTs. Book gives extended example for declaration and type checking in extended calculator grammar. Written as a pure AG, with following attributes: program errors - list of all static semantic errors (type clash, undefined/redefined names) item, expr symtab - list with types of all names declared to left item errors_in - list of all static semantic errors to left errors_out - list of all SSEs through here expr type errors - list of all SSEs inside everything location More common to make symbol table and error lists global variables insert errors, as found, into a list or tree, sorted by source location for symtab, label each construct with list of active scopes look up pairs, starting with closest scope for calculator language, which has no scopes, can enforce declare-before-use in a simple left-to-right traversal of the tree To avoid cascading errors, it's common to have an "error" value for an attribute that means "I already complained about this." So, for example, in int a real b int c a := b + c We label the '+' tree node with type "error" so we don't generate a second message for the ":=" node. A few example rules: int_decl : item -> id item -- item2 is rest of program if ∈ symtab errors.insert("redefinition of" id.name, item1.location) else symtab.insert() id : expr -> ε if ∈ symtab expr.type := A else errors.insert(id.name "undefined", id.location) expr.type := error + : expr -> expr expr if expr2.type = error or expr3.type = error expr1.type := error else if expr2.type <> expr3.type expr1.type := error errors.insert("type clash", expr1.location) else expr1.type := expr2.type We can see how these rules would be enforced while walking the syntax tree: In a more complicated language, we might make multiple passes over the tree – perhaps one to fill in the symbol table; a second to check types, check for undeclared names, match parameter lists to declarations, etc.; and a third to generate mid-level IF.