Notes for CSC 162, 16 Mar. 2010 ff Chapter 5 Hand back midterms questions? dynamic programming was the weak link Tree enumeration problem from Tuesday workshop count # of trees actually enumerate them Is everybody reading the text and my commentary? Fourth project due Tues. Mar. 30, noon. ======================================== Trees terminology: node parent, child, sibling subtree leaf, internal node root height, depth depth of root is zero height may be max depth, or max depth + 1, depending on author (our authors go with the former) binary v. general trees binary distinguish between left and right children; can have a right without a left general usually order the children, but do not have any that are empty (that's not a universal rule, though) easy way to think of this: in a binary tree every node is either empty or it has exactly two children (each of which is either empty or has exactly two children, etc.) more terminology: proper binary tree no node is missing a sibling full binary tree every non-empty level has max number of nodes implementation list binary trees: | | A and A / \ B B are ['A', ['B', [], []], []] and ['A', [], ['B', [], []]] general trees: customary to simply leave out missing children. So | A would be ['A', ['B'], ['C'], ['D']]. / | \ B C D | And the (general) tree A would be simply ['A', ['B']]. | B nodes and references customary to have a _list_ of children Though the book doesn't make a big deal of this, A the two tree implementations aren't really all / that different. Consider the binary tree shown B at right. / \ C D As a list of lists, this is ['A', ['B', ['C', [], []], ['D', [], []]]]. Inside the Python interpreter, each list is represented by an array, with elements that are ,-----------, references to objects. So the | , , , | list looks the second picture '-|---|---|-' shown at right -- a picture that | | X looks an awful lot like the 'A' | corresponding nodes and references v picture. ,-----------, | , , , | The only real difference is that '-|---|---|-' in the nodes and references | | v implementation, each box is an 'B' | ,-----------, object, which fields named "key", | | , , , | "left", and "right", while in the | '-|---|---|-' list of lists implementation, it's | | X X an array (a Python list) with slots | 'D' numbered 0, 1, and 2. v ,-----------, | , , , | '-|---|---|-' | X X 'C' One minor difference: we can represent an empty list tree as [], which Python recognizes as a list. For a node-and-reference tree, we have None, but Python can't tell that's a null reference to a tree -- it has no type associated with it. (This tends to make recursion more cumbersome in the nodes-and-references case.) Building node-and-reference trees class BinTree: def __init__(self, val, L = None, R = None): self.key = val self.left = L self.right = R (I don't care much for the notion of "pushing down" an existing node in the routines in the book. People don't usually build trees that way, and the decision of pushing to the left or right in the book code is arbitrary, which doesn't make good sense.) ---------------------------------------- Example application: syntax trees (as used internally by the Python interpreter) import operator def evaluate(t): # t should be an expression tree, as constructed by buildExprTree opers = {'+':operator.add, '-':operator.sub, '*':operator.mul, '/':operator.div} if t.left and t.right: return opers[t.key](evaluate(t.left), evaluate(t.right)) else: return t.key >>> evaluate(BinTree('+', BinTree(3), BinTree(4))) 7 Fully parenthesized expressions are easy to parse: class ParseError(Exception): pass def buildExprTree(str): # str should be a fully parenthesized expression def parse(L): # L should be a list of remaining tokens if L[0] == '(': del L[0] leftOperand = parse(L) if L[0] not in '+-*/': raise ParseError operator = L[0] del L[0] rightOperand = parse(L) if L[0] == ')': del L[0] else: raise ParseError return BinTree(operator, leftOperand, rightOperand) else: v = eval(L[0]) del L[0] return BinTree(v) return parse(str.split()) ---------------------------------------- Binary tree traversals class BinTree: # ... def preorder(self): yield self if self.left: for n in self.left.preorder(): yield n if self.right: for n in self.right.preorder(): yield n def inorder(self): if self.left: for n in self.left.inorder(): yield n yield self if self.right: for n in self.right.inorder(): yield n def postorder(self): if self.left: for n in self.left.postorder(): yield n if self.right: for n in self.right.postorder(): yield n yield self >>> t = BinTree('A', BinTree('B', BinTree('D'), None), BinTree('C')) >>> for n in t.inorder(): print n.key D B A C >>> for n in t.preorder(): print n.key A B D C >>> for n in t.postorder(): print n.key D B C A One could also handle enumeration by passing in a function to be executed at every node -- or, better yet, a triple of functions to be executed before the node, at the node, and after the node: prefix infix postfix before before before at recurse left recurse left recurse left at reccurse right resurce right recurse right at after after after If we wanted to print the tree as a fully parenthesized expression, we'd say something like myTree.infix(printLeftParen, printKey, printRightParen) Here the three print routines need to have been previously defined: def printLeftParen(n): if n.left: print "( ", def printKey(n): print n.key, " ", def printRightParen(n): if n.right: print ") ", ======================================== Binary search trees The book talks strictly in terms of dictionaries (mappings), but a binary search tree can obviously also be used to implement a set. Key idea: node is larger than everything in the left subtree; less than everything in the right subtree. Example from Fig. 5.14: 70 / \ 31 93 / / \ 14 73 94 \ 23 This can be created by starting with an empty tree and adding, in order, 70, 31, 93, 94, 14, 23, 73 or 70, 31, 14, 23, 93, 73, 94 or any other sequence in which parents in the picture appear before their children. Others insertion orders (e.g., anything not starting with 70) would produce a different tree -- but one that would still be correct (would satisfy the binary search tree property). << See code in bin_search_tree.py Based on the book, but with a lot of fixes. >> Useful to distinguish between BinarySearchTree and TreeNode. Former can have no keys, but still exist (not be None), so you can call methods on it (which examine leftChild [root] to see what to do). (Calling the root 'leftChild' and letting its parent reference refer to the tree object avoids a bug when deleting the key in the root.) Note the various double-underscore methods for built-in operators: __setitem__ T[k] = v __getitem__ v = T[k] __contains__ k in T __delitem__ del T[k] __len__ len(T) __iter__ for k in T __str__ str(T), print T (I've defined this in a little more terse way than our previous list notation.) Most of these are defined on BinSearchTree (only). __iter__ and __str__ are defined on TreeNode, too, for convenience. get method of TreeNode is a straightforward recursive routine put method is only slightly more complicated: returns 0 or 1 depending on whether it added or overwrote an existing pair. Wrapper in BinarySearchTree then updates size accordingly. get and delete_key methods raise KeyError if key is not found delete is the hard case. starts by finding node, as in get then there are three cases: node has no children -- trivial node has one child -- easy: promote it node has two children -- replace with successor (could just as easily have used predecessor) note: successor has to have zero or one children, so deletion task doesn't recurse findSuccessor if have a right child, find its left-most node (findMin helper method) else if left child of parent, successor _is_ parent (but this will never replace us, because we only replace with successor when we have two children, which falls under the previous case) else (have no right child, and am right child of parent) successor is parent's successor with me temporarily out of consideration spliceOut called only on nodes with no left child; not designed for the general case __iter__ method is basically in-order traversal ---------------------------------------- notion of balance determines Big-O cost of get, put, delete Section 5.6.3 mentions AVL and red-black binary search tree algorithms, which keep the search tree balanced -- or rather, within a constant factor of balanced. There are also balanced search algorithms that aren't always binary. 2-3 trees and their generalization (B-trees) keep all keys in the leaves of the tree (none in internal nodes), and maintain the invariant that all leaves have the same depth. Skip lists, which we will consider soon, are a set/dictionary data structures that have many of the properties of balanced search trees. All this stuff gets covered at some length in CSC 282. Here's a very brief overview. AVL trees are perhaps the easiest to understand. They maintain the invariant that the subtrees below a given node differ in height by at most 1. Every node contains (in addition to a key/value pair) an indication of the height of the tree below it. When we insert or delete a node, we go back up the path to the root adjusting the height values. If at some point we discover that we and our sibling now differ in height by more than 1 (i.e., by 2), we do a ROTATION to correct the imbalance. If this changes the height of the whole subtree, we continue upward. Worst case, we may have to proceed all the way to the root, and do O(log n) rotations. Red-black trees are more complicated. They maintain the following invariants: - every node is red of black - the root and the (null) leaves are black - the children of a red node are black - all paths from the root to the leaves contain the same number of black nodes Since one can never have two red nodes in a row on a path from the root to a leaf, the shortest and longest root-leaf paths differ by at most a factor of 2. When we insert or delete a node, we recolor and/or rotate as necessary to preserve the invariants. There are a lot of cases to consider. They all have the property, however, that at most two (necessarily adjacent) rotations suffice to restore the invariants. There's an interesting tradeoff here: red-black trees have cheaper insertion and removal (by a constant factor), but more expensive lookup (again by a constant factor). The reason for the difference is that red-black trees can be more imbalanced than AVL trees, because red-black siblings can differ in height by as much as a factor of two. It's easy to see that a red-black tree can be about 2 log n high (one long path; a bunch of short paths). It turns out that an AVL tree can be at most about 1.44 log n high (this is a little harder to show; it has to do with Fibonacci numbers). 2-3 trees aren't binary: every node has 0, 2, or 3 children. They keep all values in the leaves, and maintain the invariant that all leaves are at the same height. They're pretty simple to explain: when you insert a leaf, you "split" its parent into two nodes if it would otherwise have had four children. This means giving the grantparent an extra child, which may cause another split. You continue all the way up to the root, adding an extra level if necessary. Deletions are symmetric. B trees are a generalization of 2-3 trees in which every internal node has between (k+1)/2 and k children, for some constant k (in 2-3 trees, k is 3). B trees are widely used for indices in database systems, where we put up with fat nodes (which require internal linear searching) in order to have as few levels as possible (each of which requires a disk access). ---------------------------------------- Skip lists Can be thought of as a list of lists or, if you squint your eyes a bit, a tree with threaded levels. Probabilistic choice of whether to extend a tower. See code in skip_list.py (slightly modified from book) fewer Boolean flags helper classes nested inside SkipList class ---------------------------------------- OctTrees (Section 7.5) color spaces http://www.microscopy.fsu.edu/primer/java/primarycolors/additiveprimaries/index.html http://www.microscopy.fsu.edu/primer/java/primarycolors/subtractiveprimaries/index.html quantization bit manipulations