Oct 15 Dynamic programming. Reading: [DVP Ch 6]. Longest increasing subsequences. Edit distance. Next class: Knapsack. Two "thousand pound gorillas" of algorithms: dynamic programming linear programming Dynamic program: approach to algorithm design that generalizes - divide and conquer - shortest paths in DAGS ********************************** Relation to divide and conquer: - breaking problems into subproblems - but not necessarily disjoint subproblems! - subproblems might not be exactly same kind of problem as original one ********************************** Relation to shortest paths in DAGS: dag-shortest-path(G) initialize source node dist to 0 initialize non-source nodes dist to infinity linearize G for each v in linearized order: dist(v) = min_(u,v) {dist(u) + length(u,v)} end - order solution of substeps by DAG - but DAG may only be implicit! - saving itermediate results ********************************** Longest increaseing subsequence 5 2 8 6 3 6 9 7 longest increasing subsequence is 2 3 6 9 (show arrows) * Draw drag of increasing subsequences Subproblems: L(j) = length of longest increasing subsequence that ends at j *****> Note how subproblem is slightly different from original problem! for j = 1, ... , n: L(j) = 1 + max{ L(i) : (i, j) in E } // max is 0 if set is empty return max_j L(j) This gives length of longest increasing subsequence. To get the sequence itself: save backpointers. Notation: argmax_i f(i) = "the i that maximizes f(i)" for j = 1, ... , n: L(j) = 1 + max{ L(i) : (i, j) in E } // max is 0 if set is empty Prev(j) = argmax_i { L(i) : (i, j) in E } // argmax is 0 if set is empty return argmax_j L(j) ********************************** Edit Distance Measure of similarity between strings: - how many insertions, deletions, and substitutions needed to change string x into string y? Applications: spell checking, approximate name matching, genetic sequence matching Principle: find best alignment of strings SUNNY SNOWY S - N O W Y S U N N - Y cost? 3 - S N O W - Y S U N - - N Y cost? 5 goal: align x[1..m] and y[1..n] What would a subproblem be? One approach: consider aligning PREFIX of x with a PREFIX of y. E(i,j) = edit distance x[1..i] and y[1..j] Need to now express E(i,j) in terms of subproblems. Consider what happens with x[i] and x[j]. Four cases: x[i] / - - / y[j] x[i] y[j] and they match x[i] y[j] and they do not match diff(i,j) = 1 if i=/=j and 0 if equal E(i,j) = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) } Visualize E(i,j) as table -- DRAW TABLE can fill out in any order, as long as E(i,j) can be calculated Base cases: E(0,j) = distance between 0-length prefix of x (empty string) and first j letters y == j E(i,0) = distance between 0-length prefix of y (empty string) and first i letters x == i for i = 0 to m: E[i,0] = i for j = 0 to n: E[0,j] = j for i = 1 to m for j = 1 to n E[i,j] = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) } return E[m,n] Run Demo!