Oct 15 Dynamic programming.
Reading: [DVP Ch 6]. Longest increasing subsequences. Edit distance.
Next class: Knapsack.
Two "thousand pound gorillas" of algorithms:
dynamic programming
linear programming
Dynamic program: approach to algorithm design that generalizes
- divide and conquer
- shortest paths in DAGS
**********************************
Relation to divide and conquer:
- breaking problems into subproblems
- but not necessarily disjoint subproblems!
- subproblems might not be exactly same kind of problem
as original one
**********************************
Relation to shortest paths in DAGS:
dag-shortest-path(G)
initialize source node dist to 0
initialize non-source nodes dist to infinity
linearize G
for each v in linearized order:
dist(v) = min_(u,v) {dist(u) + length(u,v)}
end
- order solution of substeps by DAG
- but DAG may only be implicit!
- saving itermediate results
**********************************
Longest increaseing subsequence
5 2 8 6 3 6 9 7
longest increasing subsequence is 2 3 6 9 (show arrows)
* Draw drag of increasing subsequences
Subproblems: L(j) = length of longest increasing subsequence that ends at j
*****> Note how subproblem is slightly different from original problem!
for j = 1, ... , n:
L(j) = 1 + max{ L(i) : (i, j) in E } // max is 0 if set is empty
return max_j L(j)
This gives length of longest increasing subsequence.
To get the sequence itself: save backpointers.
Notation:
argmax_i f(i) = "the i that maximizes f(i)"
for j = 1, ... , n:
L(j) = 1 + max{ L(i) : (i, j) in E } // max is 0 if set is empty
Prev(j) = argmax_i { L(i) : (i, j) in E } // argmax is 0 if set is empty
return argmax_j L(j)
**********************************
Edit Distance
Measure of similarity between strings:
- how many insertions, deletions, and substitutions needed to change
string x into string y?
Applications: spell checking, approximate name matching, genetic sequence matching
Principle: find best alignment of strings
SUNNY SNOWY
S - N O W Y
S U N N - Y cost? 3
- S N O W - Y
S U N - - N Y cost? 5
goal: align x[1..m] and y[1..n]
What would a subproblem be?
One approach: consider aligning PREFIX of x with a PREFIX of y.
E(i,j) = edit distance x[1..i] and y[1..j]
Need to now express E(i,j) in terms of subproblems.
Consider what happens with x[i] and x[j]. Four cases:
x[i] / -
- / y[j]
x[i] y[j] and they match
x[i] y[j] and they do not match
diff(i,j) = 1 if i=/=j and 0 if equal
E(i,j) = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) }
Visualize E(i,j) as table -- DRAW TABLE
can fill out in any order, as long as E(i,j) can be calculated
Base cases:
E(0,j) = distance between 0-length prefix of x (empty string) and first j letters y == j
E(i,0) = distance between 0-length prefix of y (empty string) and first i letters x == i
for i = 0 to m:
E[i,0] = i
for j = 0 to n:
E[0,j] = j
for i = 1 to m
for j = 1 to n
E[i,j] = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) }
return E[m,n]
Run Demo!