CSC 282 17 Oct 2013 Dynamic programming continued: Knapsack Problem from last class: how to recover alignment from edit distance algorithm: for i = 0 to m: E[i,0] = i for j = 0 to n: E[0,j] = j for i = 1 to m for j = 1 to n E[i,j] = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) } return E[m,n] Draw rectangle with with i vertical and j horizontal. Record alignment in: AlignX[1..m] = position in Y or 0 for skip AlignY[1..n] = position in X or 0 for skip Change: E[i,j] = min{ 1+E(i-1,j), 1+E(i,j-1), diff(i,j)+E(i-1,j-1) } to up = 1+E(i-1,j) left = 1+E(i,j-1) diag = diff(i,j)+E(i-1,j-1) if up <= left && up <= diag then E[i,j] = up alignX[i] = 0 // skip else if left <= up && left <= diag then E[i,j] = left alignY[j] = 0 // skip else E[i,j] = diag alignX[i] = j alignY[j] = i ***************************************************** Knapsack: Example: given a set of jobs to run, each with some memory requirement and a priority choose a set that can run in parallel without paging that maximizes the sum of priorities Job GB Priority 1 6 30 2 3 14 3 4 16 4 2 9 Computer has 10 GB RAM. With repetitions: can select any number of instances of job. solution: one copy job 1, 2 copies of job 4 30 + 9 + 9 = 48 Without repetitions: can select at most one instance of a job. solution: one copy job 1, 1 copy of job 3 30 + 16 = 46 ***************************************************** Knapsack with repetition: C = total capacity n items weight w1, ..., wn value v1, ..., vn Breaking problem down into smaller problems: What are the two quantities we could consider to limit? - smaller capacity W - few items n Consider smaller capacities: Define: K(c) = maximum value achievable with knapsack of capacity c Express this in terms of solutions to problems with capacities < c. Discuss: how to do this??? Hint: Take one of the items out the solution for K(c). Claim: if optimal solution for K(c) includes item i, then removing i leaves an optimal solution for K(c - wi). Why? Draw picture: If removing i doesn't leave an optimal solution for K(c-wi), then could get a BETTER solution for K(c) as well! So: for SOME i, K(c) = K(c - wi) + vi We don't know which item i. Solution: try them all! K(c) = max {K(c - wi) + vi : wi <= C} algorithm: K(0) = ??? 0 for c = 1 to C: K(c) = max {K(c - wi) + vi : wi <= C} // if set is empty max returns 0 return K(C) How many iterations? C How long does each iteration (max) take? O(n) So running time is? O(nC) How to record actual solution? For K(c) should save what? item removed from the knapsack K(0) = ??? 0 for c = 1 to C: i = argmax_i {K(c - wi) + vi : wi <= C} // if set empty argmax returns 0 if i>0 then K(c) = K(c - wi) + vi item(c) = i else K(c) = 0 item(c) = 0 return K(C) ********************************************* Knapsack without repetitions: Previous solution breaks down: removing an item can lead to a non-optimal solution the smaller problem -- because the actual optimal solution to the smaller problem could have used that item itself! Draw picture: K(c,j) = max value achievable using knapsack of capacity c and items 1,...,j Either item j is needed for optimal solution or it isn't. If it isn't needed: K(c,j) = K(c, j-1) If it is needed: K(c,j) = K(c - wj, j-1) + vj So K(c,j) = max { K(c, j-1), K(c - wj, j-1) + vj } Initialize all K(0,j)=0 and all K(w,0)=0 for j = 1 to n: for c = 1 to C: if wj > c: K(c,j) = K(c, j-1) // why needed? else: K(c,j) = max { K(c, j-1), K(c - wj, j-1) + vj } return K(C,n) running time? O(nC)