Lecture 13 Memoization
22 Oct 2013
Memoization - technique for converting a recursive algorithm directly into a dynamic programming algorithm.
Idea: cache values previous computed in a hash table.
Before initial call to f(x):
h = makeHashtable;
function f(x)
if h.hasKey(x) then return h.getVal(x);
// calculate solution
h.insert(x, solution)
return solution
end
Example:
function fib(n)
if n = 0: return 0
if n = 1: return 1
return fib(n-1) + fib(n-2)
==>
function fib(n)
if h.hasKey(n) return h.getVal(n)
if n = 0: return 0
if n = 1: return 1
sol = fib(n-1) + fib(n-2)
h.insert(n,sol)
return sol;
end
*************************************************
Knapsack dynamic programming solution:
procedure knapsack()
K(0) = 0
for c = 1 to C:
K(c) = max {K(c - wi) + vi : wi <= C}
return K(C)
end
Recursive memoized version:
Top level: call K(C)
function K(c)
if h.hasKey(c) return h.getVal(c)
if c = 0 then sol = 0
else
sol = max {K(c - wi) + vi : wi <= C}
h.insert(c,sol)
return sol;
end
*************************************************
Do the dynamic and memoized versions have the same O complexity?
YES
Which is typically faster in practice?
DYNAMIC
Why?
hashtable look ups
recursion stack
When could the memoized version be faster in practice?
- If the hash table were retained between calls to the top level function,
and the same subsproblem were solved repeatedly
e.g.: Fib(50) + Fib(49)
e.g.: editDistance("snake", "snack")
editDistance("baker", "black")
- If the top level problem can be solved without filling out the dynamic programming table completely
e.g.: Knapsack where the C and each w_i are divisible by some integer K.
No subsolution K(c) for c not divisible are K are needed!
*************************************************
What about space useage - which is generally more space efficient, an array or
hash table?
Array
Suppose you do not have enough memory to store the entire dynamic programming array. How could memoization be used?
When hashtable becomes full, then start deleting elements when you need to add another one.
Which to delete?
Most frequent heuristic: least recently used
Note that this handles case of hashtable retained between problems!
method h.getVal(key)
record = ....
t = t + 1
record.accessTime = t
return record.value
end
method h.insert(key,val)
record = ...
t = t + 1
record.accessTime = t
record.value = val
end
method h.release(B)
for each record in h
if record.accessTime < t - B then
delete record from h
end
*************************************************