TA: Hao Zhang

Location: TTh 11:05am-12:20pm, CSB 703.

Text: David J. C. MacKay, Information Theory, Inference, and Learning Algorithms

Recommended: Trevor Hastie, Robert Tibshirani, Jerome Friedman, The elements of statistical learning: data mining, inference, and prediction. Dana Ballard, Natural Computation.

On | we will cover | which means that after class you will understand | if before class you have read |
---|---|---|---|

1/13 | Introduction | ||

1/18 | Probability Theory | independence, bayes rule | charniak |

1/20 | Information Theory | entropy, kl-distance, coding | mackay ch 2 |

1/25 | Probabilistic Inference | priors: bayesian reasoning, MAP | heckerman |

1/27 | Probabilistic Inference | priors on continuous variables | mackay ch 24 |

2/1 | Minimum Description Length | decision trees | mackay ch 28 |

2/3 | Probabilistic Inference | polytree | mackay ch 26 |

2/8 | Expectation Maximization | latent variable clustering | bilmes § 1-3 |

2/10 | Independent Component Analysis | source separation | mackay ch 34 |

2/15 | Learning Theory | probably approximately correct | kearns&vazirani ch 1 |

2/17 | Learning Theory | VC dimension | kearns&vazirani ch 2, 3 |

2/22 | Eigenvectors | least squares, PCA | bishop 310-314, appendix E |

2/24 | Nonlinear Dimensionality Reduction | isomap, locally linear embedding | roweis; tenenbaum |

3/1 | Optimization | conjugate gradient | shewchuk § 1-9 |

3/3 | Optimization | Gibbs Sampling, MCMC | mackay ch 29 |

3/15 | Review | ||

3/17 | Midterm | ||

3/22 | Midterm Solutions | aspect model | |

3/24 | MCMC, Gibbs | (continued from before midterm) | mackay ch 38, 39 |

3/29 | Backpropagation | the chain rule | bishop 140-148 |

3/31 | Support Vectors | the wolfe dual | hastie ch 12 |

4/5 | Support Vectors | the kernel trick | |

4/7 | Hidden Markov Models | forward-backward | ballard ch 10; bilmes § 4 |

4/12 | Reinforcement Learning | q-learning | ballard ch 11 |

4/14 | Reinforcement Learning | partial observability | ballard ch 11 |

4/19 | Games | dana ballard | |

4/21 | Games | learning to co-operate | hauert, zhu |

4/26 | Review | come to class with questions! |

- Final exam: 35%
- Homeworks: 35%
- Midterm: 25%
- Class participation: 5%

gildea @ cs rochester edu April 25, 2005