Instructor:
Dan Gildea office hours M/W 2-3:15pm
TA: Xiaochang Peng, office hours Tu/Th 4-5pm
Location: M/W 3:25pm-4:40pm, Dewey 2210E
Vector Calculus review: Fri 1/23, 1-2pm, CSB 601
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
- Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
- Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
- Larry Wasserman, All of Statistics, 2004.
Readings not taken from Bishop are available through online.
Syllabus
On | we will cover | which means that after class you will understand | if before class you have read |
1/14 |
Probability Theory |
independence, bayes rule
| bishop 1.2, 1.4, app E |
1/21 |
Information Theory |
entropy, kl-distance, coding
| bishop 1.6 |
1/26 |
Gaussians |
covariance, conditioning
| bishop 2.3 |
1/28 |
Probabilistic Inference |
priors: bayesian reasoning, MAP
| bishop 3.4 |
2/2 |
Perceptron |
stochastic gradient descent
| bishop 3.1, 4.1 |
2/4 |
Backpropagation |
DP for gradient descent
| bishop 5.1, 5.2, 5.3 |
2/9 |
Support Vectors |
strong duality
| bishop 7.1 |
2/11 |
Support Vectors |
the kernel trick
| bishop 6.1, 6.2 |
2/16 |
Probabilistic Inference |
message passing
| bishop 8.4 |
2/18 |
Tree decomposition |
cyclic graphs
| koller and friedman |
2/23 |
Expectation Maximization |
L = Q + H + D
| bishop 9 |
2/25 |
Expectation Maximization |
mixture of gaussians
| bishop 9 |
3/2 |
Sampling |
Markov Chain Monte Carlo
| bishop 11.2 |
3/4 |
Review |
| |
3/16 |
Midterm |
| |
3/18 |
Learning theory |
|
|
3/23 |
Midterm Solutions |
| |
3/25 |
Sampling cont |
Gibbs Sampling
| bishop 11.3 |
3/30 |
Learning Theory |
VC dimension
| Kearns and Vazirani |
4/1 |
Logistic Regression |
maximum entropy
| bishop 4.3 |
4/6 |
Hidden Markov Models |
forward-backward
| bishop 13.2 |
4/8 |
HMM cont'd |
minimum bayes risk
| |
4/13 |
Optimization |
Newton's method, DFP
| Nocedal 8.1, 9.1 |
4/15 |
Optimization cont'd |
LBGFS
| |
4/20 |
Particle Filters |
slam
| Thrun et al. ch 4 |
4/22 |
PCA |
Eigenvectors
| |
4/27 |
Reinforcement Learning |
q-learning
| sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 |
4/29 |
Review |
come to class with questions!
| |
Final Exam: Monday May 4, 8:30-11:30am, Dewey 2210E.
Grading
- Final exam: 30%
- Homeworks: 45%
- Midterm: 20%
- Class participation: 5%
gildea @ cs rochester edu
April 20, 2015
|