Instructor:
Dan Gildea office hours Tu/Th 23:15pm
Location: M/W 910:15am, Morey 321
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/17 
Regression and Classification 
constrained optimization, perceptron
 bishop 1.2, 1.4, app E 
1/22 
Support vectors 
stochastic gradient descent
 bishop 3.1, 4.1 
1/24 
Logistic Regression 
maximum entropy
 bishop 4.3 
1/29 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
1/31 
Deep Learning 
dropout
 Krizhevsky 2012 
2/5 
Support Vectors 
strong duality
 bishop 7.1 
2/7 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/12 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
2/14 
Graphical Models 
bayes ball
 
2/19 
Probabilistic Inference 
message passing
 bishop 8.4 
2/21 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/26 
Tree decomposition cont'd 
vertex elimination
 
2/28 
Review 
 
3/5 
Midterm 
 
3/7 
Midterm Solutions 
 
3/19 
Expectation Maximization 
L = Q + H + D
 bishop 9 
3/21 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/26 
HMM cont'd 
minimum bayes risk
 
3/28 
Learning Theory 
PAC
 Kearns and Vazirani 
4/2 
Learning Theory 
VC dimension
 
4/4 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
4/9 
Sampling cont 
Gibbs Sampling
 bishop 11.3 
4/11 
Optimization 
Newton's method, DFP
 Nocedal 8.1, 9.1 
4/16 
Optimization, cont'd 
LBFGS
 Nocedal 8.1, 9.1 
4/18 
PCA 
Eigenvectors
 
4/23 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/25 
Something fun 
 
4/30 
Review 
come to class with questions!
 
Final Exam: Wednesday May 10, 8:3011:30am, Morey 321.
Grading
 Homeworks: 50%
 Final exam: 30%
 Midterm: 20%
gildea @ cs rochester edu
October 10, 2017
