Instructor:
Dan Gildea office hours Tu/Th 23:15pm
TAs:
 Xiangru Lian, office hours Thursday 3pm5pm Wegmans Hall 3207 (no office hours 1/18)
 Parker Riley, office hours TBD
 Zhengyuan Yang, office hours Monday 2pm4pm, Wegmans Hall 3504
Location: M/W 910:15am, Morey 321
The course is full. If spaces become available, I will
add people who hand in the first homework. If you would like to
try to add, write "NEED TO ADD" on the top of your assignment, along
with your name, program (eg, BS in CS) and year.
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/17 
Regression and Classification 
constrained optimization, perceptron
 bishop 1.2, 1.4, app E 
1/22 
Support vectors 
stochastic gradient descent
 bishop 3.1, 4.1 
1/24 
Logistic Regression 
maximum entropy
 bishop 4.3 
1/29 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
1/31 
Deep Learning 
dropout
 Krizhevsky 2012 
2/5 
Support Vectors 
strong duality
 bishop 7.1 
2/7 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/12 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
2/14 
Graphical Models 
bayes ball
 
2/19 
Probabilistic Inference 
message passing
 bishop 8.4 
2/21 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/26 
Tree decomposition cont'd 
vertex elimination
 
2/28 
Review 
 
3/5 
Midterm 
 
3/7 
Midterm Solutions 
 
3/19 
Expectation Maximization 
L = Q + H + D
 bishop 9 
3/21 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/26 
HMM cont'd 
minimum bayes risk
 
3/28 
Learning Theory 
PAC
 Kearns and Vazirani 
4/2 
Learning Theory 
VC dimension
 
4/4 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
4/9 
Sampling cont 
Gibbs Sampling
 bishop 11.3 
4/11 
Optimization 
Newton's method, DFP
 Nocedal 8.1, 9.1 
4/16 
Optimization, cont'd 
LBFGS
 Nocedal 8.1, 9.1 
4/18 
PCA 
Eigenvectors
 
4/23 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/25 
Something fun 
 
4/30 
Review 
come to class with questions!
 
Final Exam: Wednesday May 10, 8:3011:30am, Morey 321.
Grading
 Homeworks: 50%
 Final exam: 30%
 Midterm: 20%
gildea @ cs rochester edu
January 18, 2018
