Instructor:
Dan Gildea office hours M/W 12pm Wegmans Hall 3019
TAs:
Location: Tu/Th 11:0512:20 Dewey 2162
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/16 
Regression and Classification 
constrained optimization, perceptron
 bishop 1.2, 1.4, app E 
1/21 
Logistic Regression 
stochastic gradient descent
 bishop 4.3 
1/23 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
1/28 
Deep Learning 
dropout
 Krizhevsky 2012 
1/30 
Support Vectors 
maxmargin
 bishop 3.1, 4.1 
2/4 
Support Vectors 
strong duality
 bishop 7.1 
2/6 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/11 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
2/13 
Graphical Models 
bayes ball
 
2/18 
Probabilistic Inference 
message passing
 bishop 8.4 
2/20 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/25 
Tree decomposition cont'd 
vertex elimination
 
2/27 
Review 
 
3/3 
Midterm 
 
3/5 
Midterm Solutions 
 
3/17 
Expectation Maximization 
L = Q + H + D
 bishop 9 
3/19 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/24 
EM for HMM 
minimum bayes risk
 
3/26 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
3/31 
Metropolis Hastings 
detailed balance
 bishop 11.2 
4/2 
Gibbs sampling 
annealing
 bishop 11.3 
4/7 
Learning Theory 
PAC
 Kearns and Vazirani 
4/9 
Learning Theory 
VC dimension
 
4/14 
Gradient Descent 
SGD convergence
 Ruder 2016 
4/16 
PCA 
Eigenvectors
 
4/21 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/23 
something fun 
 
4/28 
Review 
come to class with questions!
 
Final Exam: Fri May 8, 8:3011:30am, Douglass Ballroom.
Grading
 Homeworks: 50%
 Final exam: 30%
 Midterm: 20%
gildea @ cs rochester edu
February 3, 2020
