Instructor:
Dan Gildea office hours M/W 12pm, 3019 Wegmans
TAs:
Location: Tu/Th 11:0512:20am, Gavett 202
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/13 
Regression and Classification 
constrained optimization, perceptron
 bishop 1.2, 1.4, app E 
1/18 
Logistic Regression 
stochastic gradient descent
 bishop 4.3 
1/20 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
1/25 
Deep Learning 
dropout
 Krizhevsky 2012 
1/27 
Support Vectors 
maxmargin
 bishop 3.1, 4.1 
2/1 
Support Vectors 
strong duality
 bishop 7.1 
2/3 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/8 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
2/10 
Graphical Models 
bayes ball
 
2/15 
Probabilistic Inference 
message passing
 bishop 8.4 
2/17 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/22 
Tree decomposition cont'd 
vertex elimination
 
2/24 
Review 
 
3/1 
Midterm 
 
3/3 
Midterm Solutions 
 
3/15 
Fairness in machine learning 
 Kleinberg, CorbettDavies 
3/17 
Expectation Maximization 
L = Q + H + D
 bishop 9 
3/22 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/24 
EM for HMM 
minimum bayes risk
 
3/29 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
3/31 
Metropolis Hastings 
detailed balance
 bishop 11.2 
4/5 
Gibbs sampling 
annealing
 bishop 11.3 
4/7 
Learning Theory 
PAC
 Kearns and Vazirani 
4/12 
Gradient Descent 
SGD convergence
 Ruder 2016 
4/14 
GAN/VAE 
deep generative models
 Goodfellow et al. 2014, Kingma and Welling 2014 
4/19 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/21 
Something fun 
 
4/26 
Review 
come to class with questions!
 
Final exam: Friday May 6, 8:3011:30am
Grading
 Homeworks: 50%
 Final exam: 30%
 Midterm: 20%
gildea @ cs rochester edu
April 13, 2022
