Instructor:
Dan Gildea office hours Tu/Th 23:15pm
TAs:

Linfeng Song office hours M/W 10:3011:30 CSB 631
 Joseph Majesky office hours F 1011 CSB 727
 Chester Holtz office hours W 12:151:15 CSB 727
Location: M/W 910:15am, Morey 321
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/18 
Regression and Classification 
constrained optimization, perceptron
 bishop 1.2, 1.4, app E 
1/23 
Support vectors 
stochastic gradient descent
 bishop 3.1, 4.1 
1/25 
Logistic Regression 
maximum entropy
 bishop 4.3 
1/30 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
2/1 
Deep Learning 
dropout
 Krizhevsky 2012 
2/6 
Support Vectors 
strong duality
 bishop 7.1 
2/8 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/13 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
2/15 
Graphical Models 
bayes ball
 
2/20 
Probabilistic Inference 
message passing
 bishop 8.4 
2/22 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/27 
Tree decomposition cont'd 
vertex elimination
 
3/1 
Review 
 
3/6 
Midterm 
 
3/8 
Midterm Solutions 
 
3/20 
Expectation Maximization 
L = Q + H + D
 bishop 9 
3/22 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/27 
HMM cont'd 
minimum bayes risk
 
3/29 
Learning Theory 
PAC
 Kearns and Vazirani 
4/3 
Learning Theory 
VC dimension
 
4/5 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
4/10 
Sampling cont 
Gibbs Sampling
 bishop 11.3 
4/12 
Optimization 
Newton's method, DFP
 Nocedal 8.1, 9.1 
4/17 
Optimization, cont'd 
LBFGS
 Nocedal 8.1, 9.1 
4/19 
PCA 
Eigenvectors
 
4/24 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/26 
Random graphs 
RobbinsMonro
 Snijders (2002) 
5/1 
Review 
come to class with questions!
 
5/3 
Review 
 
Final Exam: Wednesday May 10, 8:3011:30am, Morey 321.
Grading
 Homeworks: 50%
 Final exam: 30%
 Midterm: 20%
gildea @ cs rochester edu
April 25, 2017
