Instructor:
Dan Gildea office hours M/Th 23:15pm
TAs: Xiaochang Peng,
Linfeng Song
Location: M/W 910:15am, Morey 321
Prereqs: Probability, Linear Algebra, Vector Calculus.
There will be a vector calculus review Fri 1/15, 45pm, in CSB 209.
Homeworks
Lecture notes and other readings available through NB.
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/13 
Regression and Classification 
pseudoinverse, constrained optimization
 bishop 1.2, 1.4, app E 
1/20 
Information Theory 
entropy, kldistance, coding
 bishop 1.6 
1/25 
Probabilistic Inference 
priors: bayesian reasoning, MAP
 bishop 3.4 
1/27 
Perceptron 
stochastic gradient descent
 bishop 3.1, 4.1 
2/1 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
2/3 
Deep Learning 
dropout
 Krizhevsky 2012 
2/8 
Support Vectors 
strong duality
 bishop 7.1 
2/10 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/15 
Probabilistic Inference 
message passing
 bishop 8.4 
2/17 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/22 
Expectation Maximization 
L = Q + H + D
 bishop 9 
2/24 
Expectation Maximization 
mixture of gaussians
 bishop 9 
2/29 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
3/2 
Review 
 
3/14 
Midterm 
 
3/16 
Learning theory 


3/21 
Midterm Solutions 
 
3/23 
Sampling cont 
Gibbs Sampling
 bishop 11.3 
3/28 
Learning Theory 
VC dimension
 Kearns and Vazirani 
3/30 
Logistic Regression 
maximum entropy
 bishop 4.3 
4/4 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
4/6 
HMM cont'd 
minimum bayes risk
 
4/11 
Optimization 
Newton's method, DFP
 Nocedal 8.1, 9.1 
4/13 
Particle Filters 
slam
 Thrun et al. ch 4 
4/18 
PCA 
Eigenvectors
 
4/20 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/25 
AlphaGo 
 Silver 2016 
4/27 
Review 
come to class with questions!
 
Final Exam: Wednesday May 4, 8:3011:30am, Morey 321.
Grading
 Final exam: 30%
 Homeworks: 45%
 Midterm: 20%
 Class participation: 5%
gildea @ cs rochester edu
July 7, 2016
