Instructor:
Dan Gildea office hours M/W 23:15pm
TA: Xiaochang Peng, office hours Tu/Th 45pm
Location: M/W 3:25pm4:40pm, Dewey 2210E
Vector Calculus review: Fri 1/23, 12pm, CSB 601
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
 Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
 Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
 Larry Wasserman, All of Statistics, 2004.
Readings not taken from Bishop are available through online.
Syllabus
On  we will cover  which means that after class you will understand  if before class you have read 
1/14 
Probability Theory 
independence, bayes rule
 bishop 1.2, 1.4, app E 
1/21 
Information Theory 
entropy, kldistance, coding
 bishop 1.6 
1/26 
Gaussians 
covariance, conditioning
 bishop 2.3 
1/28 
Probabilistic Inference 
priors: bayesian reasoning, MAP
 bishop 3.4 
2/2 
Perceptron 
stochastic gradient descent
 bishop 3.1, 4.1 
2/4 
Backpropagation 
DP for gradient descent
 bishop 5.1, 5.2, 5.3 
2/9 
Support Vectors 
strong duality
 bishop 7.1 
2/11 
Support Vectors 
the kernel trick
 bishop 6.1, 6.2 
2/16 
Probabilistic Inference 
message passing
 bishop 8.4 
2/18 
Tree decomposition 
cyclic graphs
 koller and friedman 
2/23 
Expectation Maximization 
L = Q + H + D
 bishop 9 
2/25 
Expectation Maximization 
mixture of gaussians
 bishop 9 
3/2 
Sampling 
Markov Chain Monte Carlo
 bishop 11.2 
3/4 
Review 
 
3/16 
Midterm 
 
3/18 
Learning theory 


3/23 
Midterm Solutions 
 
3/25 
Sampling cont 
Gibbs Sampling
 bishop 11.3 
3/30 
Learning Theory 
VC dimension
 Kearns and Vazirani 
4/1 
Logistic Regression 
maximum entropy
 bishop 4.3 
4/6 
Hidden Markov Models 
forwardbackward
 bishop 13.2 
4/8 
HMM cont'd 
minimum bayes risk
 
4/13 
Optimization 
Newton's method, DFP
 Nocedal 8.1, 9.1 
4/15 
Optimization cont'd 
LBGFS
 
4/20 
Particle Filters 
slam
 Thrun et al. ch 4 
4/22 
PCA 
Eigenvectors
 
4/27 
Reinforcement Learning 
qlearning
 sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 
4/29 
Review 
come to class with questions!
 
Final Exam: Monday May 4, 8:3011:30am, Dewey 2210E.
Grading
 Final exam: 30%
 Homeworks: 45%
 Midterm: 20%
 Class participation: 5%
gildea @ cs rochester edu
April 20, 2015
