Kearns and Vazirani exercises 3.1, 3.3, 3.4
Convert the following simple belief networks into the factor graph representation, and run the polytree (sum-product) algorithm by hand, computing expressions for each message in terms of the parameters of the original belief net, and showing that the final result gives the desired probability:
Due Friday Feb 17 5pm
Implement EM fitting of a mixture of gaussians on the two-dimensional data set hw4.dat. You should (at a minimum) try different numbers of mixtures, as well as tied vs. separate covariance matrices for each gaussian. Which model seems to fit the data best?
Implement the Decision Tree algorithm described in class, including the MDL criterion. Test it using the following dataset: voting2.dat. Turn in your code along with a report describing how the algorithm worked, and any observations about the types of trees it learned. You should try varying (at least) the amount of training data (the training/test split is up to you), whether MDL is used, and how the description length is weighted. You may use any programming language.
Wasserman ch. 1 ex. 10, 11, 12, 13, 14, 15, 17; ch. 2 ex. 14; ch. 3 ex. 4, 5, 14