CSC 576, Fall 2015, Modern (Computational) Approaches to Big Data Analytics
Lecturer: Ji Liu, Office hour: Tue & Thu 4:40-5:40pm
TA: Yuncheng Li (firstname.lastname@example.org) TA office hour: Wed 1-2pm
Big data analytics refers to the process of collecting, organizing, and analyzing large sets of data ("big data") to discover patterns and other useful information. Not only will big data analytics help you to understand the information contained within the data, but it will also help identify the data that is most important to the task and the future decisions. This course introduces some important modern approaches / computational models to big data analytics in machine learning and optimization communities.
- computational methods for large scale linear systems
- matrix / tensor decomposition / completion / recovery (recommendation system, image/video in-painting, video surveillance)
- 1st order optimization methods: unconstrained optimization methods (gradient descent, optimal 1st order method, stochastic gradient method), constrained optimization methods (ADMM, smoothing scheme)
- feature selection and sparse learning (L1 norm minimization, nuclear norm, group sparsity, greedy algorithms)
- others, e.g., submodular optimization
This course is sort of "mathematical" and especially emphasizes deep understanding of fundamental numerical algorithms. This course would NOT cover any software or tools used in data analytics. Basically, this is NOT an application course.
The following topics will be covered
Basic knowledge in linear algebra, matrix analysis, and statistics, and basic concepts in optimization (such as convex set / function) are useful for you to follow this course smoothly. Research experiences in machine learning, signal processing, and computer vision / image processing would be definitely helpful to understand this course better, but not required.
- Sep. 1 (Tuesday): Overview [slides] [syllabus]
- Sep. 3 (Thursday): No class today.
- Section 1: Review for Linear Algebra and Matrix Analysis [notes] [reference-Stanford-CS229]
- Section 2: (Large) Linear System [notes]
- Section 3: Matrix Decomposition / Completion [notes] [slides] [other materials]
- Section 4: Sparsity: Sparse Learning and Compressed Sensing [notes] [reference 1] [reference 2] [course notes]
- Section 5: Variants in Sparse Learning and Compressed Sensing [notes]
- Section 6: Optimization: Foundations [notes] [reference book]
- Section 7: Gradient Descent [notes] [reference book] [reference paper]
- Section 8: Coordinate Descent [notes]
- Section 9: Accelerated (Optimal) Gradient Descent [notes] [reference paper]
- Section 10: Stochastic Gradient "Descent" [notes]
- Section 11: ADMM [notes] [reference book]
- Section 12: Feature Selection [notes]
- Section 13: Submodular Optimization [notes]
Nov. 3, In Class
- Homework 1 (Due on Sep. 22) [pdf]
- Homework 2 (Due Oct. 8) [pdf] [training] [testing]
- Homework 3 (Due Oct. 22) [pdf]
- Homework 4 (Due Nov. 5) [pdf] [dataset]
- Homework 5 & 6 (Due Dec. 10) [pdf]