Ji Liu


CSC 576, Fall 2014, Modern (Computational) Approaches to Big Data Analytics

Course Description

    Big data analytics refers to the process of collecting, organizing, and analyzing large sets of data ("big data") to discover patterns and other useful information. Not only will big data analytics help you to understand the information contained within the data, but it will also help identify the data that is most important to the task and the future decisions. This course introduces some important modern approaches / computational models to big data analytics in machine learning and optimization communities.

    The following topics will be covered
  • computational methods for large scale linear systems
  • matrix / tensor decomposition / completion / recovery (recommendation system, image/video in-painting, video surveillance)
  • 1st order optimization methods: unconstrained optimization methods (gradient descent, optimal 1st order method, stochastic gradient method), constrained optimization methods (ADMM, smoothing scheme)
  • feature selection and sparse learning (L1 norm minimization, nuclear norm, group sparsity, greedy algorithms)

  • Basic knowledge in linear algebra, matrix analysis, and statistics, and basic concepts in optimization (such as convex set / function) are useful for you to follow this course smoothly. Research experiences in machine learning, signal processing, and computer vision / image processing would be definitely helpful to understand this course better, but not required.

Course Schedule

  • Sep. 2 (Tuesday): Overview [slides] [syllabus]
  • Sep. 4 (Thursday) Sep. 9 (Tuesday): Review for Linear Algebra and Matrix Analysis [notes] [reference-Stanford-CS229]
  • Sep. 11 (Thursday) Sep. 16 (Tuesday) Sep. 18 (Thursday): (Large) Linear System [notes]
  • Sep. 23 (Tuesday) Sep. 25 (Thursday): Matrix Decomposition / Completion [notes] [slides] [other materials]
  • Sep. 30 (Tuesday) Oct. 2 (Thursday) Oct. 7 (Tuesday): Sparsity: Sparse Learning and Compressed Sensing [notes]
  • Oct. 9 (Thursday) Oct. 16 (Thursday): Variants in Sparse Learning and Compressed Sensing [notes]
  • Oct. 21 (Tuesday): Optimization: Foundations [notes] [reference book]
  • Oct. 23 (Thursday) Oct. 28 (Tuesday): Gradient Descent [notes] [reference book]
  • [reference paper]
  • Oct. 30 (Thursday): Coordinate Descent [notes]
  • Nov. 4 (Tuesday): Accelerated (Optimal) Gradient Descent [notes] [reference paper]
  • Nov. 6 (Thursday): Stochastic Gradient "Descent" [notes]
  • Nov. 11 (Tuesday) Nov. 13 (Thursday) Nov. 18 (Tuesday): ADMM [notes] [reference book]
  • Nov. 20 (Thursday) Nov. 25 (Tuesday): Feature Selection [notes]

Homework

  • Homework 1 (Due Oct. 7) [pdf]
  • Homework 2 (Due Nov. 4) [pdf]
  • Homework 3 (Due Dec. 2) [pdf]

Paper Presentation

    Everybody is required to present an academic paper in this course. You can choose any paper published in ICML, NIPS, SIGKDD, CVPR, ICCV, JMLR, TKDD, or TPAMI. If you want to present a paper outside of this set, please let me know and discuss with me first. It would be better if the selected paper is closely related to your project. A list of recommended papers is provided below and will be updated from time to time (Of course, you do not have to select from this list at all). The date for your presentation is flexible. You can choose any class to do your presentation before 12/01/2014. Please note that each class can only contain at most three presentations. Each paper presentation lasts 15 minutes (12 mins presentation + 3 mins question section). I will send you a schedule table link by email. Please fill it asap. You are not allowed to change your schedule anymore if your original presentation date is coming in five days. If you have to, please contact me first. If you have not received the schedule link by 10/10/2014, please let me know.

  • Robust Inverse Covariance Estimation under Noisy Measurements, ICML 2014
  • Efficient Label Propagation, ICML 2014
  • Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization, ICML 2014
  • Coupled Group Lasso for Web-Scale CTR Prediction in Display Advertising, ICML 2014
  • Near-Optimal Joint Object Matching via Convex Relaxation, ICML 2014
  • A Single-Pass Algorithm for Efficiently Recovering Sparse Cluster Centers of High-dimensional Data, ICML 2014

Project

    The final project presentation starts on 12/02/2014. Our last two weeks are used for our final project presentation. Every team is required to present their project in class. I will send you a schedule table link by email. Please fill it asap. You are not allowed to change your schedule anymore if your original presentation date is coming in five days. If you have to, please contact me first. If you have not received the schedule link by 10/10/2014, please let me know.

  • Project Proposal (Due Oct. 16): Submit a proposal with 1-2 pages to describe the project you want to work on. This proposal can be a part of your final project.
  • Project Presentation (Dec. 02 - 11)
  • Final report submission by Email (Dec. 21 11:59:59pm)