CSC 575: Interactive Machine Learning

Many of the existing human interfaces require input from the human during training, in the form of labels, comments, and parameter guidance. In many of those applications, users only have implicit knowledge, and thus unable to label a large training dataset. Furthermore, for many practical applications, users have their own explicit goal of  their task which often could be translated as a stopping criterion of "performance is good enough, no further parameter tuning is necessary". The goal of this class is to explore the latest machine learning algorithms that allow humans to be part of the training process.

In addition to learning how to use the state-of-the-art machine learning framework, we will study the latest literature on machine learning algorithms that outlines new possibilities of human-machine symbiosis. Much of the course will be focused on building towards and executing a final research project that involves a machine learning algorithm with a human in the loop.


How Interactive Machine Learning class different than the Machine Learning class?
Interactive machine learning focuses on methods that empower domain experts to control and direct machine learning tools from within the deployed environment, whereas traditional machine learning does this in the development environment. This difference allows interactive machine learning systems to be more responsive, more accurate and cheaper to develop and maintain.
 


Instructor:

M. Ehsan Hoque
Email: mehoque at cs
Office: CSB 710
Office hours: Tuesdays from 10 am to noon. Additional hours are available on request.
 

Course TA
Ru Zhao
Email: rzhao2@u.rochester.edu
 

Prerequisites:

Permission from the Instructor

Grading:

This Google spread sheet contains the updated info on attendance and class participation.

Final Project = 50% (Proposal 10%, Prototype 20%, Video 10%, Presentation 10%)
Homework = 10%
Lectures and class participation = (40%) (Present one lecture 20%, present on a machine learning toolkit 10%, actively participate 10%)

This Google spread
 

Schedule

Date Topic Assigned papers
1/14/2015 Introduction + syllabus + policy  
1/21/2015 Introduction to Machine Learning Precision, Recall, ROC Curve, Validation Techniques, Feature Selection
1/26/2015 Introduction to Machine Learning (cont) A few useful things to know about Machine Learning
Support Vector Machine in Javascript
1/28/2015 Brief tutorial on how to use Weka http://www.cs.waikato.ac.nz/ml/weka/

http://wekinator.cs.princeton.edu/

2/02/2015 Brief tutorial on how to Scikit Python Scripts http://scikit-learn.org/stable/
2/4/2015 Introduction to HMM - Graphical Models  
2/9/2015 Introduction to HMM - Inference and Learning  
2/11/2015 Guest Lecture by Saleema Amershi from Microsoft Research Power to the People: The Role of Humans in Interactive Machine Learning.

ReGroup: Interactive Machine Learning for On-Demand Group Creation in Social Networks

Structured Labeling to Facilitate Concept Evolution in Machine Learning

2/16/2015 Machine Learning toolbox exploration Orange (Raul & Walter), Spider (Emre), RapidMiner (Renan)
2/18/2015 Machine Learning toolbox exploration Theano (Tanveer & Rafayet), Mallet (Yina), HCRF (David)
2/23/2015 Machine Learning toolbox exploration Infer.NET (Michelle & Kanishk), NLTK, Python code (Taylan)
2/25/2015 End-user Programming (Discussion Leader: Yina Jin)
Homework 1
End-user Programming of Mashups with Vegemite

Flock: Hybrid Crowd-Machine Learning Classifiers

3/2/2015 Attribute based and Implied feedback (Discussion Leader: Rafayet Ali) Simultaneous Active Learning of Classifiers & Attributes via Relative Feedback

Implied Feedback: Learning Nuances of User Behavior in Image Search

https://filebox.ece.vt.edu/~parikh/demos.htm

3/4/2015 Final project brainstorming session  
3/16/2015 Hybrid Human-Machine CRFs (Discussion Leader: Walter Checefsky) Final Project Proposal is due

Analyzing Semantic Segmentation using Hybrid Human-Machine CRF's

Bringing Semantics into Focus Using Visual Abstraction

3/18/2015 Active Learning - Multi-label classification (Discussion Leader: Emre and David) Active Learning for Sparse Bayesian Multi-label Classification
3/23/2015 Final Project Proposal Presentation  
3/25/2015 Active Learning - Model Selection (Discussion Leader: Renan & Kanishk) Active Learning with Model Selection

Using Multiple Models to Understand Data

3/30/2015 No class - Ehsan is away  
4/1/2015 Active Learning - Selective Sampling (Taylan Sen) Selective Sampling using the Query by Committee Algorithm
4/6/2015 Guest Lecture by Tom Howard  
4/8/2015 Active Learning (Discussion Leader: Iftekhar Tanveer & Raul) Employing EM in pool-based Active Learning for Text Classification
4/13/2015 Interactive Video Annotation (Discussion Leader: Michelle Fung) Efficiently Scaling Up Crowdsourced Video Annotation

JAABA: interactive machine learning for automatic annotation of animal behavior download

4/15/2015 Guest Lecture by Ji Liu  
4/20/2015 Guest Lecture by Jiebo Luo  
4/22/2015 Guest Lecture by Brad Knox  
4/27/2015 Final Project Presentation + videos  
4/29/2015 Review & Discussion