Chenliang Xu


Fall 2018 - CSC 577: Advanced Topics in Computer Vision

Instructor: Prof. Chenliang Xu
Time/Place: TR 09:40-10:55 in 315 B&L
Office Hours: Tue 11:00-12:00 in 3005 Wegmans
The syllabus is below.


This course covers advanced research topics in computer vision with an emphasis on learning structured representations and embeddings. Approaches for learning from unimodal (e.g., images and videos), and multimodal data (e.g., vision and language, vision and audio) will be covered and include topics from structured predications, deep learning and others. The course will be a mix of lecture, student presentation and discussion. Prerequisites: CSC 249/449 or CSC 246/446 or CSC 298/578 (Deep Learning and Graphical Models) or permission of the instructor.


PhD Students: gain research experience in computer vision; MS Students: get prepared for a software engineer career in computer vision and artificial intelligence.


This is a tentative schedule of the course.
Course materials will be posted on Blackboard.
Red: lecture; Blue: student presentation; Green: project presentation; Grey: no class or cancelled class.

W Date Topic Presenter
1 R 8/30 Overview, Boot Camp: Neural Networks
2 T 9/4 Boot Camp: Convolutional Neural Networks
2 R 9/6 Boot Camp: Recurrent Neural Networks
3 T 9/11 Wang et al. CVPR 2018: Non-local Neural Networks Songyang, Wei Z.
3 R 9/13 Feichtenhofer et al. CVPR 2018: What have we learned from deep representations for action recognition? Hanlin, Zhiheng
4 T 9/18 Zhou et al. CVPR 2016: Learning Deep Features for Discriminative Localization Chen, Ziyi
4 R 9/20 Zhang et al. CVPR 2018: Interpretable Convolutional Neural Networks Tianjie, Yutong
5 T 9/25 Castro et al. ECCV 2018: End-to-End Incremental Learning Weijian, Wei X.
5 R 9/27 Hu et al. CVPR 2018: Relation Networks for Object Detection Haitian, Jing
6 T 10/2 Hu et al. CVPR 2018: Learning to Segment Every Thing Shaojie, Yiming
6 R 10/4 Project Proposal Everyone
7 T 10/9 Gavrilyuk et al. CVPR 2018: Actor and Action Video Segmentation from a Sentence Justin, Rohit
7 R 10/11 Chen et al. CVPR 2018: Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning Wei Z., Zhiheng
8 T 10/16 Fall Break
8 R 10/18 Lan et al. CVPR 2018: FFNet: Video Fast-Forwarding via Reinforcement Learning Jing, Tianjie
9 T 10/23 Babaeizadeh et al. ICLR 2018: Stochastic Variational Video Prediction Hanlin, Ziyi
9 R 10/25 Wichers et al. ICML 2018: Hierarchical Long-term Video Prediction without Supervision Rohit, Yiming
10 T 10/30 Yan et al. ECCV 2018: MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics Shaojie, Songyang
10 R 11/1 Wiles et al. ECCV 2018: X2Face: A network for controlling face generation using images, audio, and pose codes Justin, Wei X.
11 T 11/6 Nagrani et al. CVPR 2018: Seeing Voices and Hearning Faces: Cross-modal biometric matching Haitian, Weijian
11 R 11/8 Yeh et al. CVPR 2018: Unsupervised Textual Grounding: Linking Words to Image Concepts Chen, Yutong
12 T 11/13 Bouritsas et al. CVPR 2018: Multimodal Visual Concept Learning with Weakly Supervised Techniques Hanlin, Ziyi
12 R 11/15 Cancelled for CVPR
13 T 11/20 Yang et al. ECCV 2018: Graph R-CNN for Scene Graph Generation Shaojie, Haitian
13 R 11/22 Thanksgiving Break
14 T 11/27 Wang et al. CVPR 2018: Video Captioning via Hierarchical Reinforcement Learning Jing, Tianjie
14 R 11/29 Li et al. CVPR 2018: Visual Question Generation as Dual Task of Visual Question Answering Weijian, Yiming
15 T 12/4 Hudson et al. ICLR 2018: Compositional Attention Networks for Machine Reasoning Wei Z., Rohit
15 R 12/6 Project Presentation Group 1
16 T 12/11 Project Presentation Group 2

Group 1 (unordered): Justin, Songyang, Wei Z., Wei X., Zhiheng, Weijian, Jing, and Yutong.
Group 2 (unordered): Rohit, Hanlin, Chen, Yiming, Haitian, Shaojie, Tianjie, and Ziyi.

Work and Grade

All assignments should be submitted on Blackboard and no late work will be accepted.

Textbooks: There are no required textbooks. We will read papers from CVPR, NIPS etc.

Code of Conduct: Check here for classroom "etiquette" expectations.

Academic honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at here. You are expected to read, understand and follow the policy.

Disabilities: If you need an accommodation based on the impact of a disability, please contact me privately during the first two weeks to discuss your specific needs. Alternatively, contact Center for Excellence in Teaching and Learning at 585-275-9049 in room 1-154 Dewey Hall to coordinate reasonable accommodations.