Chenliang Xu

pic

Spring 2021 - CSC 577: Advanced Topics in Computer Vision

Basic Information

Meet Time & Location: Tue/Thu 09:40-10:55 (or Fri 13:00-14:00 for CV Seminar Talks) on Zoom
Instructor: Prof. Chenliang Xu
Office Hours: By Appointment

The complete information of the course is on Blackboard.


Description

This course covers advanced research topics in computer vision with an emphasis on learning structured representations and embeddings. Approaches for learning from unimodal (e.g., images and videos), and multimodal data (e.g., vision and language, vision and audio) will be covered and include topics from structured predications, deep learning, and others. The course will be a mix of instructor-led lectures, student-led presentations, paper reading/critiquing, guest lectures, and a semester-long project.

Prerequisites

CSC 249/449 or CSC 246/446 or CSC 298/578 (Deep Learning and Graphical Models) or permission of the instructor.

Course Objectives

PhD Students: gain research experience in computer vision; MS Students: get prepared for a software engineer career in computer vision and artificial intelligence.


Schedule

This is a tentative schedule of the course and is subject to changes over the term.
Course materials will be posted on Blackboard Learning Modules.
The default time for CV Seminar Talks is Friday 1-2 PM.

W Date Topic Comment
1 Tue 2/2 Course Overview
Fri 2/5 CV Seminar Talk by the Instructor
2 Tue 2/9 T1 Fairness/Biases in Computer Vision Zhiheng Li
Thu 2/11 T1-A Fair Attribute Classification through Latent Space De-biasing. Vikam V. Ramaswamy, Sunnie S. Y. Kim, Olga Russakovsky. 2020. arXiv Guangyu Sun
3 Tue 2/16 T1-B Learning from Failure: Training Debiased Classifier from Biased Classifier. Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, Jinwoo Shin. 2020. arXiv Neil Zhang
Fri 2/19 CV Seminar Talk by Dr. Chuang Gan (MIT, MIT-IBM Watson AI Lab)
4 Tue 2/23 T2 Visual Generation, GAN, and AR/VR Lele Chen
Fri 2/26 CV Seminar Talk by Dr. Du Tran (Facebook AI)
5 Tue 3/2 T2-A First Order Motion Model for Image Animation. Aliaksandr Siarohin, Stephane Lathuiliere, Sergey Tulyakov, Elisa Ricci, Nicu Sebe. 2020. arXiv Ashwin Menon
Thu 3/4 T2-B Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler. 2020. arXiv Ali Abdulla
6 Tue 3/9 T3 Audio-Visual Modeling Yapeng Tian
Thu 3/11 T3-A SoundSpaces: Audio-Visual Navigation in 3D Environments. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman. 2020. arXiv Chao Huang
7 Tue 3/16 Project Proposal
Thu 3/18 T3-B What Makes Training Multi-modal Classification Networks Hard? Weiyao Wang, Du Tran, Matt Feiszli. 2020. arXiv Neil Zhang
8 Tue 3/23 T3-C Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation. Hang Zhou, Xudong Xu, Dahua Lin, Xiaogang Wang, Ziwei Liu. 2020. arXiv Andrew Hahn
Thu 3/25 T4 Instructional Videos and Embodied AI Jing Bi / Jing Shi
9 Tue 3/30 Study Break
Fri 4/2 CV Seminar Talk by Prof. Chen Sun (Brown Univ., Google)
10 Tue 4/6 T4-A Procedure Planning in Instructional Videos. Chien-Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, Juan Carlos Niebles. 2020. arXiv Ali Abdulla
Thu 4/8 T4-B Improving Vision-and-Language Navigation with Image-Text Pairs from the Web. Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, Dhruv Batra. 2020. arXiv Ashwin Menon
11 Tue 4/13 Midway Project Discussion
Fri 4/16 CV Seminar Talk by Prof. Vicente Ordóñez Román (Univ. of Virginia)
12 Tue 4/20 T5 Few-shot and Representation Learning Samuel Lerman
Thu 4/22 T5-A When Does Self-supervision Improve Few-shot Learning? Jong-Chyi Su, Subhransu Maji, Bharath Hariharan. 2020. arXiv Chao Huang
13 Tue 4/27 T5-B Video Playback Rate Perception for Self-supervised Spatio-Temporal Representation Learning. Yuan Yao, Chang Liu, Dezhao Luo, Yu Zhou, Qixiang Ye. 2020. arXiv Guangyu Sun
Fri 4/30 CV Seminar Talk by Prof. Andrew Owens (Univ. of Michigan)
14 T 5/4 Project Presentation
R 5/6 Project Presentation (cont.)

Course Work and Evaluation

All assignments should be submitted on Blackboard, and no late work will be accepted.


Other Policies

Code of Conduct: Check here for classroom "etiquette" expectations.

Academic Honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available here. You are expected to read, understand, and follow the policy.

Disabilities: Please see me about your required accommodations as early as possible in the term. The University of Rochester respects and welcomes students of all backgrounds and abilities. In the event you encounter any barrier(s) to full participation in this course due to the impact of a disability, please contact the Office of Disability Resources. The access coordinators in the Office of Disability Resources can meet with you to discuss the barriers you are experiencing and explain the eligibility process for establishing academic accommodations. You can reach the Office of Disability Resources at: disability@rochester.edu; (585) 276-5075; Taylor Hall; link.

Accommodations for COVID-Related Illness: Please let me know asap. At the same time, You should contact the College Center for Advising Services (CCAS) or the Graduate Education and Postdoctoral Affairs office. These offices will be able to assist you and refer you to the right resources.