Meet Time & Location: Tue/Thu 09:40-10:55 (or Fri 13:00-14:00 for CV Seminar Talks) on Zoom
Instructor: Prof. Chenliang Xu
Office Hours: By Appointment
The complete information of the course is on Blackboard.
This course covers advanced research topics in computer vision with an emphasis on learning structured representations and embeddings. Approaches for learning from unimodal (e.g., images and videos), and multimodal data (e.g., vision and language, vision and audio) will be covered and include topics from structured predications, deep learning, and others. The course will be a mix of instructor-led lectures, student-led presentations, paper reading/critiquing, guest lectures, and a semester-long project.
CSC 249/449 or CSC 246/446 or CSC 298/578 (Deep Learning and Graphical Models) or permission of the instructor.
PhD Students: gain research experience in computer vision; MS Students: get prepared for a software engineer career in computer vision and artificial intelligence.
This is a tentative schedule of the course and is subject to changes over the term.
Course materials will be posted on Blackboard Learning Modules.
The default time for CV Seminar Talks is Friday 1-2 PM.
W | Date | Topic | Comment |
---|---|---|---|
1 | Tue 2/2 | Course Overview | |
Fri 2/5 | CV Seminar Talk by the Instructor | ||
2 | Tue 2/9 | T1 Fairness/Biases in Computer Vision | Zhiheng Li |
Thu 2/11 | T1-A Fair Attribute Classification through Latent Space De-biasing. Vikam V. Ramaswamy, Sunnie S. Y. Kim, Olga Russakovsky. 2020. arXiv | Guangyu Sun | |
3 | Tue 2/16 | T1-B Learning from Failure: Training Debiased Classifier from Biased Classifier. Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, Jinwoo Shin. 2020. arXiv | Neil Zhang |
Fri 2/19 | CV Seminar Talk by Dr. Chuang Gan (MIT, MIT-IBM Watson AI Lab) | ||
4 | Tue 2/23 | T2 Visual Generation, GAN, and AR/VR | Lele Chen |
Fri 2/26 | CV Seminar Talk by Dr. Du Tran (Facebook AI) | ||
5 | Tue 3/2 | T2-A First Order Motion Model for Image Animation. Aliaksandr Siarohin, Stephane Lathuiliere, Sergey Tulyakov, Elisa Ricci, Nicu Sebe. 2020. arXiv | Ashwin Menon |
Thu 3/4 | T2-B Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler. 2020. arXiv | Ali Abdulla | |
6 | Tue 3/9 | T3 Audio-Visual Modeling | Yapeng Tian |
Thu 3/11 | T3-A SoundSpaces: Audio-Visual Navigation in 3D Environments. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman. 2020. arXiv | Chao Huang | |
7 | Tue 3/16 | Project Proposal | |
Thu 3/18 | T3-B What Makes Training Multi-modal Classification Networks Hard? Weiyao Wang, Du Tran, Matt Feiszli. 2020. arXiv | Neil Zhang | |
8 | Tue 3/23 | T3-C Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation. Hang Zhou, Xudong Xu, Dahua Lin, Xiaogang Wang, Ziwei Liu. 2020. arXiv | Andrew Hahn |
Thu 3/25 | T4 Instructional Videos and Embodied AI | Jing Bi / Jing Shi | |
9 | Tue 3/30 | Study Break | |
Fri 4/2 | CV Seminar Talk by Prof. Chen Sun (Brown Univ., Google) | ||
10 | Tue 4/6 | T4-A Procedure Planning in Instructional Videos. Chien-Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, Juan Carlos Niebles. 2020. arXiv | Ali Abdulla |
Thu 4/8 | T4-B Improving Vision-and-Language Navigation with Image-Text Pairs from the Web. Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, Dhruv Batra. 2020. arXiv | Ashwin Menon | |
11 | Tue 4/13 | Midway Project Discussion | |
Fri 4/16 | CV Seminar Talk by Prof. Vicente Ordóñez Román (Univ. of Virginia) | ||
12 | Tue 4/20 | T5 Few-shot and Representation Learning | Samuel Lerman |
Thu 4/22 | T5-A When Does Self-supervision Improve Few-shot Learning? Jong-Chyi Su, Subhransu Maji, Bharath Hariharan. 2020. arXiv | Chao Huang | |
13 | Tue 4/27 | T5-B Video Playback Rate Perception for Self-supervised Spatio-Temporal Representation Learning. Yuan Yao, Chang Liu, Dezhao Luo, Yu Zhou, Qixiang Ye. 2020. arXiv | Guangyu Sun |
Fri 4/30 | CV Seminar Talk by Prof. Andrew Owens (Univ. of Michigan) | ||
14 | T 5/4 | Project Presentation | |
R 5/6 | Project Presentation (cont.) |
All assignments should be submitted on Blackboard, and no late work will be accepted.
Code of Conduct: Check here for classroom "etiquette" expectations.
Academic Honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available here. You are expected to read, understand, and follow the policy.
Disabilities: Please see me about your required accommodations as early as possible in the term. The University of Rochester respects and welcomes students of all backgrounds and abilities. In the event you encounter any barrier(s) to full participation in this course due to the impact of a disability, please contact the Office of Disability Resources. The access coordinators in the Office of Disability Resources can meet with you to discuss the barriers you are experiencing and explain the eligibility process for establishing academic accommodations. You can reach the Office of Disability Resources at: disability@rochester.edu; (585) 276-5075; Taylor Hall; link.
Accommodations for COVID-Related Illness: Please let me know asap. At the same time, You should contact the College Center for Advising Services (CCAS) or the Graduate Education and Postdoctoral Affairs office. These offices will be able to assist you and refer you to the right resources.