Chenliang Xu

pic

Selected Publications

For a full list of publications, please refer to my CV.

2023


A Whac-AWole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others
Zhiheng Li, Ivan Evtimov,Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[ paper ]

Egocentric Audio-Visual Object Localization
Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[ paper ]

2022


Discover and Mitigate Unknown Biases with Debiasing Alternate Networks
Zhiheng Li, Anthony Hoogs, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2022
[ paper ]

Face Forgery Detection via Symmetric Transformer
Luchuan Song, Xiaodan Li, Zheng Fang, Zhenchao Jin, Yuefeng Chen, and Chenliang Xu
ACM International Conference on Multimedia (ACM MM), 2022
[ paper ]

Cross-Modal Contrastive Distillation for Instructional Activity Anticipation
Zhengyuan Yang, Jinggen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, and Jiebo Luo
International Conference on Pattern Recognition (ICPR), 2022
[ paper ]

StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis
Zhiheng Li, Martin Renqiang Min, Kai Li, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper ]

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing
Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper ]

Learning to Answer Questions in Dynamic Audio-Visual Scenarios
Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, and Di Hu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
Oral Presentation
[ paper ]

Transformer-Empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution
Guangyuan Li, Jun Lv, Yapeng Tian, Dou Qi, Chengyan Wang, Chenliang Xu, and Jing Qin
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper ]

Audio-Visual Object Localization in Egocentric Videos
Chao Huang, Yapeng Tian, Anurag Kumar, Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2022
[ paper ]

2021


Discover the Unknown Biased Attribute of an Image Classifier
Zhiheng Li and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper ]

Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning
Jing Bi, Jiebo Luo, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
Oral Presentation
[ paper ]

Explaining Local, Global, And Higher-Order Interactions In Deep Learning
Samuel Lerman, Charles Venuto, Henry Kautz, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper | code ]

A Simple Baseline for Weakly-Supervised Scene Graph Generation
Jing Shi, Yiwu Zhong, Ning Xu, Yin Li, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper ]

Learning to Generate Scene Graph from Natural Language Supervision
Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, and Yin Li
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper ]

Can audio-visual integration strengthen robustness under multimodal attacks?
Yapeng Tian and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | code ]

Cyclic Co-learning of Sounding Object Visual Grounding and Sound Separation
Yapeng Tian, Di Hu, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | code ]

High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
Lele Chen, Chen Cao, Fernando De la Torre, Jason Saragih, Chenliang Xu, and Yaser Sheikh
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | project ]

Learning by Planning: Language-Guided Global Image Editing
Jing Shi, Ning Xu, Yihang Xu, Trung Bui, Franck Dernoncourt, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | project ]

Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation
Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021
[ paper | supp ]

How to Make a BLT Sandwich? Learning VQA towards Understanding Web Instructional Videos
Shaojie Wang, Wentian Zhao, Ziyi Kou, Jing Shi, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021
[ paper | data ]

2020


Graph Neural Network based Coarse-Grained Mapping
Zhiheng Li, Geemi P. Wellawatte, Maghesree Chakraborty, Heta A. Gandhi, Chenliang Xu, and Andrew D. White
Chemical Science, 2020
[ paper ]

A Benchmark and Baseline for Language-Driven Image Editing
Jing Shi, Ning Xu, Trung Bui, Franck Dernoncourt, Zheng Wen, and Chenliang Xu
Asian Conference on Computer Vision (ACCV), 2020
Oral Presentation
[ paper | supp ]

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing
Yapeng Tian, Dingzeyu Li, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2020
Spotlight Presentation (5%)
[ paper | supp | demo | arXiv | project ]

Talking-head Generation with Rhythmic Head Motion
Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2020
[ paper | supp | demo | arXiv | project ]

Deep Grouping Model for Unified Perceptual Parsing
Zhiheng Li, Wenxuan Bao, Jiayang Zheng, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | arXiv ]

Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
Jie Chen, Zhiheng Li, Jiebo Luo, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
Oral Presentation (5.7%)
[ paper | arXiv ]

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution
Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | supp | arXiv | code | demo ]

TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | arXiv | code | demo ]

Co-Learn Sounding Object Visual Grounding and Visually Indicated Sound Separation in A Cycle
Yapeng Tian, Di Hu, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2020
[ paper ]

Deep Audio Prior: Learning Sound Source Separation from a Single Audio Mixture
Yapeng Tian, Chenliang Xu, and Dingzeyu Li
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2020
[ paper ]

Weakly-Supervised Audio-Visual Video Parsing Toward Unified Multisensory Perception
Yapeng Tian, Dingzeyu Li, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2020
[ paper ]

What comprises a good talking-head video generation?
Lele Chen, Guofeng Cui, Ziyi Kou, Haitian Zheng, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2020
[ paper ]

End-to-End Generation of Talking Faces from Noisy Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020
[ paper ]

TailorGAN: Making User-Defined Fashion Design
Lele Chen, Justin Tian, Guo Li, Cheng-Haw Wu, Erh-Kan King, Kuan-Ting Chen, Shao-Hang Hsieh, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020
[ paper | arXiv | project ]

Assembling Semantically-Disentangled Representations for Predictive-Generative Models via Adaptation from Synthetic Domain
Burkay Donderici, Caleb New, and Chenliang Xu
arXiv preprint, 2020
[ arXiv ]

Learning from Interventions Using Hierarchical Policies for Safe Learning
Jing Bi, Vikas Dhiman, Tianyou Xiao, and Chenliang Xu
AAAI Conference on Artificial Intelligence (AAAI), 2020
Oral Presentation (under 5%)
[ paper | arXiv | talk | poster ]

2019


3D Human Avatar Digitization from a Single Image
Zhong Li, Lele Chen, Celong Liu, Yu Gao, Yuanzhou Ha, Chenliang Xu, Shuxue Quan, and Yi Xu
ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI), 2019
Best Paper Award
[ paper ]

Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss
Lele Chen, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019
[ paper | poster | code ]

Not All Frames Are Equal: Weakly-Supervised Video Grounding with Contextual Similarity and Visual Clustering Losses
Jing Shi, Jia Xu, Boqing Gong, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019
[ paper | poster | code ]

GAN-EM: GAN based EM Learning Framework
Wentian Zhao, Shaojie Wang, Zhihuai Xie, Jing Shi, and Chenliang Xu
International Joint Conference on Artificial Intelligence (IJCAI), 2019
[ paper | supp | slides | poster | code ]

Dynamic Graph Modules for Modeling Object-Object Interactions in Activity Recognition
Hao Huang, Luowei Zhou, Wei Zhang, Jason J. Corso, and Chenliang Xu
British Machine Vision Conference (BMVC), 2019
[ paper | project ]

Can Multisensory Training Aid Visual Learning?: A Computational Investigation
Robert A. Jacobs and Chenliang Xu
Journal of Vision (JOV), 2019
[ paper ]

Audio-Visual Event Localization in the Wild
Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2019
Oral Presentation
[ paper ]

Audio-Visual Interpretable and Controllable Video Captioning
Yapeng Tian, Chenxiao Guan, Justin Goodman, Marc Moore, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2019
[ paper ]

Sound to Visual: Hierarchical Cross-Modal Talking Face Video Generation
Lele Chen, Haitian Zheng, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu
Sight and Sound Workshop, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR Workshop), 2019
[ paper ]

A Weakly Supervised Multi-Task Ranking Framework for Actor-Action Semantic Segmentation
Yan Yan, Chenliang Xu, Dawen Cai, and Jason J. Corso
International Journal of Computer Vision (IJCV), 2019
[ paper ]

Noise-resilient Training Method for Face Landmark Generation from Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
IEEE/ACM Transcations on Audio, Speech and Language Processing (TASLP), 2019
[ paper ]

Online Audio-Visual Source Association for Chamber Music Performances
Bochen Li, Karthik Dinesh, Chenliang Xu, Gaurav Sharma, and Zhiyao Duan
Transcations of the International Society for Music Information Retrieval (TISMIR), 2019
[ paper ]

2018


Lip Movements Generation at a Glance
Lele Chen, Zhiheng Li, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2018
[ paper | arXiv | demo | code ]
The system is also an accepted Demo at ECCV'18.

Audio-Visual Event Localization in Unconstrained Videos
Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2018
[ paper | arXiv | project]

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Li Ding and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018
[ paper | arXiv ]

Towards Automatic Learning of Procedures from Web Instructional Videos
Luowei Zhou, Chenliang Xu, and Jason J. Corso
AAAI Conference on Artificial Intelligence (AAAI), 2018
Oral Presentation
[ paper | arXiv | YouCookII ]

Improving Text-based Person Search by Spatial Matching and Adaptive Threshold
Tianlang Chen, Chenliang Xu, and Jiebo Luo
IEEE Winter Conference on Applications of Computer Vision (WACV), 2018
[ paper ]

MRI Tumor Segmentation with Densely Connected 3D CNN
Lele Chen, Yue Wu, Adora M. DSouza, Anas Z. Abidin, Axel Wismüller, and Chenliang Xu
SPIE Conference on Medical Imaging (SPIE), 2018
Oral Presentation
[ paper | arXiv | code | talk ]

Generating Talking Face Landmarks from Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
International Conference on Latent Variable Analysis and Signal Separation (LVA-ICA), 2018
[ paper | arXiv ]

Encoding and Selecting Coarse-Grain Mapping Operators with Hierarchical Graphs
Maghesree Chakraborty, Chenliang Xu, and Andrew D. White
Journal of Chemical Physics (JCP), 2018
[ paper | arXiv ]

2017


Audio-Visual Source Association for String Ensembles through Multi-Modal Vibrato Analysis
Bochen Li, Chenliang Xu, and Zhiyao Duan
Sound and Music Computing (SMC), 2017
Best Paper Award
[ paper ]

Deep Cross-Modal Audio-Visual Generation
Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, and Chenliang Xu
ACM International Conference on Multimedia Thematic Workshops (ACMMMW), 2017
[ paper ]

Watch What You Just Said: Image Captioning with Text-Conditional Attention
Luowei Zhou, Chenliang Xu, Parker Koch, and Jason J. Corso
ACM International Conference on Multimedia Thematic Workshops (ACMMMW), 2017
[ paper ]

Weakly Supervised Actor-Action Segmentation via Robust Multi-Task Ranking
Yan Yan, Chenliang Xu, Dawen Cai, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017
[ paper ]

One-Shot Video Object Segmentation with Iterative Online Fine-Tuning
Amos Newswanger and Chenliang Xu
The DAVIS Challenge on Video Object Segmentation, CVPR Workshop (CVPR Workshop), 2017
[ paper | poster ]

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
Li Ding and Chenliang Xu
arXiv Report arXiv:1705.07818
[ arXiv ]

Action Understanding with Multiple Classes of Actors
Chenliang Xu, Caiming Xiong, and Jason J. Corso
arXiv Report arXiv:1704.08723
[ arXiv ]

Dancelets Mining for Video Recommendation Based on Dance Styles
Tingting Han, Hongxun Yao, Chenliang Xu, Xiaoshuai Sun, Yanhao Zhang and Jason J. Corso
IEEE Transactions on Multimedia (TMM), 2016
[ paper ]

2016 and Before


Scale-Adaptive Video Understanding
Chenliang Xu
PhD Thesis, University of Michigan, 2016
[ PDF ]

Actor-Action Semantic Segmentation with Grouping Process Models
Chenliang Xu and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016
Spotlight Presentation (9.7%)
[ paper | video | poster ]

LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing
Chenliang Xu and Jason J. Corso
International Journal of Computer Vision (IJCV), 2016
[ paper | arXiv | project ]

Can Humans Fly? Action Understanding with Multiple Classes of Actors
Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015
[ paper | video | poster | data | code ]

A Study of Actor and Action Semantic Retention in Video Supervoxel Segmentation
Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, and Jason J. Corso
International Journal of Semantic Computing (IJSC), 2013
[ paper | arXiv ]

Flattening Supervoxel Hierarchies by the Uniform Entropy Slice
Chenliang Xu, Spencer Whitt, and Jason J. Corso
IEEE/CVF International Conference on Computer Vision (ICCV), 2013
[ paper | video | poster | project ]

Are Actor and Action Semantics Retained in Video Supervoxel Segmentation?
Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, and Jason J. Corso
IEEE International Conference on Semantic Computing (ICSC), 2013
[ paper | slides ]

A Thousand Frames in Just a Few Words: Lingual Description of Videos through Latent Topics and Sparse Object Stitching
Pradipto Das, Chenliang Xu, Richard F. Doell, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2013
[ paper | poster | project ]

TRECVID 2013 GENIE: Multimedia Event Detection and Recounting
S. Oh, A. Perera, I. Kim, M. Pandey, K. Cannons, H. Hajimirsadeghi, A. Vahdat, G. Mori, B. Miller, S. McCloskey, Y.-C. Cheng, Z. Huang, C.-H. Lee, C. Xu, R. Kumar, W. Chen, J. Corso, L. Fei-Fei, D. Koller, V. Ramanathan, K. Tang, A. Joulin, A. Alahi
NIST TRECVID Workshop, 2013
[ paper ]

Streaming Hierarchical Video Segmentation
Chenliang Xu, Caiming Xiong, and Jason J. Corso
European Conference on Computer Vision (ECCV), 2012
Oral Presentation (2.8%)
[ paper | oral video | spotlight video | project ]

Evaluation of Super-Voxel Methods for Early Video Processing
Chenliang Xu and Jason J. Corso
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012
[ paper | poster | project ]

TRECVID 2012 GENIE: multimedia event detection and recounting
A. Perera, S. Oh, M. Pandey, T. Ma, A. Hoogs, A. Vahdat, K. Cannons, G. Mori, S. McCloskey, B. Miller, S. Venkatesh, P. Davalos, P. Das, C. Xu, J. Corso, R. Srihari, I. Kim, Y.-C. Cheng, Z. Huang, C.-H. Lee, K. Tang, L. Fei-Fei, D. Koller
NIST TRECVID Workshop, 2012
[ paper ]