Chenliang Xu

pic

Selected Papers

For a full list of publications, please refer to my CV.

2024

OSCaR: Object State Captioning and State Change Representation
Nguyen Nguyen, Jing Bi, Ali Vosoughi, Yapeng Tian, Pooyan Fazli, and Chenliang Xu
Findings of the Association for Computational Linguistics: NAACL 2024 (NAACL), 2024
[ paper | project ]
Discover and Mitigate Multiple Biased Subgroups in Image Classifiers
Zeliang Zhang, Mingqian Feng, Zhiheng Li, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[ paper | project ]
Learning to Transform Dynamically for Better Adversarial Transferability
Rongyi Zhu, Zeliang Zhang, Susan Liang, Zhuo Liu, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[ paper | project ]
Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA
Ali Vosoughi, Shijian Deng, Yapeng Tian, Chenliang Xu, and Jiebo Luo
IEEE Transactions on Multimedia (TMM), 2024
[ paper | project ]
Random Smooth-based Certified Defense against Text Adversarial Attack
Zeliang Zhang, Wei Yao, Susan Liang, and Chenliang Xu
Findings of the Association for Computational Linguistics: EACL 2024 (EACL), 2024
[ paper ]
One Foward is Enough for Neural Network Training via Likelihood Ratio Method
Jinyang Jiang, Zeliang Zhang, Chenliang Xu, Zhaofei Yu, and Yijie Peng
International Conference on Learning Representations (ICLR), 2024
[ paper | code ]
Adaptive Super Resolution for One-Shot Talking Head Generation
Luchuan Song, Pinxin Liu, Guojun Yin, and Chenliang Xu
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024
[ paper | project ]
Learning Audio Concepts from Counterfactual Natural Language
Ali Vosoughi, Luca Bondi, Ho-Hsiang Wu, and Chenliang Xu
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024
[ paper | project ]

2023

Video Understanding with Large Language Models: A Survey
Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, Ali Vosoughi, Chao Huang, Zeliang Zhang, Feng Zheng, Jianguo Zhang, Ping Luo, Jiebo Luo, and Chenliang Xu
arXiv:2312.17432, 2023
[ paper | project ]
Automated Classification of Big X-Ray Diffraction Data using Deep Learning Models
Jerardo E. Salgado, Samuel Lerman, Zhaotong Du, Chenliang Xu, and Niaz Abdolrahim
npj Cimputational Materials, 2023
[ paper ]
AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu
Neural Information Processing Systems (NeurIPS), 2023
(An early 4-page version was presented at CVPR'23 Sight and Sound Workshop.)
[ paper | project ]
Emotional Listener Portrait: Neural Listener Head Generation with Emotion
Luchuan Song, Guojun Yin, Zhenchao Jin, Xiaoyi Dong, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
[ paper | video ]
Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fields
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu
ICCV 2023 Workshop on AV4D: Visual Learning of Sounds in Spaces, 2023
[ project ]
MISAR: A Multimodal Instructional System with Augmented Reality
Jing Bi, Nguyen Nguyen, Ali Vosoughi, and Chenliang Xu
ICCV 2023 Workshop on AV4D: Visual Learning of Sounds in Spaces, 2023
[ paper ]
Separating Invisible Sounds Toward Universal Audio-Visual Scene-Aware Sound Separation
Yiyang Su, Ali Vosoughi, Shijian Deng, Yapeng Tian, and Chenliang Xu
ICCV 2023 Workshop on AV4D: Visual Learning of Sounds in Spaces, 2023
[ paper ]
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data
Zheng Zhang, Zheng Ning, Chenliang Xu, Yapeng Tian, and Toby Jia-Jun Li
ACM Symposium on User Interface Software and Technology (UIST), 2023
[ paper ]
A Whac-AWole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others
Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian C. Ferrer, Chenliang Xu, and Mark Ibrahim
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[ paper | project | video ]
Egocentric Audio-Visual Object Localization
Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
(An early 4-page version was presented at CVPR'22 Sight and Sound Workshop.)
[ paper | video ]
Audio-Visual Action Prediction with Soft-Boundary in Egocentric Videos
Luchuan Song, Jing Bi, Chao Huang, and Chenliang Xu
CVPR 2023 Workshop on Sight and Sound, 2023
[ paper ]

2022

Discover and Mitigate Unknown Biases with Debiasing Alternate Networks
Zhiheng Li, Anthony Hoogs, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2022
[ paper | project | video ]
Face Forgery Detection via Symmetric Transformer
Luchuan Song, Xiaodan Li, Zheng Fang, Zhenchao Jin, Yuefeng Chen, and Chenliang Xu
ACM International Conference on Multimedia (ACM MM), 2022
[ paper | video | code ]
Cross-Modal Contrastive Distillation for Instructional Activity Anticipation
Zhengyuan Yang, Jinggen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, and Jiebo Luo
International Conference on Pattern Recognition (ICPR), 2022
[ paper ]
StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis
Zhiheng Li, Martin Renqiang Min, Kai Li, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper | project | video ]
SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing
Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper ]
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, and Di Hu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
Oral Presentation
[ paper | project ]
Transformer-Empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution
Guangyuan Li, Jun Lv, Yapeng Tian, Dou Qi, Chengyan Wang, Chenliang Xu, and Jing Qin
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[ paper ]

2021

Discover the Unknown Biased Attribute of an Image Classifier
Zhiheng Li and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper | project | video ]
Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning
Jing Bi, Jiebo Luo, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
Oral Presentation
[ paper ]
Explaining Local, Global, And Higher-Order Interactions In Deep Learning
Samuel Lerman, Charles Venuto, Henry Kautz, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper | code ]
A Simple Baseline for Weakly-Supervised Scene Graph Generation
Jing Shi, Yiwu Zhong, Ning Xu, Yin Li, and Chenliang Xu
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper ]
Learning to Generate Scene Graph from Natural Language Supervision
Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, and Yin Li
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[ paper ]
Can audio-visual integration strengthen robustness under multimodal attacks?
Yapeng Tian and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | code ]
Cyclic Co-learning of Sounding Object Visual Grounding and Sound Separation
Yapeng Tian, Di Hu, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
(An early 4-page version was presented at CVPR'20 Sight and Sound Workshop.)
[ paper | supp | code ]
High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
Lele Chen, Chen Cao, Fernando De la Torre, Jason Saragih, Chenliang Xu, and Yaser Sheikh
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | project ]
Learning by Planning: Language-Guided Global Image Editing
Jing Shi, Ning Xu, Yihang Xu, Trung Bui, Franck Dernoncourt, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[ paper | supp | project ]
Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation
Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021
[ paper | supp ]
How to Make a BLT Sandwich? Learning VQA towards Understanding Web Instructional Videos
Shaojie Wang, Wentian Zhao, Ziyi Kou, Jing Shi, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021
[ paper | data ]

2020

Graph Neural Network based Coarse-Grained Mapping
Zhiheng Li, Geemi P. Wellawatte, Maghesree Chakraborty, Heta A. Gandhi, Chenliang Xu, and Andrew D. White
Chemical Science, 2020
[ paper ]
A Benchmark and Baseline for Language-Driven Image Editing
Jing Shi, Ning Xu, Trung Bui, Franck Dernoncourt, Zheng Wen, and Chenliang Xu
Asian Conference on Computer Vision (ACCV), 2020
Oral Presentation
[ paper | supp ]
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing
Yapeng Tian, Dingzeyu Li, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2020
Spotlight Presentation (5%)
(An early 4-page version was presented at CVPR'20 Sight and Sound Workshop.)
[ paper | supp | demo | arXiv | project ]
Talking-head Generation with Rhythmic Head Motion
Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2020
[ paper | supp | demo | arXiv | project ]
Deep Grouping Model for Unified Perceptual Parsing
Zhiheng Li, Wenxuan Bao, Jiayang Zheng, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | arXiv ]
Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
Jie Chen, Zhiheng Li, Jiebo Luo, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
Oral Presentation (5.7%)
[ paper | arXiv ]
Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution
Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | supp | arXiv | code | demo ]
TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
[ paper | arXiv | code | demo ]
Deep Audio Prior: Learning Sound Source Separation from a Single Audio Mixture
Yapeng Tian, Chenliang Xu, and Dingzeyu Li
CVPR 2020 Workshop on Sight and Sound, 2020
[ paper ]
What comprises a good talking-head video generation?
Lele Chen, Guofeng Cui, Ziyi Kou, Haitian Zheng, and Chenliang Xu
CVPR 2020 Workshop on Sight and Sound, 2020
[ paper ]
End-to-End Generation of Talking Faces from Noisy Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020
[ paper ]
TailorGAN: Making User-Defined Fashion Design
Lele Chen, Justin Tian, Guo Li, Cheng-Haw Wu, Erh-Kan King, Kuan-Ting Chen, Shao-Hang Hsieh, and Chenliang Xu
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020
[ paper | arXiv | project ]
Learning from Interventions Using Hierarchical Policies for Safe Learning
Jing Bi, Vikas Dhiman, Tianyou Xiao, and Chenliang Xu
AAAI Conference on Artificial Intelligence (AAAI), 2020
Oral Presentation (under 5%)
[ paper | arXiv | talk | poster ]

2019

3D Human Avatar Digitization from a Single Image
Zhong Li, Lele Chen, Celong Liu, Yu Gao, Yuanzhou Ha, Chenliang Xu, Shuxue Quan, and Yi Xu
ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI), 2019
Best Paper Award
[ paper ]
Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss
Lele Chen, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019
(A 4-page version was presented at CVPR'19 Workshop on Sight and Sound.)
[ paper | poster | code ]
Not All Frames Are Equal: Weakly-Supervised Video Grounding with Contextual Similarity and Visual Clustering Losses
Jing Shi, Jia Xu, Boqing Gong, and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019
[ paper | poster | code ]
GAN-EM: GAN based EM Learning Framework
Wentian Zhao, Shaojie Wang, Zhihuai Xie, Jing Shi, and Chenliang Xu
International Joint Conference on Artificial Intelligence (IJCAI), 2019
[ paper | supp | slides | poster | code ]
Dynamic Graph Modules for Modeling Object-Object Interactions in Activity Recognition
Hao Huang, Luowei Zhou, Wei Zhang, Jason J. Corso, and Chenliang Xu
British Machine Vision Conference (BMVC), 2019
[ paper | project ]
Can Multisensory Training Aid Visual Learning?: A Computational Investigation
Robert A. Jacobs and Chenliang Xu
Journal of Vision (JOV), 2019
[ paper ]
Audio-Visual Interpretable and Controllable Video Captioning
Yapeng Tian, Chenxiao Guan, Justin Goodman, Marc Moore, and Chenliang Xu
CVPR 2019 Workshop on Sight and Sound, 2019
[ paper ]
A Weakly Supervised Multi-Task Ranking Framework for Actor-Action Semantic Segmentation
Yan Yan, Chenliang Xu, Dawen Cai, and Jason J. Corso
International Journal of Computer Vision (IJCV), 2019
[ paper ]
Noise-resilient Training Method for Face Landmark Generation from Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
IEEE/ACM Transcations on Audio, Speech and Language Processing (TASLP), 2019
[ paper ]
Online Audio-Visual Source Association for Chamber Music Performances
Bochen Li, Karthik Dinesh, Chenliang Xu, Gaurav Sharma, and Zhiyao Duan
Transcations of the International Society for Music Information Retrieval (TISMIR), 2019
[ paper ]

2018

Lip Movements Generation at a Glance
Lele Chen, Zhiheng Li, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2018
[ paper | arXiv | demo | code ]
The system is also an accepted Demo at ECCV'18.
Audio-Visual Event Localization in Unconstrained Videos
Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chenliang Xu
European Conference on Computer Vision (ECCV), 2018
(A 4-page version was presented at CVPR'19 Sight and Sound Workshop.)
[ paper | arXiv | project]
Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Li Ding and Chenliang Xu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018
[ paper | arXiv ]
Towards Automatic Learning of Procedures from Web Instructional Videos
Luowei Zhou, Chenliang Xu, and Jason J. Corso
AAAI Conference on Artificial Intelligence (AAAI), 2018
Oral Presentation
[ paper | arXiv | YouCookII ]
Improving Text-based Person Search by Spatial Matching and Adaptive Threshold
Tianlang Chen, Chenliang Xu, and Jiebo Luo
IEEE Winter Conference on Applications of Computer Vision (WACV), 2018
[ paper ]
MRI Tumor Segmentation with Densely Connected 3D CNN
Lele Chen, Yue Wu, Adora M. DSouza, Anas Z. Abidin, Axel Wismüller, and Chenliang Xu
SPIE Conference on Medical Imaging (SPIE), 2018
Oral Presentation
[ paper | arXiv | code | talk ]
Generating Talking Face Landmarks from Speech
Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, and Zhiyao Duan
International Conference on Latent Variable Analysis and Signal Separation (LVA-ICA), 2018
[ paper | arXiv ]
Encoding and Selecting Coarse-Grain Mapping Operators with Hierarchical Graphs
Maghesree Chakraborty, Chenliang Xu, and Andrew D. White
Journal of Chemical Physics (JCP), 2018
[ paper | arXiv ]

2017

Audio-Visual Source Association for String Ensembles through Multi-Modal Vibrato Analysis
Bochen Li, Chenliang Xu, and Zhiyao Duan
Sound and Music Computing (SMC), 2017
Best Paper Award
[ paper ]
Deep Cross-Modal Audio-Visual Generation
Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, and Chenliang Xu
ACM International Conference on Multimedia Thematic Workshops (ACMMMW), 2017
[ paper ]
Watch What You Just Said: Image Captioning with Text-Conditional Attention
Luowei Zhou, Chenliang Xu, Parker Koch, and Jason J. Corso
ACM International Conference on Multimedia Thematic Workshops (ACMMMW), 2017
[ paper ]
Weakly Supervised Actor-Action Segmentation via Robust Multi-Task Ranking
Yan Yan, Chenliang Xu, Dawen Cai, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017
[ paper ]
One-Shot Video Object Segmentation with Iterative Online Fine-Tuning
Amos Newswanger and Chenliang Xu
CVPR 2019 Workshop on The DAVIS Challenge on Video Object Segmentation, 2017
[ paper | poster ]
Dancelets Mining for Video Recommendation Based on Dance Styles
Tingting Han, Hongxun Yao, Chenliang Xu, Xiaoshuai Sun, Yanhao Zhang and Jason J. Corso
IEEE Transactions on Multimedia (TMM), 2016
[ paper ]

2016 and Before

Scale-Adaptive Video Understanding
Chenliang Xu
PhD Thesis, University of Michigan, 2016
[ PDF ]
Actor-Action Semantic Segmentation with Grouping Process Models
Chenliang Xu and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016
Spotlight Presentation (9.7%)
[ paper | video | poster ]
LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing
Chenliang Xu and Jason J. Corso
International Journal of Computer Vision (IJCV), 2016
[ paper | arXiv | project ]
Can Humans Fly? Action Understanding with Multiple Classes of Actors
Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015
[ paper | video | poster | data | code ]
A Study of Actor and Action Semantic Retention in Video Supervoxel Segmentation
Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, and Jason J. Corso
International Journal of Semantic Computing (IJSC), 2013
[ paper | arXiv ]
Flattening Supervoxel Hierarchies by the Uniform Entropy Slice
Chenliang Xu, Spencer Whitt, and Jason J. Corso
IEEE/CVF International Conference on Computer Vision (ICCV), 2013
[ paper | video | poster | project ]
Are Actor and Action Semantics Retained in Video Supervoxel Segmentation?
Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, and Jason J. Corso
IEEE International Conference on Semantic Computing (ICSC), 2013
[ paper | slides ]
A Thousand Frames in Just a Few Words: Lingual Description of Videos through Latent Topics and Sparse Object Stitching
Pradipto Das, Chenliang Xu, Richard F. Doell, and Jason J. Corso
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2013
[ paper | poster | project ]
TRECVID 2013 GENIE: Multimedia Event Detection and Recounting
S. Oh, A. Perera, I. Kim, M. Pandey, K. Cannons, H. Hajimirsadeghi, A. Vahdat, G. Mori, B. Miller, S. McCloskey, Y.-C. Cheng, Z. Huang, C.-H. Lee, C. Xu, R. Kumar, W. Chen, J. Corso, L. Fei-Fei, D. Koller, V. Ramanathan, K. Tang, A. Joulin, A. Alahi
NIST TRECVID Workshop, 2013
[ paper ]
Streaming Hierarchical Video Segmentation
Chenliang Xu, Caiming Xiong, and Jason J. Corso
European Conference on Computer Vision (ECCV), 2012
Oral Presentation (2.8%)
[ paper | oral video | spotlight video | project ]
Evaluation of Super-Voxel Methods for Early Video Processing
Chenliang Xu and Jason J. Corso
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012
[ paper | poster | project ]
TRECVID 2012 GENIE: multimedia event detection and recounting
A. Perera, S. Oh, M. Pandey, T. Ma, A. Hoogs, A. Vahdat, K. Cannons, G. Mori, S. McCloskey, B. Miller, S. Venkatesh, P. Davalos, P. Das, C. Xu, J. Corso, R. Srihari, I. Kim, Y.-C. Cheng, Z. Huang, C.-H. Lee, K. Tang, L. Fei-Fei, D. Koller
NIST TRECVID Workshop, 2012
[ paper ]