SELECTED PUBLICATIONS

Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting

investigate how to utilize visual content for disambiguation and latent space alignment in unsupervised MMT

Po-Yao Huang, Junjie Hu, Xiaojun Chang and Alexander Hauptmann

ACL 2020

Vision-language Navigation with Self-Supervised Auxiliary Reasoning Tasks

Introducing Auxiliary Reasoning Navigation (AuxRN), a framework with four self-supervised auxiliary reasoning tasks to take advantage of the additional training signals derived from the semantic information

Fengda Zhu, Yi Zhu, Xiaojun Chang, Xiaodan Liang

paper | DEMO | CODE

ZSTAD - Zero-Shot Temporal Activity Detection

Proposing a novel problem setting for temporal activity detection in which activities that are not seen during the training stage can be recognized and localized simultaneously

Lingling Zhang, Xiaojun Chang, Jun Liu, Sen Wang, Zongyuan Ge, Minnan Luo, Alexander Hauptmann

Unity Style Transfer for Person Re-Identification

smooth the style disparities within the same camera and across different cameras

Chong Liu, Xiaojun Chang, Yi-Dong Shen

Neural Architecture Search by Block-wisely Distilling Architecture Knowledge

modularize the large search space of NAS into blocks to ensure that the potential candidate architectures are fully trained

Changlin Li, Jiefeng Peng, Liuchun Yuan, Guangrun Wang, Xiaodan Liang, Liang Lin, Xiaojun Chang

Vision Dialogue Navigation by Exploring Cross-modal Memory

learning an agent endowed with the capability of constant conversation for help with natural language and navigating according to human responses
propose a Cross-modal Memory Network (CMN) for remembering and understanding the rich information relevant to historical navigation actions

Yi Zhu, Fengda Zhu, Zhao huan Zhan, Bingqian Lin, Jianbin Jiao, Xiaojun Chang, Xiaodan Liang

Overcoming Multi-Model Forgetting in One-Shot NAS with Diversity Maximization

formulate the supernet training in the One-Shot NAS as a constrained optimization problem of continual learning that the learning of current architecture should not degrade the performance of previous architectures during the supernet training

Miao Zhang, Huiqi Li, Shirui Pan, Xiaojun Chang, Steven Su

Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations

leveraging visual object detection and propose a model with diverse multi-head attention to learn grounded multilingual multimodal representations

Po-Yao Huang, Xiaojun Chang, Alexander G. Hauptmann

EMNLP 2019

Annotation Efficient Cross-Modal Retrieval with Adversarial Attentive Alignment

propose a novel framework to leverage automatically extracted regional semantics from un-annotated images as additional weak supervision to learn visual-semantic embeddings

Po-Yao Huang, Guoliang Kang, Wenhe Liu, Xiaojun Chang, Alexander G. Hauptmann

ACM MM 2019

RCAA Relational Context-Aware Agents for Person Search

made the earliest attempt to address the person search problem and built the first deep reinforcement learning based person search framework

Xiaojun Chang, Po-Yao Huang, Yi-Dong Shen, Xiaodan Liang, Yi Yang and Alexander G. Hauptmann

ECCV 2018

Reinforcement Cutting-Agent Learning for Video Object Segmentation

make a pioneer effort to formulate the video object segmentation problem as a Markov Decision Process and propose a novel reinforcement cutting-agent learning framework to tackle this problem

Junwei Han, Le Yang, Dingwen Zhang, Xiaojun Chang, Xiaodan Liang

CVPR 2018

Complex Event Detection by Identifying Reliable Shots from Untrimmed Videos

simultaneously learns a linear SVM classifier and infers a binary indicator for each instance in order to select reliable training instances from each positive or negative bag

Hehe Fan, Xiaojun Chang, De Cheng, Yi Yang, Dong Xu, Alexander G. Hauptmann

ICCV 2017

They are Not Equally Reliable - Semantic Event Search Using Differentiated Concept Classifiers

combine the concept classifiers based on a principled estimate of their accuracy on the unlabeled test videos

Xiaojun Chang, Yao-Liang Yu, Yi Yang, Eric P. Xing

CVPR 2016

Complex Event Detection using Semantic Saliency and Nearly-Isotonic SVM

define a novel notion of semantic saliency that assesses the relevance of each shot with the event of interest

Xiaojun Chang, Yi Yang, Eric P. Xing, Yao-Liang Yu

ICML 2015

Searching Persuasively - Joint Event Detection and Evidence Recounting with Limited Supervision

propose a joint framework that simultaneously detects high-level events and localizes the indicative concepts of the events

Xiaojun Chang, Yao-Liang Yu, Yi Yang, Alexander G. Hauptmann

ACM MM 2015