Skip to footer
Home Research Artificial Intelligence / Machine Learning Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction

Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction


Self-driving vehicles plan around both static and dynamic objects, applying predictive models of behavior to estimate future locations of the objects in the environment. However, future behavior is inherently uncertain, and models of motion that produce deterministic outputs are limited to short timescales. Particularly difficult is the prediction of human behavior. In this work, we propose the discrete residual flow network (DRF-Net), a convolutional neural network for human motion prediction that captures the uncertainty inherent in long-range motion forecasting. In particular, our learned network effectively captures multimodal posteriors over future human motion by predicting and updating a discretized distribution over spatial locations. We compare our model against several strong competitors and show that our model outperforms all baselines.


Ajay Jain*, Sergio Casas*, Renjie Liao*, Yuwen Xiong*, Song Feng, Sean Segal, Raquel Urtasun


NeurIPS 2019

Full Paper

‘Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction’ (PDF)

Uber ATG

Previous article Incremental Few-Shot Learning with Attention Attractor Networks
Next article Identifying Unknown Instances for Autonomous Driving
Sergio is a Masters' student at the University of Toronto with a variety of research interests including Deep Learning for Computer Vision, Reinforcement Learning and Imitation Learning. His supervisor is Prof. Raquel Urtasun. At the same time, he is working full-time at Uber ATG Toronto to apply his research to the development of Self-Driving Cars technology, focusing on Perception and Prediction problems. Before coming to UofT, he completed two Bachelor degrees at Universitat Politecnica de Catalunya (UPC), in Computer Science and Industrial Engineering. He is originally from Barcelona, Spain.
Yuwen Xiong is a graduate student in Machine Learning Group at the University of Toronto, and a Research Scientist at Uber ATG Toronto, both supervised by Prof. Raquel Urtasun. Before that he received his bachelor degree in Computer Science from Zhejiang University in June 2018. His research interests include Computer Vision and Machine Learning, especially Deep Learning.
Song Feng is a research engineer at Uber ATG. She works on several projects on perception and predictions. Before joining ATG, she works on search and recommnedation systems in UberEats using machine learning and infra work related.
Sean is a research scientist at Uber ATG Toronto. He is also a Master’s student at the University of Toronto supervised by Professor Raquel Urtasun. His research interests include deep learning, computer vision and scenario recognition. Before joining the University of Toronto, Sean studied Computer Science & Economics at Brown University.
Raquel Urtasun is the Chief Scientist for Uber ATG and the Head of Uber ATG Toronto. She is also a Professor at the University of Toronto, a Canada Research Chair in Machine Learning and Computer Vision and a co-founder of the Vector Institute for AI. She is a recipient of an NSERC EWR Steacie Award, an NVIDIA Pioneers of AI Award, a Ministry of Education and Innovation Early Researcher Award, three Google Faculty Research Awards, an Amazon Faculty Research Award, a Connaught New Researcher Award, a Fallona Family Research Award and two Best Paper Runner up Prize awarded CVPR in 2013 and 2017. She was also named Chatelaine 2018 Woman of the year, and 2018 Toronto’s top influencers by Adweek magazine