Skip to footer

Deep Rigid Instance Scene Flow


In this paper we tackle the problem of scene flow estimation in the context of self-driving. We leverage deep learning techniques as well as strong priors as in our application domain the motion of the scene can be composed by the motion of the robot and the 3D motion of the actors in the scene. We formulate the problem as energy minimization in a deep structured model, which can be solved efficiently in the GPU by unrolling a Gaussian-Newton solver. Our experiments in the challenging KITTI scene flow dataset show that we outperform the state-of-the-art by a very large margin, while being 800 times faster.


Wei-Chiu Ma, Shenlong Wang, Rui Hu, Yuwen Xiong, Raquel Urtasun


CVPR 2019

Full Paper

‘Deep Rigid Instance Scene Flow’ (PDF)

Uber ATG

Previous article Dimensionality Reduction for Representing the Knowledge of Probabilistic Models
Next article Multi-Task Multi-Sensor Fusion for 3D Object Detection
Wei-Chiu Ma is a PhD student at MIT advised by Prof. Antonio Torralba. His research interests lie in the intersection of computer vision and machine learning, in particular low-level vision and 3D vision. He also works part-time at Uber ATG Toronto with Prof. Raquel Urtasun to apply his research to self-driving vehicles.
Shenlong Wang is research scientist at Uber ATG Toronto working on the development of self-driving cars. He is also a PhD student at University of Toronto. His advisor is Prof. Raquel Urtasun. He has a broad interest in computer vision, machine learning and robotics. He is particularly interested in 3D vision and deep structured models.
Rui Hu is a research engineer at Uber ATG Toronto. Before that he was a staff engineer at Qualcomm and senior engineer at Apple SPG. His research interests include neural networks, computer vision and GPU optimization.
Yuwen Xiong is a graduate student in Machine Learning Group at the University of Toronto, and a Research Scientist at Uber ATG Toronto, both supervised by Prof. Raquel Urtasun. Before that he received his bachelor degree in Computer Science from Zhejiang University in June 2018. His research interests include Computer Vision and Machine Learning, especially Deep Learning.
Raquel Urtasun is the Chief Scientist for Uber ATG and the Head of Uber ATG Toronto. She is also a Professor at the University of Toronto, a Canada Research Chair in Machine Learning and Computer Vision and a co-founder of the Vector Institute for AI. She is a recipient of an NSERC EWR Steacie Award, an NVIDIA Pioneers of AI Award, a Ministry of Education and Innovation Early Researcher Award, three Google Faculty Research Awards, an Amazon Faculty Research Award, a Connaught New Researcher Award, a Fallona Family Research Award and two Best Paper Runner up Prize awarded CVPR in 2013 and 2017. She was also named Chatelaine 2018 Woman of the year, and 2018 Toronto’s top influencers by Adweek magazine