End-to-end Interpretable Neural Motion Planner


    In this paper, we propose a neural motion planner for learning to drive autonomously in complex urban scenarios that include traffic-light handling, yielding, and interactions with multiple road-users. Towards this goal, we design a holistic model that takes as input raw LIDAR data and an HD map and produces interpretable intermediate representations in the form of 3D detections and their future trajectories, as well as a cost volume defining the goodness of each position that the self-driving car can take within the planning horizon. We then sample a set of diverse physically possible trajectories and choose the one with the minimum learned cost. Importantly, our cost volume is able to naturally capture multi-modality and uncertainty. We demonstrate the effectiveness of our approach in real-world driving data captured in several cities in North America.


    Wenyuan Zeng, Wenjie Luo, Simon Suo, Abbas Sadat, Bin Yang, Sergio Casas, Raquel Urtasun


    CVPR 2019

    Full Paper

    ‘End-to-end Interpretable Neural Motion Planner’ (PDF)

    Uber ATG

    Previous articleLearning to Localize through Compressed Binary Maps
    Next articleUnderstanding and Designing for Deaf or Hard of Hearing Drivers on Uber
    Wenyuan Zeng is currently a PhD student at the University of Toronto, supervised by Prof. Raquel Urtasun. His research interest mainly lies in deep learning, computer vision and decision making process. At the same time, he is also working full-time at Uber ATG Toronto to apply his research work to the development of self-driving cars, focusing on perception, prediction and planning. Before coming to University of Toronto, Wenyuan Zeng finished his Bachelor degree in Tsinghua University, China, majored in Mathematics and Physics.
    Wenjie Luo
    Wenjie is a senior research scientist, founding member of the Uber ATG R&D team. His research interests include computer vision and machine learning, and his work spans the full autonomy stack including perception, prediction and planning. Previously, he did master in TTI-Chicago and continued to the PhD program in University of Toronto, both under Prof. Raquel Urtasun. He also spent some time at Apple SPG prior to join Uber.
    Simon is a research scientist at Uber ATG Toronto and a graduate student at Univeristy of Toronto, supervised by Prof. Raquel Urtasun. His research interest mainly lies in machine learning and robotics. At ATG, he aims to use understanding of interactive scenarios to improve planning and simulation. Before joining University of Toronto, Simon studied Computer Science at Universtiy of Waterloo.
    Abbas Sadat is a senior research scientist at Uber ATG Toronto working on self-driving vehicles. His research interest lies in Robotics, focusing on safe decision-making and motion planning, leveraging machine-learning. Before joining ATG, he worked as autonomous driving research engineer at Bosch Research. He obtained his computing science PhD from SFU in 2016. His PhD research was on planning and decision making for small-size UAVs.
    Bin Yang
    Bin Yang is a research scientist at Uber ATG Toronto. He's also a PhD student at University of Toronto, supervised by Prof. Raquel Urtasun. His research interest lies in computer vision and deep learning, with a focus on 3D perception in autonomous driving scenario.
    Sergio Casas
    Sergio is a Masters' student at the University of Toronto with a variety of research interests including Deep Learning for Computer Vision, Reinforcement Learning and Imitation Learning. His supervisor is Prof. Raquel Urtasun. At the same time, he is working full-time at Uber ATG Toronto to apply his research to the development of Self-Driving Cars technology, focusing on Perception and Prediction problems. Before coming to UofT, he completed two Bachelor degrees at Universitat Politecnica de Catalunya (UPC), in Computer Science and Industrial Engineering. He is originally from Barcelona, Spain.
    Raquel Urtasun
    Raquel Urtasun is the Chief Scientist for Uber ATG and the Head of Uber ATG Toronto. She is also a Professor at the University of Toronto, a Canada Research Chair in Machine Learning and Computer Vision and a co-founder of the Vector Institute for AI. She is a recipient of an NSERC EWR Steacie Award, an NVIDIA Pioneers of AI Award, a Ministry of Education and Innovation Early Researcher Award, three Google Faculty Research Awards, an Amazon Faculty Research Award, a Connaught New Researcher Award, a Fallona Family Research Award and two Best Paper Runner up Prize awarded CVPR in 2013 and 2017. She was also named Chatelaine 2018 Woman of the year, and 2018 Toronto’s top influencers by Adweek magazine