Multi-Task Multi-Sensor Fusion for 3D Object Detection

    Abstract

    In this paper we propose to exploit multiple related tasks for accurate multi-sensor 3D object detection. Towards this goal we present an end-to-end learnable architecture that reasons about 2D and 3D object detection as well as ground estimation and depth completion. Our experiments show that all these tasks are complementary and help the network learn better representations by fusing information at various levels. Importantly, our approach leads the KITTI benchmark on 2D, 3D and bird’s eye view object detection, while being real-time.

    Authors

    Ming Liang, Bin Yang, Yun Chen, Rui Hu, Raquel Urtasun

    Conference

    CVPR 2019

    Full Paper

    ‘Multi-Task Multi-Sensor Fusion for 3D Object Detection’ (PDF)

    Uber ATG

    Comments
    Previous articleDeep Rigid Instance Scene Flow
    Next articleConvolutional Recurrent Network for Road Boundary Extraction
    Avatar
    Ming Liang is a research scientist at Uber ATG Toronto. Before that he was a senior engineer at Apple SPG. His research interests include neural networks and computer vision.
    Avatar
    Bin Yang is a research scientist at Uber ATG Toronto. He's also a PhD student at University of Toronto, supervised by Prof. Raquel Urtasun. His research interest lies in computer vision and deep learning, with a focus on 3D perception in autonomous driving scenario.
    Avatar
    Yun Chen is a graduate student in Pattern Recognition and Intelligent System Laboratory (PRIS Lab) at Beijing University of Posts and Telecommunications, and an AI Resident at Uber ATG Toronto, supervised by Prof. Raquel Urtasun. His research interests include Computer Vision especially Deep Learning.
    Avatar
    Rui Hu is a research engineer at Uber ATG Toronto. Before that he was a staff engineer at Qualcomm and senior engineer at Apple SPG. His research interests include neural networks, computer vision and GPU optimization.
    Raquel Urtasun
    Raquel Urtasun is the Chief Scientist for Uber ATG and the Head of Uber ATG Toronto. She is also a Professor at the University of Toronto, a Canada Research Chair in Machine Learning and Computer Vision and a co-founder of the Vector Institute for AI. She is a recipient of an NSERC EWR Steacie Award, an NVIDIA Pioneers of AI Award, a Ministry of Education and Innovation Early Researcher Award, three Google Faculty Research Awards, an Amazon Faculty Research Award, a Connaught New Researcher Award, a Fallona Family Research Award and two Best Paper Runner up Prize awarded CVPR in 2013 and 2017. She was also named Chatelaine 2018 Woman of the year, and 2018 Toronto’s top influencers by Adweek magazine