Raquel Urtasun
Raquel Urtasun is the Head of Uber ATG Toronto, as well as an Associate Professor in the Department of Computer Science at the University of Toronto and a co-founder of the Vector Institute for AI.
Engineering Blog Articles
Announcing the 2019 Uber AI Residency
The Uber AI Residency is a 12-month training program for academics and professionals interested in becoming an AI researcher with Uber AI Labs or Uber ATG.
Introducing the Uber AI Residency
Interested in accelerating your career by tackling some of Uber’s most challenging AI problems? Apply for the Uber AI Residency, a research fellowship dedicated to fostering the next generation of AI talent.
SBNet: Leveraging Activation Block Sparsity for Speeding up Convolutional Neural Networks
Uber ATG Toronto developed Sparse Blocks Network (SBNet), an open source algorithm for TensorFlow, to speed up inference of our 3D vehicle detection systems while lowering computational costs.
Research Papers
LanczosNet: Multi-Scale Deep Graph Convolutional Networks
Renjie Liao, Zhizhen Zhao, Raquel Urtasun, Richard S. Zemel
Relational data can generally be represented as graphs. For processing such graph structured data, we propose LanczosNet, which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. [...] [PDF]
Neural Information Processing Systems (NIPS), 2018
Relational data can generally be represented as graphs. For processing such graph structured data, we propose LanczosNet, which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. [...] [PDF]
Neural Information Processing Systems (NIPS), 2018
Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a...
W. Luo, B. Yang, R. Urtasun
In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor. By jointly reasoning about these tasks, our holistic approach is more robust to occlusion as well as sparse data at range. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor. By jointly reasoning about these tasks, our holistic approach is more robust to occlusion as well as sparse data at range. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
SurfConv: Bridging 3D and 2D Convolution for RGBD Images
H. Chu, W. Ma, K. Kundu, R. Urtasun, S. Fidler
The last few years have seen approaches trying to combine the increasing popularity of depth sensors and the success of the convolutional neural networks. Using depth as additional channel alongside the RGB input has the scale variance problem present in image convolution based approaches. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
The last few years have seen approaches trying to combine the increasing popularity of depth sensors and the success of the convolutional neural networks. Using depth as additional channel alongside the RGB input has the scale variance problem present in image convolution based approaches. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
IntentNet: Learning to Predict Intention from Raw Sensor Data
S. Casas, W. Luo, R. Urtasun
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high level behaviors as well as continuous trajectories describing future motion. In this paper we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. [...] [PDF]
Conference on Robot Learning (CORL), 2018
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high level behaviors as well as continuous trajectories describing future motion. In this paper we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. [...] [PDF]
Conference on Robot Learning (CORL), 2018
HDNET: Exploiting HD Maps for 3D Object Detection
B. Yang, M. Liang, R. Urtasun
In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. [...] [PDF]
Conference on Robot Learning (CORL), 2018
In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. [...] [PDF]
Conference on Robot Learning (CORL), 2018
Learning to Localize Using a LiDAR Intensity Map
I. Bârsan, S. Wang, A. Pokrovsky, R. Urtasun
In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars. Our method learns to embed the online LiDAR sweeps and intensity map into a joint deep embedding space. [...] [PDF]
Conference on Robot Learning (CORL), 2018
In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars. Our method learns to embed the online LiDAR sweeps and intensity map into a joint deep embedding space. [...] [PDF]
Conference on Robot Learning (CORL), 2018
IntentNet: Learning to Predict Intention from Raw Sensor Data
S. Casas, W. Luo, R. Urtasun
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high level behaviors as well as continuous trajectories describing future motion. [...] [PDF]
Conference on Robot Learning (CORL), 2018
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high level behaviors as well as continuous trajectories describing future motion. [...] [PDF]
Conference on Robot Learning (CORL), 2018
Deep Continuous Fusion for Multi-Sensor 3D Object Detection
M. Liang, B. Yang, S. Wang, R. Urtasun
In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization. Towards this goal, we design an end-to-end learnable architecture that exploits continuous convolutions to fuse image and LIDAR feature maps at different levels of resolution. [...] [PDF]
European Conference on Computer Vision (ECCV), 2018
In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization. Towards this goal, we design an end-to-end learnable architecture that exploits continuous convolutions to fuse image and LIDAR feature maps at different levels of resolution. [...] [PDF]
European Conference on Computer Vision (ECCV), 2018
End-to-End Deep Structured Models for Drawing Crosswalks
J. Liang, R. Urtasun
In this paper we address the problem of detecting crosswalks from LiDAR and camera imagery. Towards this goal, given multiple Li-DAR sweeps and the corresponding imagery, we project both inputs onto the ground surface to produce a top down view of the scene. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
In this paper we address the problem of detecting crosswalks from LiDAR and camera imagery. Towards this goal, given multiple Li-DAR sweeps and the corresponding imagery, we project both inputs onto the ground surface to produce a top down view of the scene. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Efficient Convolutions for Real-Time Semantic Segmentation of 3D Point Clouds
C. Zhang, W. Luo, R. Urtasun
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
International Conference on 3D Vision (3DV), 2018
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
International Conference on 3D Vision (3DV), 2018
Single Image Intrinsic Decomposition Without a Single Intrinsic Image
W. Ma, H, Chu, B. Zhou, R. Urtasun, A. Torralba
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
European Conference on Computer Vision (ECCV), 2018
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
European Conference on Computer Vision (ECCV), 2018
Neural Guided Constraint Logic Programming for Program Synthesis
Lisa Z., G. Rosenblatt, E. Fetaya, R. Liao, W. Byrd, M. Might, R. Urtasun, R. Zemel
Synthesizing programs using example input/outputs is a classic problem in artificial intelligence. We present a method for solving Programming By Example (PBE) problems by using a neural model to guide the search of a constraint logic programming system called miniKanren. [...] [PDF]
Advances in Neural Information Processing Systems (NIPS), 2018
Synthesizing programs using example input/outputs is a classic problem in artificial intelligence. We present a method for solving Programming By Example (PBE) problems by using a neural model to guide the search of a constraint logic programming system called miniKanren. [...] [PDF]
Advances in Neural Information Processing Systems (NIPS), 2018
End-to-end Learning of Multi-sensor 3D Tracking by Detection
D. Frossard, R. Urtasun
In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories. Towards this goal, we formulate the problem as a linear program that can be solved exactly, and learn convolutional networks for detection as well as matching in an end-to-end manner. [...] [PDF]
International Conference on Robotics and Automation (ICRA), 2018
In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories. Towards this goal, we formulate the problem as a linear program that can be solved exactly, and learn convolutional networks for detection as well as matching in an end-to-end manner. [...] [PDF]
International Conference on Robotics and Automation (ICRA), 2018
Matching Adversarial Networks
G. Mattyus, R. Urtasun
Generative Adversarial Nets (GANs) and Conditonal GANs (CGANs) show that using a trained network as loss function (discriminator) enables to synthesize highly structured outputs (e.g. natural images). However, applying a discriminator network as a universal loss function for common supervised tasks (e.g. semantic segmentation, line detection, depth estimation) is considerably less successful. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Generative Adversarial Nets (GANs) and Conditonal GANs (CGANs) show that using a trained network as loss function (discriminator) enables to synthesize highly structured outputs (e.g. natural images). However, applying a discriminator network as a universal loss function for common supervised tasks (e.g. semantic segmentation, line detection, depth estimation) is considerably less successful. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Deep Parametric Continuous Convolutional Neural Networks
S. Wang, S. Suo, W. Ma, A. Pokrovsky, R. Urtasun
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Hierarchical Recurrent Attention Networks for Structured Online Maps
N. Homayounfar, W. Ma, S. Lakshmikanth, R. Urtasun
In this paper, we tackle the problem of online road network extraction from sparse 3D point clouds. Our method is inspired by how an annotator builds a lane graph, by first identifying how many lanes there are and then drawing each one in turn. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
In this paper, we tackle the problem of online road network extraction from sparse 3D point clouds. Our method is inspired by how an annotator builds a lane graph, by first identifying how many lanes there are and then drawing each one in turn. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
X. Qi, R. Liao, Z. Liu, R. Urtasun, J. Jia
In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image. Building on top of two-stream CNNs, our GeoNet incorporates geometric relation between depth and surface normal via the new depth-to-normal and normal-to-depth networks. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image. Building on top of two-stream CNNs, our GeoNet incorporates geometric relation between depth and surface normal via the new depth-to-normal and normal-to-depth networks. [...] [PDF]
Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Sports Field Localization via Deep Structured Models
N. Homayounfar, S. Fidler, R. Urtasun
In this work, we propose a novel way of efficiently localizing a soccer field from a single broadcast image of the game. Related work in this area relies on manually annotating a few key frames and extending the localization to similar images, or installing fixed specialized cameras in the stadium from which the layout of the field can be obtained. [...] [PDF]
Reference & Citations: [LINK]
Conference on Computer Vision and Pattern Recognition (CVPR), 2017
In this work, we propose a novel way of efficiently localizing a soccer field from a single broadcast image of the game. Related work in this area relies on manually annotating a few key frames and extending the localization to similar images, or installing fixed specialized cameras in the stadium from which the layout of the field can be obtained. [...] [PDF]
Reference & Citations: [LINK]
Conference on Computer Vision and Pattern Recognition (CVPR), 2017
Learning to Reweight Examples for Robust Deep Learning
M. Ren, W. Zeng, B. Yang, R. Urtasun
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. [...] [PDF]
Conference on Computer Vision and Pattern ( ICML), 2018
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. [...] [PDF]
Conference on Computer Vision and Pattern ( ICML), 2018
Inference in Probabilistic Graphical Models by Graph Neural Networks
K. Yoon, R. Liao, Y. Xiong, L. Zhang, E. Fetaya, R. Urtasun, R. Zemel, X. Pitkow
A fundamental computation for statistical inference and accurate decision-making is to compute the marginal probabilities or most probable states of task-relevant variables. Probabilistic graphical models can efficiently represent the structure of such complex data, but performing these inferences is generally difficult. [...] [PDF]
International Conference on Learning Representations (ICLR), 2018
A fundamental computation for statistical inference and accurate decision-making is to compute the marginal probabilities or most probable states of task-relevant variables. Probabilistic graphical models can efficiently represent the structure of such complex data, but performing these inferences is generally difficult. [...] [PDF]
International Conference on Learning Representations (ICLR), 2018
