Skip to footer

Matching Adversarial Networks


Generative Adversarial Nets (GANs) and Conditonal GANs (CGANs) show that using a trained network as loss function (discriminator) enables to synthesize highly structured outputs (e.g. natural images). However, applying a discriminator network as a universal loss function for common supervised tasks (e.g. semantic segmentation, line detection, depth estimation) is considerably less successful. We argue that the main difficulty of applying CGANs to supervised tasks is that the generator training consists of optimizing a loss function that does not depend directly on the ground truth labels. To overcome this, we propose to replace the discriminator with a matching network taking into account both the ground truth outputs as well as the generated examples. As a consequence, the generator loss function also depends on the targets of the training examples, thus facilitating learning. We demonstrate on three computer vision tasks that this approach can significantly outperform CGANs achieving comparable or superior results to task-specific solutions and results in stable training. Importantly, this is a general approach that does not require the use of task-specific loss functions.


Gellert Mattyus, Raquel Urtasun


CVPR 2018

Full Paper

‘Matching Adversarial Networks’ (PDF)

Uber AI

Previous article Deep Parametric Continuous Convolutional Neural Networks
Next article GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
Gellert Mattyus
Gellert Mattyus is a research scientist at Uber ATG Toronto working on computer vision and machine learning problems related to self-driving with an emphasis on perceiving maps. Gellert Mattyus has earned his PhD at the Remote Sensing Technology Chair of the Technical University of Munich (TUM) while working as a research scientist at the Photogrammetry and Image Analysis Department of the German Aerospace Center (DLR). After earning his PhD, Gellert Mattyus has spent nearly a year as a post-doc at the Machine Learning Group of the University of Toronto under the supervision of Professor Raquel Urtasun.
Raquel Urtasun
Raquel Urtasun is the Chief Scientist for Uber ATG and the Head of Uber ATG Toronto. She is also a Professor at the University of Toronto, a Canada Research Chair in Machine Learning and Computer Vision and a co-founder of the Vector Institute for AI. She is a recipient of an NSERC EWR Steacie Award, an NVIDIA Pioneers of AI Award, a Ministry of Education and Innovation Early Researcher Award, three Google Faculty Research Awards, an Amazon Faculty Research Award, a Connaught New Researcher Award, a Fallona Family Research Award and two Best Paper Runner up Prize awarded CVPR in 2013 and 2017. She was also named Chatelaine 2018 Woman of the year, and 2018 Toronto’s top influencers by Adweek magazine