Pyro: Deep Universal Probabilistic Programming

    Abstract

    Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a modern GPU-accelerated deep learning framework. To accommodate complex or model-specific algorithmic behavior, Pyro leverages Poutine, a library of composable building blocks for modifying the behavior of probabilistic programs.

    Authors

    Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman

    Conference

    JMLR

    Full Paper

    ‘Pyro: Deep Universal Probabilistic Programming’ (PDF)Uber AI

    Comments
    Previous articleDynamic Pricing and Matching in Ride-Hailing Platforms
    Next articleProbabilistic Meta-Representations Of Neural Networks
    Eli Bingham
    Eli is a research scientist working on probabilistic programming, approximate Bayesian inference, and grounded language understanding. He has previously worked on condensed matter physics, computational biology, climatology, multiscale dictionary learning, and deep learning for computer vision. In his spare time he hangs out in a lab tinkering with his nanopore DNA sequencer.
    Martin Jankowiak
    Martin is a former particle physicist whose interest in data and modeling goes back to the Large Hadron Collider. After physics stops at Stanford and Heidelberg, he became a Research Scientist at the Center for Urban Science and Progress at NYU with a focus on applied machine learning research. Martin then joined a small machine learning start-up (Geometric Intelligence) with the happy end result that he joined AI Labs in March 2017.
    Fritz Obermeyer
    Fritz is a research engineer at Uber AI focusing on probabilistic programming. He is the engineering lead for the Pyro team.
    Neeraj Pradhan
    Neeraj is a software engineer on Pyro, a deep probabilistic programming language built over PyTorch. He was earlier an engineer on uberPool, working on algorithms and infrastructure for pooling riders. He is interested in synthesizing techniques from machine learning research to build intelligent systems that scale to real world problems. Prior to Uber, Neeraj worked on search, personalized recommendations, and learning to rank at Groupon. He did his masters in Operations Research from Stanford University, and undergraduate in Physics and EE from BITS Pilani.
    Theofanis Karaletsos
    Theofanis took his first steps as a machine learner at the Max Planck Institute For Intelligent Systems in collaboration with Microsoft Research Cambridge with work focused on unsupervised knowledge extraction from unstructured data, such as generative modeling of images and phenotyping for biology. He then moved to Memorial Sloan Kettering Cancer Center in New York, where he worked on machine learning in the context of cancer therapeutics. He joined a small AI startup Geometric Intelligence in 2016 and with his colleagues formed the new Uber AI Labs. Theofanis' research interests are focused on rich probabilistic modeling, approximate inference and probabilistic programming. His main passion are structured models, examples of which are spatio-temporal processes, models of image formation, deep probabilistic models and the tools needed to make them work on real data. His past in the life sciences has also made him keenly interested in how to make models interpretable and quantify their uncertainty, non-traditional learning settings such as weakly supervised learning and model criticism.
    Rohit Singh
    Rohit is an AI researcher with a PhD from MIT Computer Science and Artificial Intelligence Lab (CSAIL). He is currently working on applications of various AI techniques with the Pyro programming language across product teams at Uber. His previous work has involved applications of Machine Learning, Quantitative Game Theory and Program Synthesis in multiple domains from the fields of Compilers and Databases. Rohit has worked as an intern at Google where he used the Google Brain deep-learning framework for an application with the YouTube team and as a PM intern at Yelp where he worked on a Machine Learning application on Ad CTR prediction.
    Paul Szerlip
    Paul Szerlip earned his PhD with Dr. Kenneth Stanley at the University of Central Florida focusing on open-source infrastructure for collaborative evolutionary software. This open-source platform enables researchers to quickly integrate crowd-sourced human contributions with automated algorithms, while making the results easily accessible online. His later research highlighted new ways to integrate neuroevolutionary techniques like HyperNEAT and Novelty Search into deep learning frameworks.
    Noah Goodman
    In addition to working at Uber AI Labs, Noah is also an Associate Professor of Psychology, Computer Science, and Linguistics at Stanford University, where he runs the Computation and Cognition Lab.