Performing updates of individual records in Uber's over 100 petabyte Apache Hadoop data lake required building Global Index, a component that manages data bookkeeping and lookups at scale.
Noticing increased latency in our metrics platform, Uber engineers track down a bug related to stack growth in a goroutine, resulting in a fix elevated to the Go open source GitHub repository.
How engineers and data scientists at Uber came together to come up with a means of partially replicating Vertica clusters to better scale our data volume.
Uber developed Michelangelo PyML to run identical copies of machine learning models locally in both real time experiments and large-scale offline prediction jobs.
Facing the need for a resilient data structure over thousands of storage nodes to serve the 15 million rides per day that occur on our platform, Uber engineers developed Herb, our data replication solution. Herb ensures data availability and integrity across our data centers.
deck.gl v5 incorporates simplified APIs, scripting support, and framework agnosticism, making the popular open source data visualization software more accessible than ever before.
Uber Engineering extended our anomaly detection platform's ability to integrate new forecast models, allowing this critical on-call service to scale to meet more complex use cases.
Gleaning Insights from Uber’s Partner Activity Matrix with Genomic Biclustering and Machine Learning
Uber Engineering's partner activity matrix leverages biclustering and machine learning to better understand the diversity of user experiences on our driver app.
In this article, we highlight how Uber leverages machine learning and artificial intelligence to tackle engineering challenges at scale.
Uber Engineering's data science workbench (DSW) is an all-in-one toolbox that leverages aggregate data for interactive analytics and machine learning.
Uber Engineering introduces Horovod, an open source framework that makes it faster and easier to train deep learning models with TensorFlow.
Uber Engineering built AthenaX, our open source streaming analytics platform, to bring large-scale event stream processing to everyone.
The UberEATS Restaurant Manager gives restaurant partners insight into their business by measuring customer satisfaction, sales, and service quality.
Uber Engineering introduces Michelangelo, our machine learning-as-a-service system that enables teams to easily build, deploy, and operate ML solutions at scale.
Uber Engineering’s Data Visualization team uses their deck.gl and Voyager visualization platforms to map rider behavior during the August 21, 2017 solar eclipse.
Uber Engineering's Data Visualization Team and ATG built a new web-based platform that helps engineers and operators better understand information collected during testing of its self-driving vehicles.
Recurrent neural networks equip Uber Engineering's new forecasting model to more accurately predict rider demand during extreme events.
In this article, we discuss how Uber Engineering uses Locality Sensitive Hashing on Apache Spark to reliably detect fraudulent trips at scale.
Composed of a staged rollout and intelligent analytics tool, Uber Engineering's experimentation platform is capable of stably deploying new features at scale across our apps. In this article, we discuss the challenges and opportunities we faced when building this product.
How Uber Engineering re-architected the content delivery feed and backend ecosystem of our new driver app to deliver an enhanced user experience.
A daylong event at Uber’s Palo Alto office, sponsored by our LadyEng group, showcased the technical work across Uber Engineering as well as the people who are leading and building these projects. Here are some of the resulting presentations.
Uber Engineering's data processing platform team recently built and open sourced Hudi, an incremental processing framework that supports our business critical data pipelines. In this article, we see how Hudi powers a rich data ecosystem where external sources can be ingested into Hadoop in near real-time.
Uber Engineering explains the technical reasoning behind its switch in database technologies, from Postgres to MySQL.
The end of a two-part series on the tech stack that Uber Engineering uses to make transportation as reliable as running water, everywhere, for everyone, as of spring 2016.
Uber’s mission is transportation as reliable as running water, everywhere, for everyone. Here's the first of a two-part series on the tech stack that Uber Engineering uses to make this happen.
The data visualization team in Uber Engineering delivers intelligence through crafting visual exploratory data analysis tools. Here's what some of these visualizations look like.
Here we look at Hadoop data ingestion, and how Uber Engineering streams diverse data into a cohesive layer for querying in near real-time using our in-house developed Streamific.
Uber’s mission is transportation as reliable as running water, for everyone, everywhere. This past month, Uber Engineering talked about what it takes to get site reliability engineering right.
Imagine you have to store data whose massive influx increases by the hour. Your first priority, after making sure you can easily add storage capacity, is to try and reduce the data’s footprint to save space. But how? This is the story of Uber Engineering’s comprehensive encoding protocol and compression algorithm test and how this discipline saved space in our Schemaless datastores.
The details and examples of Schemaless triggers, a key feature of the datastore that’s kept Uber Engineering scaling since October 2014. This is the third installment of a three-part series on Schemaless; the first part is a design overview and the second part is a discussion of architecture.
How Uber’s infrastructure works with Schemaless, the datastore using MySQL that’s kept Uber Engineering scaling since October 2014. This is part two of a three-part series on Schemaless; part one is on designing Schemaless.
The making of Schemaless, Uber Engineering’s custom designed datastore using MySQL, which has allowed us to scale from 2014 to beyond. This is part one of a three-part series on Schemaless.