Presto® on Apache Kafka® At Uber Scale
Uber’s goal is to ignite opportunity by setting the world in motion, and big data is a very important part of that. Presto® and...
Securing Kafka® Infrastructure at Uber
Background
Uber has one of the largest deployments of Apache Kafka® in the world. It empowers a large number of real-time workflows at Uber, including pub-sub...
Introducing Ballast: An Adaptive Load Test Framework
As Uber's architecture has grown to encompass thousands of interdependent microservices, we need to test our mission-critical components at max load in order to...
Project RADAR: Intelligent Early Fraud Detection System with Humans in the Loop
Introduction
Uber is a worldwide marketplace of services, processing thousands of monetary transactions every second. As a marketplace, Uber takes on all of the risks...
Capacity Recommendation Engine: Throughput and Utilization Based Predictive Scaling
Introduction
Capacity is a key component of reliability. Uber's services require enough resources in order to handle daily peak traffic and to support our different...
Cadence Multi-Tenant Task Processing
Introduction
Cadence is a multi-tenant orchestration framework that helps developers at Uber to write fault-tolerant, long-running applications, also known as workflows. It scales horizontally to...
How Uber Migrated Financial Data from DynamoDB to Docstore
Introduction
Each day, Uber moves millions of people around the world and delivers tens of millions of food and grocery orders. This generates a large...
Introducing uGroup: Uber’s Consumer Management Framework
Background
Apache Kafka® is widely used across Uber’s multiple business lines. Take the example of an Uber ride: When a user opens up the Uber app,...
Improving HDFS I/O Utilization for Efficiency
Scaling our data infrastructure with lower hardware costs while maintaining high performance and service reliability has been no easy feat. To accommodate the exponential...
Building Uber’s Fulfillment Platform for Planet-Scale using Google Cloud Spanner
Introduction
The Fulfillment Platform is a foundational Uber domain that enables the rapid scaling of new verticals. The platform handles billions of database transactions each...
Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot
Uber recently launched a new capability: Ads on UberEats. With this new ability came new challenges that needed to be solved at Uber, such...