Scaling our data infrastructure with lower hardware costs while maintaining high performance and service reliability has been no easy feat. To accommodate the exponential growth in both Data Storage and Analytics Compute at Uber, the Data Infrastructure team massively overhauled …
Jeffrey Zhong
Jeffrey Zhong is a former Engineering manager on Uber’s Core Data team, and managed data lake storage (Hadoop HDFS & Apache Hudi), batch compute scheduling (Hadoop YARN), and Data’s hardware and performance. Before Uber, he worked on Apache HBase and Apache Phoenix.
Engineering Blog Articles
Containerizing Apache Hadoop Infrastructure at Uber
Introduction
As Uber’s business grew, we scaled our Apache Hadoop (referred to as ‘Hadoop’ in this article) deployment to 21000+ hosts in 5 years, to support the various analytical and machine learning use cases. We built a team with varied …