Introduction
In 2017, we introduced Horovod, an open source framework for scaling deep learning training across hundreds of GPUs in parallel. At the time, most of the deep learning use cases at Uber were related to the research and …
Xu Ning
Xu Ning is an engineering manager in Uber’s Seattle Engineering office, currently leading multiple development teams in Uber’s Michelangelo Machine Learning Platform. He previously led Uber's Cherami distributed task queue, Hadoop observability, and Data security teams.
Engineering Blog Articles
First Uber Science Symposium: Discussing the Next Generation of RL, NLP, ConvAI, and DL
At Uber, hundreds of data scientists, economists, AI researchers and engineers, product analysts, behavioral scientists, and other practitioners leverage scientific methods to solve challenges on our platform. From modeling and experimentation to data analysis, algorithm development, and fundamental research, the …
JVM Profiler: An Open Source Tool for Tracing Distributed JVM Applications at Scale
Computing frameworks like Apache Spark have been widely adopted to build large-scale data applications. For Uber, data is at the heart of strategic decision-making and product development. To help us better leverage this data, we manage massive deployments of Spark …
Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go
Cherami is a distributed, scalable, durable, and highly available message queue system we developed at Uber Engineering to transport asynchronous tasks. We named our task queue after a heroic carrier pigeon with the hope that this system would be just …
