Skip to main content
Uber logo

Schedule rides in advance

Reserve a rideReserve a ride

Schedule rides in advance

Reserve a rideReserve a ride
AI, Data / ML

Manifold: A Model-Agnostic Visual Debugging Tool for Machine Learning at Uber

January 14, 2019 / Global
Featured image for Manifold: A Model-Agnostic Visual Debugging Tool for Machine Learning at Uber
Figure 1. The Manifold interface consists of a performance comparison view (left) and a feature attribution view (right).
Figure 2. Visual encodings of Manifold’s Performance Comparison view is composed of an x-axis (user-selected performance metric, e.g., log-loss or squared-log-error, or raw prediction), a y-axis (data segments), and colors (models). Curve height shows performance distribution of each model on each data segment.
Figure 3. The visual encodings of the Feature Attribution view incorporate an x-axis (the feature value range), a y-axis (the number of data points), and colors (data segment groups). Features are ranked by their distribution difference with regard to the two data segment groups.
Figure 3a
Figure 3b
Figure 3c
Figure 4. This visualization prototype displays model performance in data space, with each data point positioned based on its performance (x-axis) and value of one of its features (y-axis).
Figure 5. All data points are collected from each model and are given a performance metric after being evaluation against ground truth.
Figure 7. The Manifold architecture is composed of three distinct sections: data source, backend, and frontend.
Figure 8. Manifold can compare the performance of two models (with or without new features) on four data subsets.
Figure 9. Manifold compares the performance of all positive instances in a dataset to determine if there are any false negatives.
Figure 10. In this scenario, Manifold determined that the model’s false negatives tended to have low values for Features A, B, C, D, and E.
Figure 11. Manifold identified that false negatives generated from this model tended to have low values for Features A, B, C, D, or E.
Lezhi Li

Lezhi Li

Lezhi Li is a software engineer on Uber's Machine Learning Platform team.

Posted by Lezhi Li, Tim

Category: