Field noteSep 23, 20261 min read1 source

Taming the ML Firehose: Scaling Feature Consistency

In one minute

Uber’s design makes the online inference record, rather than a reconstructed offline approximation, the basis for the next training set.

The mechanism is selective capture: allow-listed features are serialized with compact integer aliases, then prediction records are joined to client impression events in Flink. Uber reports peak traffic of 8 million QPS, estimates that full logging could reach about 1.7 PB per day, and says its allow list reduces payload size by 4–5×. The immediate implication is that feature consistency becomes an operational data-contract problem: retain the serving values and the outcome together, while avoiding the cost of retaining every scored candidate.

Why now: transformer-backed recommendation traffic makes a canonical record desirable but economically infeasible without aggressive filtering and stream-state discipline. The evidence boundary is one company’s engineering description; it reports architecture and internal estimates, not an independently audited model-lift result. Watch whether Uber publishes measured changes in model quality, training freshness, or incident rates after deployment. This starts a distinct analytical record because the supplied material documents a specific inference-to-training logging architecture and its scaling trade-offs, while no prior New Runtime record is supplied for comparison.

Evidence / sources

  1. 1uber.com/us/en/blog/taming-ml-firehose
All Field Notes

Next reads

Read next

Open the graph
  1. 01archiveField NotesOpen the latest editorial analysis.
  2. 02source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract