The first change targeted allocation structure; the later hybrid design addressed how much translation data stayed in memory. This is an operational case study, not evidence that a disk-backed cache is the right default for every service.
The mechanism was to keep compact hash-based metadata and hot entries in memory while placing other translation values on disk, with sharding, eviction, and selective preloading used to manage contention and latency. The service kept compact hash-based metadata and hot entries in memory while placing other translation values on disk, using sharding, eviction, and selective preloading to manage contention and latency. Uber reports that replacing sync.Map with a map guarded by an RWMutex cut CPU usage by more than 30% in the service discussed. In its example, memory fell from about 2.7 GB to 750 MB, while the cache hit ratio remained above 99.9%.
Why now: memory and garbage collection remain production costs even when the application’s core data is simple key-value content. The implication is that cache design should be evaluated against both allocation pressure and the resident working set, not just lookup speed. The evidence boundary is Uber’s own workload and reported measurements; the post does not provide independent validation or a general benchmark across services. Watch whether the disk-backed design preserves latency and hit rate as translation usage and repository size grow.