The company says Habitat serves data in milliseconds across 40 geographic regions for more than 1 billion users and manages more than 500 PB of data.
The post describes work with Azure on the Azure Cosmos DB storage layer. OpenAI says the resulting architecture improved global median read latency by 12x. Habitat moved from an initial shared Cosmos DB container, through single-tenant backing containers, to a multi-tenant-by-default design using co-managed containers.
The company identifies hot logical partition keys and noisy neighbors as operational problems. Cosmos DB documents a 10,000 RU/s limit per logical partition key, but OpenAI says effective capacity can be lower because physical partitions enforce the limits shared by colocated keys. The post describes OpenAI's own system and does not establish that its design, latency result, or operating assumptions will transfer to other Cosmos DB deployments.