---
type: "post"
id: "freight-search-composing-a-shared-search-index-from-17718c"
slug: "freight-search-composing-a-shared-search-index-from-17718c"
title: "Freight Search: Composing a Shared Search Index from Independently Owned Fragments"
description: "Uber Freight has described a shared search platform that composes OpenSearch documents from fragments owned by independent microservices."
retrieval_nugget: "Uber Freight has described a shared search platform that composes OpenSearch documents from fragments owned by independent microservices. Its relevance to AI builders is practical: retrieval quality and freshness depend less on a single embedding step than on explicit ownership, lineage, ingestion, and validation across the systems that hold operational truth."
published_at: "2026-10-28"
updated_at: "2026-10-28"
record_date: "2026-10-28"
date_kind: "published_at"
topics: ["retrieval","infrastructure"]
entities: ["Uber","Uber Freight","Freight Search","OpenSearch","Apache Spark","Apache Flink","Apache Kafka"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Uber describes a declarative fragment-composition framework for building and maintaining a shared Freight Search index across independently contributing teams."
prior_refs: []
source_url: "https://www.uber.com/us/en/blog/high-performance-search-platform"
source_urls: ["https://www.uber.com/us/en/blog/high-performance-search-platform"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:freight-search-composing-a-shared-search-index-from-17718c"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/freight-search-composing-a-shared-search-index-from-17718c/","markdown":"https://newruntime.com/posts/freight-search-composing-a-shared-search-index-from-17718c.md","json":"https://newruntime.com/posts/freight-search-composing-a-shared-search-index-from-17718c.json"}
---

# Freight Search: Composing a Shared Search Index from Independently Owned Fragments

## Retrieval answer

Uber Freight has described a shared search platform that composes OpenSearch documents from fragments owned by independent microservices. Its relevance to AI builders is practical: retrieval quality and freshness depend less on a single embedding step than on explicit ownership, lineage, ingestion, and validation across the systems that hold operational truth.

## Summary
Uber Freight has described a shared search platform that composes OpenSearch documents from fragments owned by independent microservices. Its relevance to AI builders is practical: retrieval quality and freshness depend less on a single embedding step than on explicit ownership, lineage, ingestion, and validation across the systems that hold operational truth.

## What happened
On October 27, 2026, Uber published an engineering account of Freight Search, the business-layer search platform used across Uber Freight. The company frames the problem as unusually difficult because Freight runs more than 40 microservices and search draws on almost 100 data sources and tables, including transactional stores, tracking systems, and external APIs. A query such as finding a shipper's loads that will pick up in Chicago within 24 hours can therefore require fields from several services with different shapes and service-level expectations.

Freight Search sits above Uber's central Search Platform, which hosts OpenSearch clusters. Individual Freight services do not connect directly to the search engine. Instead, developers define two main abstractions. An index is a searchable entity mapped to an OpenSearch index; Uber uses a load, a shipment from one location to another, as an example. A fragment is the portion of that index supplied by one source. The Load service can own a load identifier and creation time, the Facility service can own locations, and the Financial service can own cargo value.

The platform merges those fragments into one denormalized OpenSearch document, so a query can span fields from multiple services without joins at query time. Uber says every field in an index entity comes from exactly one fragment. That makes ownership explicit and permits partial updates: when a change-data-capture event changes a fragment, the system can write only the fields that fragment owns rather than rebuild the full document. Some fragment values fan out; for example, a facility location may need to be copied to every load that references it.

There are two ingestion stages. Bootstrap reads full source snapshots through connectors behind a common interface and runs as an Apache Spark job. Connectors may scan a live database in partitions, read an offline Hive table, or call a service over RPC. Live ingestion runs as an Apache Flink job, consumes CDC events from an offset aligned to the bootstrap timestamp, converts source rows to fragments, denormalizes them where necessary, and writes affected documents within seconds. The same connector model supports continuous validation against source rows.

Uber says developers define index and source behavior through a Java annotation DSL; a generator derives OpenSearch mappings from the annotated classes. The stated current scale is more than 70 source topics, 300 fragments, and over 20 indices. The service handles thousands of complex queries per second across Freight products, with p99 query latency in the hundreds of milliseconds and data freshness of a few seconds.

## What the archive adds
This is a new event, but it belongs to the broader enterprise-search context also represented by [Zoom's acquisition of Onyx](https://newruntime.com/posts/zoom-acquires-onyx-open-source-enterprise-search-that-c840e0/). That earlier note concerned an enterprise-search product used to supply context to AI agents; Uber's account is instead an internal architecture for constructing and operating the underlying shared index.

## Trend signal
No listed pattern cleanly fits. The source is an account of search-platform composition and data operations, not evidence that a particular agent, protocol, or AI-native organizational pattern has changed.

## Why it matters
Teams building AI retrieval systems often encounter the same underlying problem before they encounter a model problem: the relevant facts live in independently operated systems. Freight Search offers a useful boundary. Business teams own small, declared fragments; the platform owns connectors, ingestion mechanics, index construction, and validation. That division avoids giving every service a bespoke search pipeline while preserving a trace from an indexed field to its system of record.

For an AI application, this does not automatically solve authorization, ranking, or answer correctness. It does show a way to make a retrieval corpus operationally credible: define the source and owner of every field, distinguish initial backfill from live updates, and validate the index continuously. Those are prerequisites for asking an agent to rely on retrieved operational data.

## What happens next
- **Uber Freight will continue to add sources or searchable fields through fragment definitions rather than bespoke indexing services** (12 months, 70%). Watch for: follow-on engineering material describing new connectors, validation coverage, or additional business indices.

## Limits and uncertainty
The account is Uber's own engineering description and does not provide independent benchmarks, cost figures, failure rates, or a comparison with alternative designs. “Hundreds of milliseconds” p99 latency and “a few seconds” freshness are aggregate claims without workload definitions, index sizes, read/write mixes, or validation-error results. The article also does not establish how access controls are applied at query time, how schema changes and deletions are handled, or whether this design would transfer directly to a smaller organization without Uber's central platform and internal connector ecosystem.

## Skill to build now
Make a small ownership map for one retrieval-backed workflow this week. For every field an agent or search experience needs, record its system of record, the team responsible for it, its update signal, its expected freshness, and how it can be checked against the source. Then identify one document currently rebuilt in full after a minor change and test whether a field-scoped update is possible. The exercise exposes whether an index is a trustworthy projection or an undocumented copy.

## Reader takeaway
The reusable idea in Freight Search is not OpenSearch itself. It is treating a shared index as a composed, continuously validated projection of independently owned data: one declared owner per field, common ingestion infrastructure, and updates scoped to the changed fragment. That is a sturdier base for retrieval and AI applications than a one-off aggregation pipeline.

## Open question
Can the fragment model preserve equally clear authorization and deletion semantics when the same denormalized document contains data from many systems of record?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Retrieval](https://newruntime.com/topics/retrieval.md): Explore the retrieval topic hub.
- topic hub: [Infrastructure](https://newruntime.com/topics/infrastructure.md): Explore the infrastructure topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)