Thesis
The useful unit in this search cold start is a query paired with a behavior-grounded relevance distinction, not a plausible query alone.
Observed facts
Airbnb says CASTLE generates queries from contrastive listing pairs derived from de-identified booking sessions, selected according to privacy permissions.
1Airbnb reports KL divergence of 1.01 against real query distributions versus 9.33 for its best baseline.
1Airbnb reports 91–93% expert agreement across three raters reviewing 200 triplets.
1
Mechanism
The system conditions a language model on both listings and asks for a query satisfied by the positive but not the negative, so the relevance label follows the observed choice rather than a separate model judgment. For pairs without booking history, Airbnb adds an LLM Virtual Judge.
Why now
Airbnb says natural-language search had to be trained before real user queries existed, while synthetic-query methods risk mismatching actual guest phrasing and require labels.
Implications
Search teams facing a traffic cold start can treat existing choice behavior as structured supervision for both query generation and relevance, rather than relying only on generated text plus post-hoc judgments.
This may reduce circularity in synthetic labeling, while leaving gaps where behavior signals are unavailable.
Evidence boundary
These metrics and implementation details are Airbnb's own published account; the supplied source does not provide independent replication or enough detail to assess all deployment effects.
Watch conditions
Whether later evaluations show the same alignment and label quality on new markets, listing types, or independently reproduced datasets.