---
type: "post"
id: "speeding-up-language-models-for-travel-applications-877a80"
slug: "speeding-up-language-models-for-travel-applications-877a80"
title: "Speeding up language models for travel applications"
description: "Amazon Science and Booking.com report a deployment pipeline that combines knowledge distillation with Medusa-2 decoding heads for travel-language-model workloads."
retrieval_nugget: "Amazon Science and Booking.com report a deployment pipeline that combines knowledge distillation with Medusa-2 decoding heads for travel-language-model workloads. Their smallest reported model, a 1.1-billion-parameter TinyLlama student, matched a 70-billion-parameter Llama-3.1 teacher in the described evaluation while reducing latency and cost in specific product tasks."
published_at: "2026-10-30"
updated_at: "2026-10-30"
record_date: "2026-10-30"
date_kind: "published_at"
topics: ["models"]
entities: ["Amazon Science","Booking.com","Medusa-2","TinyLlama","Llama-3.1","GPT-3.5","ACL"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Amazon’s travel-focused language-model research is a distinct research topic; no prior describes its acceleration methods or results."
prior_refs: []
source_url: "https://www.amazon.science/blog/speeding-up-language-models-for-travel-applications"
source_urls: ["https://www.amazon.science/blog/speeding-up-language-models-for-travel-applications"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:speeding-up-language-models-for-travel-applications-877a80"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/speeding-up-language-models-for-travel-applications-877a80/","markdown":"https://newruntime.com/posts/speeding-up-language-models-for-travel-applications-877a80.md","json":"https://newruntime.com/posts/speeding-up-language-models-for-travel-applications-877a80.json"}
---

# Speeding up language models for travel applications

## Retrieval answer

Amazon Science and Booking.com report a deployment pipeline that combines knowledge distillation with Medusa-2 decoding heads for travel-language-model workloads. Their smallest reported model, a 1.1-billion-parameter TinyLlama student, matched a 70-billion-parameter Llama-3.1 teacher in the described evaluation while reducing latency and cost in specific product tasks.

## Summary

Amazon Science and Booking.com report a deployment pipeline that combines knowledge distillation with Medusa-2 decoding heads for travel-language-model workloads. Their smallest reported model, a 1.1-billion-parameter TinyLlama student, matched a 70-billion-parameter Llama-3.1 teacher in the described evaluation while reducing latency and cost in specific product tasks.

## What happened

Amazon Science published a report on October 29, 2026, describing work with Booking.com on speeding up language models for travel applications. The problem is operational rather than purely academic: travel services answer high volumes of questions, recommend hotels, and help users plan trips, while large language models can be expensive to run and slow enough to damage a web or mobile experience. The article notes that even a 100-millisecond delay can matter for customer conversion and revenue, but does not provide a separate measurement for that statement.

The work combines two techniques. Knowledge distillation transfers behavior from a larger teacher model to a smaller student. In the process described here, the teacher is fine-tuned on labeled data and then generates predictions for additional unlabeled, domain-relevant text. Those machine-generated examples are combined with the labeled data to train the student. The stated goal is a smaller model that retains much of the teacher's task performance while being cheaper and faster to deploy.

The second component is Medusa, here identified as Medusa-2. Ordinary autoregressive language models emit one token at a time. Medusa attaches prediction modules, or heads, to a base model so it can predict multiple tokens in parallel; the proposed outputs are then handled through the framework's decoding process. Amazon says GPU hardware can process this parallel decoding with little extra overhead and characterizes typical Medusa gains as two- to fourfold without compromising generation quality. In this pipeline, the heads are attached during student training rather than added as an entirely separate post-training step.

The teams applied the system to three Booking.com use cases. AI Trip Planner is a conversational assistant that extracts travel preferences from dialogue in less than one second. Smart Filters turns natural-language search queries into structured filters in real time. Accommodation Description Generation creates property descriptions across millions of listings, where cost effectiveness is the stated priority. The reported final model was TinyLlama with 1.1 billion parameters; the teacher was Llama-3.1 with 70 billion parameters. Amazon says the combined model matched the teacher's quality while running 10 to 20 times faster and at less than one-third the cost of the best open-source models cited in the article.

The most detailed latency result concerns Smart Filters: the combined approach delivered responses 15 times faster for 99% of queries. Amazon says distillation alone closed more than 90% of the student-teacher quality gap and made inference roughly six times faster; Medusa heads then supplied another 2-to-3.6-times speedup. For the other two use cases, the article reports a 2.9% increase in AI Trip Planner user engagement versus GPT-3.5 and 1.4% more “helpful” ratings for generated accommodation descriptions, at lower cost than other models. The related work was presented at ACL 2025 and received that conference's Industry Track best-paper award, according to the source.

## What the archive adds

This is a new production case for a known efficiency technique. Our earlier note on [knowledge distillation](https://newruntime.com/posts/knowledge-distillation-lets-smaller-language-models-inherit-capabilities-d1d19a/) covered the broader problem of capability transfer and the train-test gap in standard distillation. The Amazon report adds a domain-specific deployment design, Medusa heads trained with the student, and product metrics from travel workflows.

## Trend signal

No supplied pattern genuinely fits this evidence. The source measures speed, cost, engagement, and helpfulness in bounded travel tasks, but it does not establish a broader shift in agent architecture, interoperability, security, or completed-work economics.

## Why it matters

For teams serving a narrow, repeated workflow, the relevant comparison is not always a general-purpose frontier model against another general-purpose model. It can be a teacher-student system against the latency budget and quality threshold of one product action. This case suggests a sequence: define the target output precisely, collect labeled and domain-relevant unlabeled examples, test whether a compact student retains the required behavior, and then optimize decoding.

That sequence matters because compression and serving improvements solve different constraints. Distillation is intended to reduce the model needed for the task; Medusa is intended to reduce the serial decoding bottleneck at runtime. Combining them can improve the user-facing response time without requiring a team to choose between a capable teacher and a deployable student at the outset. The reported gains are most relevant where request volume, responsiveness, and output format are all measurable.

## Limits and uncertainty

This is an Amazon Science account of work conducted with Booking.com, not an independently replicated evaluation. The source does not provide the full datasets, quality rubric, traffic mix, hardware configuration, absolute latency values, or cost figures needed to reproduce the claims. “Matched quality” is not accompanied in the supplied material by a complete metric definition or confidence intervals. The results also concern three travel-specific tasks and do not establish that TinyLlama, Medusa, or this exact training recipe will perform similarly for open-ended support, coding, multilingual work, or safety-sensitive decisions. The engagement and helpfulness results should not be read as a universal model ranking.

## Skill to build now

Choose one high-volume language-model step with an externally checkable output, such as query-to-filter extraction, routing, classification, or structured form completion. Build a held-out evaluation set from that task before attempting model compression. Measure task accuracy, p95 latency, and cost per successful output separately. If a smaller fine-tuned student reaches the required quality, test a decoding optimization afterward; do not infer that faster token generation alone preserves the product outcome.

## Reader takeaway

The important result is a deployment pattern, not a claim that small models broadly replace large ones. In a constrained task with a clear quality measure, a large teacher can create training signal for a compact student, and parallel decoding can further reduce latency. Validate both stages against the product metric that users actually experience.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)