---
type: "post"
id: "the-fastest-delta-weight-syncs-in-the-world-9f8e72"
slug: "the-fastest-delta-weight-syncs-in-the-world-9f8e72"
title: "The fastest delta weight syncs in the world"
description: "Baseten says it has redesigned delta weight synchronization so a roughly 770 GB GLM-5.3 update reaches serving replicas in 15 seconds on average, with a 0.8-second request pause."
retrieval_nugget: "Baseten says it has redesigned delta weight synchronization so a roughly 770 GB GLM-5.3 update reaches serving replicas in 15 seconds on average, with a 0.8-second request pause. For teams running RL rollouts or frequently refreshing large models, the relevant change is not a new model: it is a smaller interruption window and more host memory available for KV cache."
published_at: "2026-10-31"
updated_at: "2026-10-31"
record_date: "2026-10-31"
date_kind: "published_at"
topics: ["harness-engineering","agent-runtime"]
entities: ["Baseten","GLM-5.3","Kimi K3","GB300","SGLang","AgentX","Baseten Delivery Network"]
related_patterns: ["harness-architecture-outlives-model-choice"]
novelty_label: "new_fact"
what_changed: "Builds on Baseten’s earlier delta-weight-sync work: the same 770GB sync now takes 15 seconds with a 0.8-second pause, and the redesigned path frees host memory for KV cache."
prior_refs: ["memory:55fef327-562e-8d5a-a540-0ebccb607a62"]
source_url: "https://www.baseten.co/blog/the-fastest-delta-weight-syncs-in-the-world"
source_urls: ["https://www.baseten.co/blog/the-fastest-delta-weight-syncs-in-the-world"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:the-fastest-delta-weight-syncs-in-the-world-9f8e72"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/the-fastest-delta-weight-syncs-in-the-world-9f8e72/","markdown":"https://newruntime.com/posts/the-fastest-delta-weight-syncs-in-the-world-9f8e72.md","json":"https://newruntime.com/posts/the-fastest-delta-weight-syncs-in-the-world-9f8e72.json"}
---

# The fastest delta weight syncs in the world

## Retrieval answer

Baseten says it has redesigned delta weight synchronization so a roughly 770 GB GLM-5.3 update reaches serving replicas in 15 seconds on average, with a 0.8-second request pause. For teams running RL rollouts or frequently refreshing large models, the relevant change is not a new model: it is a smaller interruption window and more host memory available for KV cache.

## Summary

Baseten says it has redesigned delta weight synchronization so a roughly 770 GB GLM-5.3 update reaches serving replicas in 15 seconds on average, with a 0.8-second request pause. For teams running RL rollouts or frequently refreshing large models, the relevant change is not a new model: it is a smaller interruption window and more host memory available for KV cache.

## What happened

In an October 30, 2026 technical post, [Baseten](https://www.baseten.co/blog/the-fastest-delta-weight-syncs-in-the-world) describes a new path for applying sparse weight updates to models already serving requests. Its previous implementation synchronized a roughly 770 GB GLM-5.3 checkpoint in about 36 seconds, including a six-second request pause. The company says the revised design completes the same synchronization in 15 seconds with a 0.8-second pause, which it characterizes as 7.5 times faster than the previous state of the art. It also reports throughput gains of up to 3.2 times in a specific higher-concurrency experiment.

The system starts with a trainer publishing full checkpoints and sparse deltas to object storage. A delta is represented as zstd-compressed parent bytes XOR new bytes. Rollout clusters pull the objects through Baseten Delivery Network, reconstruct the policy, and update GPUs in place; the trainer and rollout GPUs do not share infrastructure. Previously, every serving replica also retained a complete copy of the parent checkpoint in CPU memory, because that copy was used to decode a future delta and then had to be promoted after each update.

The redesign treats weights already resident in GPU memory as the parent state. At startup, Baseten verifies served tensors against a local disk copy, records per-tensor Adler-32 checksums, then deletes that copy. When a delta arrives, a native decoder walks zstd commands and retains only changed positions and XOR masks rather than creating a dense XOR tensor. While the old policy continues to serve, each worker gathers old values only at changed positions, calculates and verifies the new values, and prepares writes in the engine's native tensor layout. The short pause closes admission, pauses running work, writes the prepared values, refreshes derived weights, publishes the version, and reopens admission.

The post also describes an incremental checksum technique: because each changed byte's Adler-32 contribution can be corrected from its position and values, unchanged bytes need not be read. Baseten says this reduced one Kimi K3, 30 GB-delta update from about 230 to about 100 seconds, while rejecting corrupted downloads or an incorrect parent before GPU writes. It further separates update planning from SGLang's Python scheduler, pipelines download and decoding, and partitions delta work by GPU ownership.

For Kimi K3, which Baseten says occupies 1.7 TB in its served MXFP4 layout, the company reports operation across two GB300 nodes with four GPUs each. An 8.4 GB delta becomes live in 75 seconds, with a 4.1-second pause. Freed host memory can instead hold offloaded KV cache. In its AgentX test at concurrency 48, Baseten reports cache hits rising from 2% to 92% and output throughput per GPU rising 3.2 times; it says both configurations showed no difference at concurrency 16 because the cache fit on GPU.

## What the archive adds

We have not covered this before; it opens a new thread.

## Trend signal

This supports [Harness architecture outlives model choice](https://newruntime.com/patterns/harness-architecture-outlives-model-choice/). The reported gains come from checkpoint representation, integrity checks, memory allocation, scheduling isolation, and update orchestration around a served model—not from changing the model itself. The measurements remain vendor-reported, but the direction is clear: operating a large model depends heavily on the serving harness.

## Why it matters

Frequent policy updates are awkward when serving replicas must reserve a second full checkpoint in host RAM and pause traffic long enough to rebuild state. Baseten's design suggests a different operational trade: prepare and validate changed GPU locations while the old policy remains live, then make the actual version switch a bounded write operation. For rollout-heavy systems, that can make tighter train-to-serve loops more plausible.

The memory consequence may be equally important. A full CPU-side parent copy competes directly with host-resident KV cache. Baseten's reported AgentX result is therefore not simply a synchronization benchmark: it links update-path design to repeated-prefix performance under load. Teams should treat those results as a description of this implementation and workload, not a general throughput guarantee.

## What happens next

- **Serving platforms will expose more explicit mechanisms for low-pause weight or policy refreshes.** (6 months, 60%). Watch for: documented APIs with measured update and pause budgets.
- **More rollout stacks will measure host-memory allocation as a joint trade-off between checkpoint state and KV cache.** (12 months, 65%). Watch for: benchmark reports separating cache-hit effects from model execution speed.

## Limits and uncertainty

The evidence is a vendor technical post, and all performance results are Baseten's own measurements. The 15-second figure applies to a roughly 770 GB GLM-5.3 checkpoint under its stated setup; the Kimi K3 result uses a different delta size, model layout, and two-node configuration. AgentX results are reported for one replica and trainer-shaped fixture deltas, timed from an S3 marker to the first response using new weights. The post does not establish comparable performance for other inference engines, networking conditions, update distributions, hardware, or customer traffic patterns. It also does not provide independently reproducible benchmark artifacts in the supplied material.

## Skill to build now

Instrument model-refresh operations as a service event, not merely a file-transfer event. This week, record end-to-end update time, traffic admission pause, verification failures, host-memory use, and cache-hit rate at several concurrency levels. Then separate work that can happen before a cutover from the minimal operations that must happen inside it. That framing makes it possible to see whether a checkpoint copy is protecting correctness, masking a decoder limitation, or consuming capacity that would be more valuable as cache.

## Reader takeaway

The useful claim is not that every large-model deployment needs Baseten's implementation. It is that rapid policy refresh depends on treating GPU-resident weights, integrity verification, memory budgets, and serving continuity as one systems problem. If updates are part of your runtime loop, measure the pause and the displaced cache capacity—not just download bandwidth.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Harness Engineering](https://newruntime.com/topics/harness-engineering.md): Explore the harness-engineering topic hub.
- topic hub: [Agent Runtime](https://newruntime.com/topics/agent-runtime.md): Explore the agent-runtime topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)