---
type: "post"
id: "don-t-write-kernels-derive-them-aee769"
slug: "don-t-write-kernels-derive-them-aee769"
title: "Don't Write Kernels, Derive Them"
description: "Municipal Robotics presents Tang, a research compiler that derives GPU kernels from a compact tensor intermediate representation instead of requiring a hand-written kernel for each operator, fusion, and chip."
retrieval_nugget: "Municipal Robotics presents Tang, a research compiler that derives GPU kernels from a compact tensor intermediate representation instead of requiring a hand-written kernel for each operator, fusion, and chip. Its results show both the promise and the boundary of that approach: some workloads approach or beat hand-tuned systems, while others remain materially slower or fail an accuracy gate."
published_at: "2026-11-12"
updated_at: "2026-11-12"
record_date: "2026-11-12"
date_kind: "published_at"
topics: ["compute","ai-infrastructure"]
entities: ["Municipal Robotics","Tang","Qwen3-8B","PyTorch","FlashAttention-2","llama.cpp","vLLM"]
related_patterns: ["harness-architecture-outlives-model-choice"]
novelty_label: "new_fact"
what_changed: "Tang describes a tensor-IR compiler that derives kernels across several GPU and graphics backends, with measured transformer and Qwen3-8B performance and explicit cases where derived attention is slower."
prior_refs: ["https://newruntime.com/posts/forgekernel-self-evolving-kernels-catching-and-surpassing-experts-30654b/","https://newruntime.com/posts/how-we-built-tlx-a-hardware-native-gpu-31371f/"]
source_url: "https://muni.works/research/tang"
source_urls: ["https://muni.works/research/tang"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:don-t-write-kernels-derive-them-aee769"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/don-t-write-kernels-derive-them-aee769/","markdown":"https://newruntime.com/posts/don-t-write-kernels-derive-them-aee769.md","json":"https://newruntime.com/posts/don-t-write-kernels-derive-them-aee769.json"}
markdown_url: "https://newruntime.com/posts/don-t-write-kernels-derive-them-aee769.md"
---

# Don't Write Kernels, Derive Them

## Retrieval answer

Municipal Robotics presents Tang, a research compiler that derives GPU kernels from a compact tensor intermediate representation instead of requiring a hand-written kernel for each operator, fusion, and chip. Its results show both the promise and the boundary of that approach: some workloads approach or beat hand-tuned systems, while others remain materially slower or fail an accuracy gate.

## Summary
Municipal Robotics presents Tang, a research compiler that derives GPU kernels from a compact tensor intermediate representation instead of requiring a hand-written kernel for each operator, fusion, and chip. Its results show both the promise and the boundary of that approach: some workloads approach or beat hand-tuned systems, while others remain materially slower or fail an accuracy gate.

## What happened
Municipal Robotics describes Tang as a `no_std` Rust library with a tensor IR built from two constructs: affine views, which map iteration coordinates to tensor coordinates, and comprehensions, which express reductions under a monoid. Scalar expression graphs form the body of a comprehension. The authors say fusion, flash attention, logsumexp, Welford variance, top-k routing, paged KV-cache access, and other familiar operations emerge through rewrites and scheduling rather than operator-specific kernel implementations. Tang also derives gradients of comprehensions, including an attention backward path that avoids writing intermediates on the three measured GPU backends: CUDA, WGSL, and Metal.

The same graph can be lowered to WGSL, CUDA tensor cores, Metal simdgroup matrices, OpenCL, GLSL, and C. The article says the compiler has been measured on a datacenter GPU, a desktop GPU, a laptop, and a phone-class system. Schedule choices include tiling, fusion, and storage and accumulation precision. The authors say precision choices are checked against an error budget using an emulating interpreter before execution; the mathematical graph itself remains the same, though rounding can differ.

Results are workload-dependent. On an idle RTX 3090, a derived transformer block's forward pass at sequence length 512 is reported as 1.2 times faster than PyTorch captured in a CUDA graph with TF32; at length 128, PyTorch is 1.2 times faster, and forward plus backward ties. Tang decodes Qwen3-8B as one IR graph at 148.7 tokens per second, matching llama.cpp at short context and reaching 92% of its speed at 4,000 tokens. The paper also says derived attention trails FlashAttention-2 at every tested size, with a case on the RTX 3090 where no tensor-core forward is generated and performance is about 150 times slower.

The production-engine comparison is similarly qualified. The authors derive kernels for an 80-billion-parameter mixture-of-experts engine running on one H200. With equivalent work, the derived expert layer is 18% slower than hand kernels at batch 1, 7% slower at batch 4, tied at batch 8, and 5% faster at batch 64. In the full model, a derived decode step takes 5–12% less time at 16–48 concurrent streams, but misses the engine's 99% agreement gate at 48 streams, reporting 98.93%. For time to first token at 44,000 tokens, derived attention and gated-delta scan reduce the engine's result from 8.33 to 5.54 seconds; the article notes that vLLM on an FP8 checkpoint is about 12 times faster than that engine.

## What the archive adds
New Runtime's earlier coverage described AgentForge as a Rust research project that compiles agent programs together with a model and machine description ([31 October 2026](https://newruntime.com/posts/agentforge-research-preview-open-source-rust-compiler-and-40ab5b/)). It also covered ForgeKernel's reported kernel-generation results ([6 November 2026](https://newruntime.com/posts/forgekernel-self-evolving-kernels-catching-and-surpassing-experts-30654b/)) and Meta's hardware-oriented TLX compiler extension ([10 November 2026](https://newruntime.com/posts/how-we-built-tlx-a-hardware-native-gpu-31371f/)). Tang is a new research system in this thread, adding a particularly explicit algebraic IR, derived backward passes, and mixed performance results against optimized systems.

## Trend signal
Tang supports the view that compiler and harness design can be a more durable boundary than choosing a single model or kernel library: the same graph is scheduled across targets, while the compiler searches implementation choices. That fits [harness architecture outlives model choice](https://newruntime.com/patterns/harness-architecture-outlives-model-choice/), though Tang is a compiler research project rather than an agent harness. The reported regressions and accuracy gate also show that abstraction alone does not guarantee production performance.

## Why now
The paper frames its work around a growing mismatch between a small set of model operators and the thousands of kernels needed across backends, fusions, shapes, and devices. It argues that schedules vary with graph, chip, and size, and presents Tang as a response to that combinatorial maintenance problem.

## Why it matters
For teams shipping inference or training across hardware generations, a compact representation with derived schedules could reduce duplicated kernel engineering and make transformations easier to validate. Tang's result is not that hand-tuned kernels are obsolete: its own comparisons show cases where specialized systems are much faster, and its 80B engine misses a stated agreement threshold in one regime. The practical question is whether compiler-generated kernels can cover enough workload space at acceptable latency and numerical error to reduce the number of bespoke paths a team must own. The article's strongest contribution is making that trade-off measurable rather than presenting derivation as a universal speedup.

## Limits and uncertainty
These are results reported by Municipal Robotics, not an independent benchmark. The figures come from specific devices, shapes, precision settings, baselines, and measurement procedures, so they do not establish broad performance across models or hardware. The article itself reports missing or slower schedules, accuracy failures, and a production-engine agreement miss. Tang is described as research; the reported evidence does not establish operational maturity, maintenance burden, or how reliably its search performs on workloads beyond those measured.

## Reader takeaway
Tang makes a strong case for treating kernels as derivable schedules over a shared mathematical graph, but its own numbers show why teams still need workload-specific benchmarking and correctness gates. The opportunity is fewer hand-maintained implementations, not guaranteed wins over the best specialized kernels.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Compute](https://newruntime.com/topics/compute.md): Explore the compute topic hub.
- topic hub: [Ai Infrastructure](https://newruntime.com/topics/ai-infrastructure.md): Explore the ai-infrastructure topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)