---
type: "post"
id: "training-a-text-to-image-model-for-21k-d1268a"
slug: "training-a-text-to-image-model-for-21k-d1268a"
title: "Training a Text-to-Image Model for $21K"
description: "Linum says it trained an experimental 2.18-billion-parameter Pyramid-JiT image model on 660 million samples for $21,000 in compute."
retrieval_nugget: "Linum says it trained an experimental 2.18-billion-parameter Pyramid-JiT image model on 660 million samples for $21,000 in compute. The two-person lab has published the model code and weights under Apache 2.0, presenting the run as a test of an architecture intended to make larger image and video models more tractable."
published_at: "2026-11-04"
updated_at: "2026-11-06"
record_date: "2026-11-04"
date_kind: "published_at"
topics: ["compute"]
entities: ["Linum","Pyramid-JiT","Qwen3.5-4B","DINOv3-L","Apache License 2.0"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Linum reports an experimental Apache 2.0 release of its 2B Pyramid-JiT text-to-image model, trained on 660M samples for $21K."
prior_refs: ["https://newruntime.com/posts/pruna-ai-open-sources-p-h3-image-an-76af40/"]
source_url: "https://www.linum.ai/field-notes/pjit-21k"
source_urls: ["https://www.linum.ai/field-notes/pjit-21k"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:training-a-text-to-image-model-for-21k-d1268a"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/training-a-text-to-image-model-for-21k-d1268a/","markdown":"https://newruntime.com/posts/training-a-text-to-image-model-for-21k-d1268a.md","json":"https://newruntime.com/posts/training-a-text-to-image-model-for-21k-d1268a.json"}
markdown_url: "https://newruntime.com/posts/training-a-text-to-image-model-for-21k-d1268a.md"
---

# Training a Text-to-Image Model for $21K

## Retrieval answer

Linum says it trained an experimental 2.18-billion-parameter Pyramid-JiT image model on 660 million samples for $21,000 in compute. The two-person lab has published the model code and weights under Apache 2.0, presenting the run as a test of an architecture intended to make larger image and video models more tractable.

## Summary
Linum says it trained an experimental 2.18-billion-parameter Pyramid-JiT image model on 660 million samples for $21,000 in compute. The two-person lab has published the model code and weights under Apache 2.0, presenting the run as a test of an architecture intended to make larger image and video models more tractable.

## What happened
In a November 4, 2026 field note, Linum co-CEOs Sahil Chopra and Manu Chopra describe a longer training run of Pyramid-JiT (P-JiT), a pixel-space text-to-image diffusion model. Linum calls the work an experimental release and a stepping stone toward larger video models. The reported run covers 660 million samples and cost $21,000 in compute. The page makes model code and weights available under Apache 2.0 and displays 249 generated samples. The available material does not specify a hosted service or a commercial price for using the model.

The architectural premise is to remove the separate variational autoencoder (VAE) commonly used to compress images before diffusion. Linum argues that VAE compression has an empirical ceiling and that improving reconstruction in the VAE can sometimes hurt the downstream diffusion model. P-JiT instead operates in pixel space and aims to learn a representation suited to the diffusion task itself. The team says its earlier P-JiT work represented a 512-by-512 image with the same token count that Linum v2 used for a 256-by-256 image: 32-by-32 rather than 16-by-16 token reduction. In that earlier comparison, the architecture converged with 11 times fewer samples and reduced the attention context window by a factor of four.

The model has 2.18 billion parameters at sampling time; training adds about 0.24 billion parameters in auxiliary heads, for roughly 2.42 billion. Its text representation comes from a frozen Qwen3.5-4B model, reading layers 7, 15 and 27. Text and image each pass through two single-modality refiner blocks before joining a 22-block trunk. The image is divided into 32-by-32 patches and compressed to a 1,024-dimensional representation. Intermediate readout heads predict lower-resolution clean images, while another training component, PixelREPA, aligns internal representations with DINOv3-L features. The auxiliary readouts and alignment head are dropped at sampling time.

Linum describes this run as a pressure test after earlier experiments trained on roughly 50–100 million samples. The motivation is that image models are a useful step toward video: the team says models must learn people, places and things before actions and movement. Its stated longer-term goal is to reduce the cost and improve the controllability of generative video tools. The supplied article excerpt does not give a complete, independently verifiable training recipe, full hardware inventory, benchmark protocol, or detailed access instructions beyond the code and weights release.

## What the archive adds
This is a new training result within a recurring image-model thread, not a restatement of the earlier releases. [Pruna AI open-sources P-H3-Image](https://newruntime.com/posts/pruna-ai-open-sources-p-h3-image-an-76af40/) covered an 8-step image generation and editing model; it is related by the practical question of what open image-generation systems can do, but it is not the same architecture or result. [How to train your own Jev for $17](https://newruntime.com/posts/how-to-train-your-own-jev-for-17-9a96f0/) covered low-cost classifier fine-tuning, a different task and training regime.

## Trend signal
The supplied patterns do not directly describe low-cost image-model training or pixel-space diffusion. The closest is [generative media becomes application infrastructure](https://newruntime.com/patterns/generative-media-becomes-application-infrastructure/), but this source is about an experimental model and its training, not evidence that applications are composing generation as controllable infrastructure. No pattern claim is therefore treated as materially supported by this release.

## Why it matters
A $21,000 training bill for a multi-billion-parameter image model is a useful data point, but not yet a portable unit cost. Teams considering whether to train a specialized generative model can inspect an open architecture and compare its stated compression strategy with the conventional VAE-plus-diffusion setup. The important operational question is whether lower attention cost and fewer training samples translate into acceptable output quality under a repeatable recipe.

The release also illustrates a distinction between model size and training expense: the headline figure concerns compute for this run, not data creation, research labor, evaluation, deployment, or the cost of serving generated images. For builders exploring video, Linum's result may justify watching the architecture, but it does not establish that its approach solves video inference speed, controllability, or storytelling needs. Reproduction and independent evaluation matter more than the headline cost alone.

## Limits and uncertainty
The claims about fourfold token reduction and elevenfold sample efficiency are Linum's comparisons with its own earlier Linum v2 setup, not evidence of superiority over all current image models. The supplied material does not establish output quality through an independent benchmark, nor does it show that the $21,000 figure includes anything beyond compute for this training run. It does not provide enough information here to verify the exact hardware, dataset composition, full duration, or repeatability of the result. The release is explicitly experimental and aimed at future video work; no claim of a production-ready video model follows from it.

## Reader takeaway
Linum has published an open experimental image model trained for a reported $21,000 in compute, along with an architecture designed to reduce attention context by working in pixel space. Treat the cost and efficiency figures as a promising lab result, not a turnkey budget estimate. The useful next step for a team is to inspect the code and weights and look for independent reproduction or comparable evaluations.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Compute](https://newruntime.com/topics/compute.md): Explore the compute topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)