---
type: "post"
id: "unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e"
slug: "unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e"
title: "Unsloth released dynamic GGUF quants of Ornith-1.5-122B, a 122B MoE reasoning and coding model."
description: "Unsloth has published GGUF quantizations of Ornith-1.5-122B, a 122B-parameter mixture-of-experts reasoning model."
retrieval_nugget: "Unsloth has published GGUF quantizations of Ornith-1.5-122B, a 122B-parameter mixture-of-experts reasoning model."
published_at: "2026-11-17"
updated_at: "2026-11-17"
record_date: "2026-11-17"
date_kind: "published_at"
topics: ["models","local-models","open-models"]
entities: ["Unsloth AI","Ornith-1.5-122B","llama.cpp","Ollama","vLLM","SGLang"]
related_patterns: ["agent-ready-software-exposes-capabilities"]
novelty_label: "new_fact"
what_changed: "Unsloth has added dynamic GGUF quantizations of Ornith-1.5-122B for local inference."
prior_refs: ["https://newruntime.com/posts/ornith-released-ornith-1-5-122b-a-122b-037713/"]
source_url: "https://huggingface.co/unsloth/Ornith-1.5-122B-GGUF"
source_urls: ["https://huggingface.co/unsloth/Ornith-1.5-122B-GGUF"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e/","markdown":"https://newruntime.com/posts/unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e.md","json":"https://newruntime.com/posts/unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e.json"}
markdown_url: "https://newruntime.com/posts/unsloth-released-dynamic-gguf-quants-of-ornith-1-4c102e.md"
---

# Unsloth released dynamic GGUF quants of Ornith-1.5-122B, a 122B MoE reasoning and coding model.

## Retrieval answer

Unsloth has published GGUF quantizations of Ornith-1.5-122B, a 122B-parameter mixture-of-experts reasoning model.

## Summary

Unsloth has published GGUF quantizations of Ornith-1.5-122B, a 122B-parameter mixture-of-experts reasoning model. The files provide multiple quantization levels and instructions for local runtimes, giving teams another way to evaluate or serve the model without relying on a hosted API.

## What happened

The Hugging Face repository, maintained by Unsloth AI, publishes GGUF files for Ornith-1.5-122B. It includes BF16 and Q8_0 builds, Unsloth Dynamic quantizations from IQ1_M through Q2_K_XL, and K_XL variants from Q3 through Q6. Larger builds are split into multiple files; the repository also includes BF16 and F16 multimodal projector files. These are downloadable model artifacts, not a separate model release.

The repository gives run instructions for llama.cpp, Ollama, LM Studio, Jan, vLLM, Unsloth Desktop and other local applications. For llama.cpp, it shows a command using the UD-Q4_K_XL variant with `llama serve -hf`, which starts an OpenAI-compatible local server, and a corresponding CLI command for direct inference. It also gives an Ollama command to run that quantization. The model can therefore be evaluated through familiar local tooling, though the exact hardware and performance depend on the selected quantization and deployment setup.

The model card identifies Ornith-1.5-122B as a 122B MoE reasoning model and describes it as the lightest member of the Ornith-1.5 family. It says the model is designed for efficient single-GPU deployment; the serving instructions for the full model specify a single 80GB GPU. The card recommends recent serving runtimes, including Transformers 5.8.1 or later, vLLM 0.19.1 or later, and SGLang 0.5.9 or later. It provides vLLM and SGLang launch examples with a 262,144-token context window and tool-call and reasoning parsers enabled.

The card also documents using YaRN scaling to extend the effective context to roughly one million tokens with a factor of 4.0. It cautions that the open-source runtimes apply this scaling statically, which can slightly reduce quality on ordinary-length inputs, and recommends enabling it only when longer context is needed. Its sampling guidance distinguishes general tasks from precise coding tasks. The published page includes benchmark results across coding, reasoning and agentic tasks, but those are the model publisher’s reported results, not independent confirmation of the quantized builds’ performance.

## What the archive adds

New Runtime covered [Ornith-1.5-122B’s original release](https://newruntime.com/posts/ornith-released-ornith-1-5-122b-a-122b-037713/) on 2026-11-07, describing the 122B MoE model and its MIT license. This is a new fact on that known story: Unsloth has now published a set of GGUF quantizations and local-runtime instructions, rather than announcing a new model.

Unsloth has also published GGUF quantizations for other models, including [Qwen-Image-2.1-Turbo](https://newruntime.com/posts/unsloth-released-gguf-quants-of-qwen-image-2-936672/) and [Qwen3.8-Flash-Next](https://newruntime.com/posts/unsloth-released-dynamic-v3-1-ggufs-for-qwen3-09e489/). Those releases provide context for this as part of Unsloth’s model-packaging work, not evidence that these quantizations have equivalent performance.

## Trend signal

This release modestly supports the idea that a model’s usable deployment boundary is shaped by packaging and runtime compatibility as well as model weights. The relevant New Runtime pattern is [Agent-ready software exposes capabilities](https://newruntime.com/patterns/agent-ready-software-exposes-capabilities/): model artifacts that can be served through compatible local interfaces are easier to compose into builder workflows. The connection is limited, however; a quantized checkpoint is not itself a permissioned capability interface.

## Why it matters

A local GGUF option gives teams a practical route to test Ornith in existing inference setups and coding or agent tools that can connect to a local OpenAI-compatible endpoint. That can matter for experimentation involving private code or data, where an externally hosted service may not be the preferred evaluation environment. The range of quantizations lets builders choose among memory footprint and fidelity trade-offs, although the repository excerpt does not provide a measured comparison for these files.

The release does not make a 122B model small by default. The full-precision serving recipe calls for an 80GB GPU, and actual requirements vary with quantization, context length, runtime, and workload. Teams should treat the variants as deployment choices to test against their own tasks, not assume that lower-bit files preserve benchmark performance or meet production latency targets. Compatibility examples reduce setup friction, but do not replace evaluation, monitoring, or access controls in an agent system.

## Limits and uncertainty

The repository lists many quantized artifacts, but the publisher’s benchmark table does not establish how each quantization performs relative to the original model. No independent evaluation of the GGUF variants is cited here, and download or like counts do not demonstrate quality. Hardware requirements, throughput, memory use, and latency for each quantization are not established by the included benchmark results. The model card’s benchmark figures are attributed to the model publisher and depend on the stated harnesses, settings, and evaluators. Teams should verify runtime compatibility and test representative tasks before relying on a quantized build in production.

## Reader takeaway

Unsloth has made Ornith-1.5-122B available in a broad set of GGUF quantizations, with recipes for common local runtimes. The practical gain is a new deployment path for evaluation and local use, not proof that a smaller quantization matches the original model. Choose a variant based on your hardware, then measure quality and latency on the tasks that matter to your team.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- topic hub: [Local Models](https://newruntime.com/topics/local-models.md): Explore the local-models topic hub.
- topic hub: [Open Models](https://newruntime.com/topics/open-models.md): Explore the open-models topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
