---
type: "post"
id: "hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5"
slug: "hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5"
title: "Hugging Face Transformers merges 'Add Kolibri 1' (PR #49463), adding Aleph Alpha's MoE model"
description: "Hugging Face Transformers merged pull request #49463, adding a native `kolibri1` model architecture for Aleph Alpha’s Kolibri 1."
retrieval_nugget: "Hugging Face Transformers merged pull request #49463, adding a native `kolibri1` model architecture for Aleph Alpha’s Kolibri 1."
published_at: "2026-10-24"
updated_at: "2026-10-24"
record_date: "2026-10-24"
date_kind: "published_at"
topics: ["models","open-models","open-source"]
entities: ["Hugging Face","Transformers","Kolibri 1","Aleph Alpha","Qwen3-MoE","vLLM"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "The merged Kolibri 1 support PR adds implementation details: its Qwen3-MoE-based architecture and attention, normalization and routing differences."
prior_refs: ["memory:4a2bf6cd-ba1e-8503-a2b1-d24529c4c8a5"]
source_url: "https://github.com/huggingface/transformers/pull/49463"
source_urls: ["https://github.com/huggingface/transformers/pull/49463"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5/","markdown":"https://newruntime.com/posts/hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5.md","json":"https://newruntime.com/posts/hugging-face-transformers-merges-add-kolibri-1-pr-d3aab5.json"}
---

# Hugging Face Transformers merges 'Add Kolibri 1' (PR #49463), adding Aleph Alpha's MoE model

## Retrieval answer

Hugging Face Transformers merged pull request #49463, adding a native `kolibri1` model architecture for Aleph Alpha’s Kolibri 1.

Hugging Face Transformers merged pull request #49463, adding a native `kolibri1` model architecture for Aleph Alpha’s Kolibri 1. The implementation is modular on top of Qwen3-MoE and includes a conversion mapping for the model’s weights.

The PR describes four sliding-window attention layers for each full-attention layer, with RoPE used only on the sliding-window layers. It also implements sigmoid-based expert routing with a correction bias and an ungated shared expert. The submitted code reports that the Hub configuration loads with `AutoConfig` and that every indexed safetensors weight key maps through the conversion layer.

The contributor reports 155 passing CPU tests and 157 skipped tests, plus GPU checks of the matrix-multiplication path and a backward pass. It also reports a 94.2% GSM8K result through vLLM’s Transformers backend using real FP8 weights.

The pull request does not establish general model quality, inference cost, licence terms, or whether the model weights are publicly accessible. It is evidence of library-level architecture support, not a comparative model evaluation.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- topic hub: [Open Models](https://newruntime.com/topics/open-models.md): Explore the open-models topic hub.
- topic hub: [Open Source](https://newruntime.com/topics/open-source.md): Explore the open-source topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)