---
type: "post"
id: "release-candidate-v1-0-0-rc-3-huggingface-32da6a"
slug: "release-candidate-v1-0-0-rc-3-huggingface-32da6a"
title: "Release candidate v1.0.0-rc.3 · huggingface/tokenizers"
description: "Hugging Face has released Tokenizers v1.0.0-rc.3, a pre-release centered on new WebAssembly bindings."
retrieval_nugget: "Hugging Face has released Tokenizers v1.0.0-rc.3, a pre-release centered on new WebAssembly bindings."
published_at: "2026-10-24"
updated_at: "2026-10-24"
record_date: "2026-10-24"
date_kind: "published_at"
topics: ["developer-tools","open-source"]
entities: ["Hugging Face","Tokenizers","tokenizers-wasm","ModernBERT"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "A new tokenizers v1.0.0-rc.3 release adds WebAssembly bindings and fixes."
prior_refs: ["memory:b3a1e0fa-3754-874a-ad01-a6da1c3924e5"]
source_url: "https://github.com/huggingface/tokenizers/releases/tag/v1.0.0-rc.3"
source_urls: ["https://github.com/huggingface/tokenizers/releases/tag/v1.0.0-rc.3"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:release-candidate-v1-0-0-rc-3-huggingface-32da6a"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/release-candidate-v1-0-0-rc-3-huggingface-32da6a/","markdown":"https://newruntime.com/posts/release-candidate-v1-0-0-rc-3-huggingface-32da6a.md","json":"https://newruntime.com/posts/release-candidate-v1-0-0-rc-3-huggingface-32da6a.json"}
---

# Release candidate v1.0.0-rc.3 · huggingface/tokenizers

## Retrieval answer

Hugging Face has released Tokenizers v1.0.0-rc.3, a pre-release centered on new WebAssembly bindings.

Hugging Face has released Tokenizers v1.0.0-rc.3, a pre-release centered on new WebAssembly bindings. The bindings are published on npm as `tokenizers-wasm`; the project says release candidates use the `next` dist-tag, with installation via `npm install tokenizers-wasm@next`.

The release also adds Python support for encoding pairs of texts. Its fixes cover byte-level BPE vocabularies missing bytes that cannot occur in UTF-8, including a cited ModernBERT case; WordPiece vocabularies containing an empty-string token; and `tokenizer.json` files using regex Split pre-tokenizers or Replace normalizers, which had been rejected by Python, Node, and WASM bindings.

Other changes make a 64-bit vocabulary-hash collision an error rather than silently dropping a token, remove an unnecessary Rust/Python encoding copy, and document feature selection through the umbrella Rust crate. The release notes say feature parity with version 0.23 remains the main milestone before stable 1.0.0. They do not establish a stable-release date, browser performance figures, or compatibility guarantees beyond the listed fixes.

Source: https://github.com/huggingface/tokenizers/releases/tag/v1.0.0-rc.3

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Developer Tools](https://newruntime.com/topics/developer-tools.md): Explore the developer-tools topic hub.
- topic hub: [Open Source](https://newruntime.com/topics/open-source.md): Explore the open-source topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)