{"type":"post","id":"transformers-5-22-adds-lora-peft-training-on-d1c416","slug":"transformers-5-22-adds-lora-peft-training-on-d1c416","title":"Transformers 5.22 adds LoRA/PEFT training on frozen FP8 checkpoints","description":"Hugging Face Transformers 5.22.0 adds backward support for dense and grouped FP8 matrix multiplications with frozen weights, allowing PEFT adapters such as LoRA to train on supported FP8 checkpoints.","retrieval_nugget":"Hugging Face Transformers 5.22.0 adds backward support for dense and grouped FP8 matrix multiplications with frozen weights, allowing PEFT adapters such as LoRA to train on supported FP8 checkpoints. It removes a specific adapter-tuning blocker without making FP8 base weights trainable; FP4 and several expert paths remain unsupported.","published_at":"2026-11-01","updated_at":"2026-11-01","record_date":"2026-11-01","date_kind":"published_at","topics":["harness-engineering","agent-runtime"],"entities":["Hugging Face","Transformers","PEFT","LoRA","MiniMax-M2-Tiny-FP8"],"related_patterns":["harness-architecture-outlives-model-choice"],"novelty_label":"new_fact","what_changed":"Transformers 5.22 adds backward support for LoRA/PEFT training on frozen FP8 checkpoints, with FP8 input gradients close to an fp32 dequantized baseline; some FP4 and expert paths remain unsupported.","prior_refs":["memory:0272194d-b54f-8a7c-ad85-f07f72239003","memory:6f946127-ec4e-8d1e-aaf5-d2620cda013a"],"source_url":"https://github.com/huggingface/transformers/pull/50031","source_urls":["https://github.com/huggingface/transformers/pull/50031"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:transformers-5-22-adds-lora-peft-training-on-d1c416","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/transformers-5-22-adds-lora-peft-training-on-d1c416/","markdown":"https://newruntime.com/posts/transformers-5-22-adds-lora-peft-training-on-d1c416.md","json":"https://newruntime.com/posts/transformers-5-22-adds-lora-peft-training-on-d1c416.json"},"next_reads":[{"type":"topic","path":"/topics/harness-engineering/","url":"https://newruntime.com/topics/harness-engineering/","title":"Harness Engineering","reason":"Explore the harness-engineering topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/harness-engineering.md","json_url":"https://newruntime.com/topics/harness-engineering.json"},{"type":"topic","path":"/topics/agent-runtime/","url":"https://newruntime.com/topics/agent-runtime/","title":"Agent Runtime","reason":"Explore the agent-runtime topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/agent-runtime.md","json_url":"https://newruntime.com/topics/agent-runtime.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}