---
type: "post"
id: "nvidia-model-optimizer-analysis-6652fd"
slug: "nvidia-model-optimizer-analysis-6652fd"
title: "NVIDIA’s Model Optimizer repository now includes an end-to-end tutorial for Qwen3.6-35B-A3B combining NVFP4"
description: "NVIDIA is packaging aggressive low-precision inference as a repeatable optimization workflow rather than as a standalone quantization step."
retrieval_nugget: "NVIDIA is packaging aggressive low-precision inference as a repeatable optimization workflow rather than as a standalone quantization step."
published_at: "2026-09-24"
updated_at: "2026-09-24"
record_date: "2026-09-24"
date_kind: "published_at"
topics: []
source_urls: ["https://github.com/NVIDIA/Model-Optimizer"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:nvidia-model-optimizer-analysis-6652fd"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:14b114867838926c29abd5b7763736d9c6c976757b29be41cfd5252855256adc","reviewed_at":"2026-09-24T16:03:14.657Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":2,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"NVIDIA is packaging aggressive low-precision inference as a repeatable optimization workflow rather than as a standalone quantization step.","observed_facts":[{"text":"The official repository lists an end-to-end Qwen3.6-35B-A3B tutorial combining NVFP4 W4A4 post-training quantization with quantization-aware distillation.","source_urls":["https://github.com/NVIDIA/Model-Optimizer"]},{"text":"NVIDIA reports up to 1.30× vLLM throughput over BF16 and 3.1× smaller checkpoints for that tutorial.","source_urls":["https://github.com/NVIDIA/Model-Optimizer"]},{"text":"The repository states that Model Optimizer accepts Hugging Face, PyTorch, and ONNX models and exports optimized checkpoints for SGLang, TensorRT-LLM, TensorRT, and vLLM.","source_urls":["https://github.com/NVIDIA/Model-Optimizer"]}],"mechanism":"The mechanism is to pair a highly compressed W4A4 representation with a training-based recovery stage: post-training quantization cuts the precision and checkpoint footprint, while quantization-aware distillation is intended to recover accuracy lost by that compression.","why_now":"Why now is that model-serving economics increasingly depend on fitting larger models into constrained memory and extracting more output from the same accelerator fleet.","implications":["The first implication is that teams evaluating W4A4 can treat accuracy recovery as part of the deployment design, not merely an after-the-fact benchmark concern.","Export compatibility can make the optimizer a control point between model preparation and serving infrastructure."],"evidence_boundary":"The evidence boundary is that the throughput, size, and accuracy-recovery claims are NVIDIA’s repository statements for a specific Qwen3.6-35B-A3B tutorial; they do not establish equivalent results for other models, hardware, prompts, batch sizes, or serving configurations.","watch_conditions":["The first watch condition is whether independent deployments reproduce the claimed throughput and quality trade-off on their own hardware and workload mix.","Whether the workflow gains broader model coverage and whether its artifacts remain portable across the listed serving frameworks."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied with enough content to establish continuity; this begins a distinct record for NVIDIA’s newly documented W4A4 plus quantization-aware-distillation workflow."}
routes: {"html":"https://newruntime.com/posts/nvidia-model-optimizer-analysis-6652fd/","markdown":"https://newruntime.com/posts/nvidia-model-optimizer-analysis-6652fd.md","json":"https://newruntime.com/posts/nvidia-model-optimizer-analysis-6652fd.json"}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
---

# NVIDIA’s Model Optimizer repository now includes an end-to-end tutorial for Qwen3.6-35B-A3B combining NVFP4

## Retrieval answer

NVIDIA is packaging aggressive low-precision inference as a repeatable optimization workflow rather than as a standalone quantization step.

NVIDIA is packaging aggressive low-precision inference as a repeatable optimization workflow rather than as a standalone quantization step.

The official repository lists an end-to-end Qwen3.6-35B-A3B tutorial that combines NVFP4 W4A4 post-training quantization with quantization-aware distillation. It reports up to 1.30× vLLM throughput over BF16 and 3.1× smaller checkpoints. The repository also says Model Optimizer accepts Hugging Face, PyTorch, and ONNX models, and can export optimized quantized checkpoints to SGLang, TensorRT-LLM, TensorRT, and vLLM.

The mechanism is to pair a highly compressed W4A4 representation with a training-based recovery stage: post-training quantization cuts the precision and checkpoint footprint, while quantization-aware distillation is intended to recover accuracy lost by that compression. This makes the optimization sequence composable with downstream deployment targets rather than leaving teams to stitch together conversion and serving stages themselves.

Why now is that model-serving economics increasingly depend on fitting larger models into constrained memory and extracting more output from the same accelerator fleet. NVIDIA’s new tutorial turns that pressure into a named, reproducible path for one Qwen model, while its library positions the same family of techniques across several inference runtimes.

The first implication is that teams evaluating W4A4 can treat accuracy recovery as part of the deployment design, not merely an after-the-fact benchmark concern. A second implication is that export compatibility can make the optimizer a control point between model preparation and serving infrastructure.

The evidence boundary is that the throughput, size, and accuracy-recovery claims are NVIDIA’s repository statements for a specific Qwen3.6-35B-A3B tutorial; they do not establish equivalent results for other models, hardware, prompts, batch sizes, or serving configurations.

The first watch condition is whether independent deployments reproduce the claimed throughput and quality trade-off on their own hardware and workload mix. Also watch whether the workflow gains broader model coverage and whether its artifacts remain portable across the listed serving frameworks.

Source: https://github.com/NVIDIA/Model-Optimizer

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)