The official repository lists an end-to-end Qwen3.6-35B-A3B tutorial that combines NVFP4 W4A4 post-training quantization with quantization-aware distillation. It reports up to 1.30× vLLM throughput over BF16 and 3.1× smaller checkpoints. The repository also says Model Optimizer accepts Hugging Face, PyTorch, and ONNX models, and can export optimized quantized checkpoints to SGLang, TensorRT-LLM, TensorRT, and vLLM.
The mechanism is to pair a highly compressed W4A4 representation with a training-based recovery stage: post-training quantization cuts the precision and checkpoint footprint, while quantization-aware distillation is intended to recover accuracy lost by that compression. This makes the optimization sequence composable with downstream deployment targets rather than leaving teams to stitch together conversion and serving stages themselves.
Why now is that model-serving economics increasingly depend on fitting larger models into constrained memory and extracting more output from the same accelerator fleet. NVIDIA’s new tutorial turns that pressure into a named, reproducible path for one Qwen model, while its library positions the same family of techniques across several inference runtimes.
The first implication is that teams evaluating W4A4 can treat accuracy recovery as part of the deployment design, not merely an after-the-fact benchmark concern. A second implication is that export compatibility can make the optimizer a control point between model preparation and serving infrastructure.
The evidence boundary is that the throughput, size, and accuracy-recovery claims are NVIDIA’s repository statements for a specific Qwen3.6-35B-A3B tutorial; they do not establish equivalent results for other models, hardware, prompts, batch sizes, or serving configurations.
The first watch condition is whether independent deployments reproduce the claimed throughput and quality trade-off on their own hardware and workload mix. Also watch whether the workflow gains broader model coverage and whether its artifacts remain portable across the listed serving frameworks.