NVIDIA reports up to 1.30× vLLM throughput versus BF16, 3.1× smaller checkpoints, and accuracy recovery from the aggressive W4A4 conversion. The library composes optimization methods and exports quantized checkpoints for inference frameworks including SGLang, TensorRT-LLM, TensorRT, and vLLM.
Raw SignalSep 24, 20261 min read1 source
NVIDIA’s Model Optimizer repository now includes an end-to-end tutorial for Qwen3.6-35B-A3B combining NVFP4
In one minute
NVIDIA’s Model Optimizer repository now includes an end-to-end tutorial for Qwen3.6-35B-A3B combining NVFP4 W4A4 post-training quantization with quantization-aware distillation.