{"type":"post","id":"baseten-published-a-technical-explainer-on-selecting-layer-b2914a","slug":"baseten-published-a-technical-explainer-on-selecting-layer-b2914a","title":"Baseten published a technical explainer on selecting layer precision for NVFP4 quantization","description":"Thesis: High-quality NVFP4 quantization depends on choosing precision per layer, and estimating those choices requires accounting for interactions among quantized layers.","retrieval_nugget":"Thesis: High-quality NVFP4 quantization depends on choosing precision per layer, and estimating those choices requires accounting for interactions among quantized layers.","published_at":"2026-10-09","updated_at":"2026-10-09","record_date":"2026-10-09","date_kind":"published_at","topics":[],"source_urls":["https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:baseten-published-a-technical-explainer-on-selecting-layer-b2914a","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:d25f6410eff5d9ec7a715a9e59b60d2f6f4549a3d2dde6f94c65c9a1fb747305","reviewed_at":"2026-10-09T20:05:06.596Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"High-quality NVFP4 quantization depends on choosing precision per layer, and estimating those choices requires accounting for interactions among quantized layers.","observed_facts":[{"text":"The post says NVFP4 divides tensors into blocks of 16 values, each with its own scale.","source_urls":["https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization"]},{"text":"The post says activation scales require calibration data because activations depend on inputs, while weights are fixed.","source_urls":["https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization"]},{"text":"The post describes SaturationQuant using Aumann–Shapley scoring to account for interactions among quantized layers.","source_urls":["https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization"]}],"mechanism":"The post contrasts architecture-based heuristics with sensitivity scoring. Isolated-layer scores measure the effect of quantizing one layer while other layers remain at baseline; Baseten argues those scores cannot simply be added for a mixed-precision recipe because loss can saturate as more layers are quantized. SaturationQuant uses Aumann–Shapley scoring to build this effect into scores that can be summed.","why_now":"NVFP4 offers smaller weights and potentially faster computation on hardware with native FP4 support, so preserving quality while assigning precision has direct deployment relevance.","implications":["Quantization choices should be judged as a whole recipe, not by ranking layers in isolation.","Calibration data and the target workload matter when assigning precision."],"evidence_boundary":"The source explains the method and its rationale, but the supplied material does not provide independent comparative validation.","watch_conditions":["Independent evaluations across models and representative calibration datasets would show whether the described scoring approach predicts quality reliably in practice."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied; this technical explainer therefore starts a distinct record on NVFP4 layer selection and saturation-aware quantization scoring."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/baseten-published-a-technical-explainer-on-selecting-layer-b2914a/","markdown":"https://newruntime.com/posts/baseten-published-a-technical-explainer-on-selecting-layer-b2914a.md","json":"https://newruntime.com/posts/baseten-published-a-technical-explainer-on-selecting-layer-b2914a.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}