{"type":"post","id":"toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555","slug":"toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555","title":"Toward RL on mixed-precision, mixed-accelerator samplers","description":"**Thesis:** Heterogeneous RL samplers can be useful when training explicitly corrects for the policy mismatch they introduce.","retrieval_nugget":"**Thesis:** Heterogeneous RL samplers can be useful when training explicitly corrects for the policy mismatch they introduce.","published_at":"2026-10-13","updated_at":"2026-10-13","record_date":"2026-10-13","date_kind":"published_at","topics":[],"source_urls":["https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:7da481c900c062f878713bbd5f8aedd1a8bb74ad61d84f5f10d9ef2137c12693","reviewed_at":"2026-10-13T20:07:20.881Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Heterogeneous RL samplers can be useful when training explicitly corrects for the policy mismatch they introduce.","observed_facts":[{"text":"Modal trained Qwen3.6-35B-A3B on agentic coding tasks with fully asynchronous RL, using samplers across six accelerator types and a BF16 B200 trainer.","source_urls":["https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers"]},{"text":"The source reports that only SC + MIS, with or without top-p, trained stably to the end; uncorrected runs collapsed.","source_urls":["https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers"]},{"text":"Reported pass@4 reached roughly 65–75% on HARD-51 and about 94% on the full SWE-bench Pro V2 set.","source_urls":["https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers"]}],"mechanism":"Masked importance sampling reweights token contributions and masks extreme ratios, while score centering removes expected gradient drift under the sampling policy; together they address the sampler-trainer policy mismatch.","why_now":"Modal says sampling consumes 80–85% of GPUs in large RL runs, making serving-side efficiency consequential to total training economics.","implications":["RL infrastructure teams can evaluate sampler hardware and precision independently, while treating off-policy correction as part of the training design."],"evidence_boundary":"This is a vendor-reported experiment on one model, coding-task setup, and benchmark protocol; it does not establish broad generalization or independent replication.","watch_conditions":["Whether independent teams reproduce stable convergence and cost savings on other models and workloads using mixed-accelerator samplers."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this creates a distinct record about off-policy correction enabling mixed-accelerator samplers in asynchronous RL."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555/","markdown":"https://newruntime.com/posts/toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555.md","json":"https://newruntime.com/posts/toward-rl-on-mixed-precision-mixed-accelerator-samplers-e45555.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}