{"type":"post","id":"the-hitchhiker-s-guide-to-training-on-compute-f68cc8","slug":"the-hitchhiker-s-guide-to-training-on-compute-f68cc8","title":"The Hitchhiker's Guide to Training on Compute Islands","description":"Core Automation reports training a roughly 32-billion-parameter sparse mixture-of-experts model across two GPU clusters about 9,000 miles apart, using different accelerators and parallelism schemes.","retrieval_nugget":"Core Automation reports training a roughly 32-billion-parameter sparse mixture-of-experts model across two GPU clusters about 9,000 miles apart, using different accelerators and parallelism schemes. The case suggests distributed training software may make otherwise disconnected compute usable, though the evidence is a company-reported demonstration rather than a general validation.","published_at":"2026-11-03","updated_at":"2026-11-06","record_date":"2026-11-03","date_kind":"published_at","topics":["compute"],"entities":["Core Automation","GB300","B300","Distributed Shampoo","DiLoCo"],"related_patterns":[],"novelty_label":"new_angle","what_changed":"It frames scattered, heterogeneous compute clusters as a practical way to address power-constrained GPU availability, demonstrating cross-cluster MoE training and butterfly mixing.","prior_refs":[],"source_url":"https://www.coreauto.com/hitchhikers-guide-to-training-on-compute-islands","source_urls":["https://www.coreauto.com/hitchhikers-guide-to-training-on-compute-islands"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:the-hitchhiker-s-guide-to-training-on-compute-f68cc8","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/the-hitchhiker-s-guide-to-training-on-compute-f68cc8/","markdown":"https://newruntime.com/posts/the-hitchhiker-s-guide-to-training-on-compute-f68cc8.md","json":"https://newruntime.com/posts/the-hitchhiker-s-guide-to-training-on-compute-f68cc8.json"},"markdown_url":"https://newruntime.com/posts/the-hitchhiker-s-guide-to-training-on-compute-f68cc8.md","next_reads":[{"type":"topic","path":"/topics/compute/","url":"https://newruntime.com/topics/compute/","title":"Compute","reason":"Explore the compute topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/compute.md","json_url":"https://newruntime.com/topics/compute.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}