The generator proposes programs for a universal Turing machine; their executions yield byte sequences for the learner to autoregressively predict.
The learner uses cross-entropy, while the generator is optimized with reinforcement learning to generate sequences near the learner’s capability frontier. The authors report that, despite neither model training on natural data, zero-shot loss on several natural datasets improves predictably as self-play compute increases. They also report in-context learning and the discovery of recognizable mathematical sequences during training.