Observed facts: Unslop Harbor contains 200 tasks: 100 tweet continuations and 100 HumanEval function bodies, split across English and Russian. The dataset card says Claude Opus 5.5 grades task-specific rubrics, while a separate rule handles tweet length and code receives reward only if tests pass. It reports that the judge selected human text over assistant-style rewrites in 98.3% of 30 held-out tweet tasks.
Mechanism: The release decomposes a fuzzy preference into a bounded scoring procedure: length is handled independently from judged quality for tweets, and code correctness gates the judge score. Its calibration ties the reward scale to Qwen3.8-27B answers, while a pinned judge-prompt hash is intended to prevent silent drift.
Why now: Teams experimenting with reinforcement learning for writing need more than a slogan about reducing slop; an inspectable benchmark and executable grader make the proposed objective easier to reproduce and adapt.
Implications: Teams can test anti-slop reward designs against explicit tasks and failure conditions before treating subjective style scores as a production signal. The dataset offers a compact example of separating preference quality from mechanical constraints, but its calibration is specific to its judge and task mix.
Evidence boundary: The public card reports benchmark construction and results, not independent replication or evidence that the trained model generalizes to all writing. Its authors say the judge was qualified to catch degraded text in known-order pairs; ranking two good answers remains unvalidated. Calibration also depends on Opus 5.5 and should not be assumed to transfer to another judge.
Watch condition: Watch for independent evaluations on stronger, genuinely competitive answers and for recalibration when the judge model changes.
The release is a distinct analytical record because no prior New Runtime record was supplied; nearby archive snippets concern other products or a separate anti-slop linting tool, not this dataset and reward environment.