{"type":"post","id":"rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de","slug":"rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de","title":"rawmodels/unslop-harbor | Datasets at Hugging Face","description":"**Thesis:** Anti-slop training becomes more inspectable when a reward environment exposes task-level rubrics, calibration, and held-out tests rather than presenting a model claim alone.","retrieval_nugget":"**Thesis:** Anti-slop training becomes more inspectable when a reward environment exposes task-level rubrics, calibration, and held-out tests rather than presenting a model claim alone.","published_at":"2026-10-16","updated_at":"2026-10-16","record_date":"2026-10-16","date_kind":"published_at","topics":["evals","models","open-models"],"source_urls":["https://huggingface.co/datasets/rawmodels/unslop-harbor"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:bdb5c2c18efc8b02b0129bcdf0456ea12075276c902d5fc2fc9128cb42e15cb3","reviewed_at":"2026-10-16T20:11:23.635Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Anti-slop training becomes more inspectable when a reward environment exposes task-level rubrics, calibration, and held-out tests rather than presenting a model claim alone.","observed_facts":[{"text":"Unslop Harbor contains 200 tasks: 100 tweet continuations and 100 HumanEval function bodies, split across English and Russian.","source_urls":["https://huggingface.co/datasets/rawmodels/unslop-harbor"]},{"text":"The dataset card says Claude Opus 5.5 grades task-specific rubrics, while a separate rule handles tweet length and code receives reward only if tests pass.","source_urls":["https://huggingface.co/datasets/rawmodels/unslop-harbor"]},{"text":"It reports that the judge selected human text over assistant-style rewrites in 98.3% of 30 held-out tweet tasks.","source_urls":["https://huggingface.co/datasets/rawmodels/unslop-harbor"]}],"mechanism":"The release decomposes a fuzzy preference into a bounded scoring procedure: length is handled independently from judged quality for tweets, and code correctness gates the judge score. Its calibration ties the reward scale to Qwen3.8-27B answers, while a pinned judge-prompt hash is intended to prevent silent drift.","why_now":"Teams experimenting with reinforcement learning for writing need more than a slogan about reducing slop; an inspectable benchmark and executable grader make the proposed objective easier to reproduce and adapt.","implications":["Teams can test anti-slop reward designs against explicit tasks and failure conditions before treating subjective style scores as a production signal.","The dataset offers a compact example of separating preference quality from mechanical constraints, but its calibration is specific to its judge and task mix."],"evidence_boundary":"The public card reports benchmark construction and results, not independent replication or evidence that the trained model generalizes to all writing. Its authors say the judge was qualified to catch degraded text in known-order pairs; ranking two good answers remains unvalidated. Calibration also depends on Opus 5.5 and should not be assumed to transfer to another judge.","watch_conditions":["Watch for independent evaluations on stronger, genuinely competitive answers and for recalibration when the judge model changes."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; archive snippets concern other products or a separate anti-slop linting tool, not this dataset and reward environment."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de/","markdown":"https://newruntime.com/posts/rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de.md","json":"https://newruntime.com/posts/rawmodels-unslop-harbor-datasets-at-hugging-face-analysis-7de3de.json"},"related_patterns":["ai-engineering-evaluations-are-splitting-into-task-specific"],"next_reads":[{"type":"topic","path":"/topics/evals/","url":"https://newruntime.com/topics/evals/","title":"Evals","reason":"Explore the evals topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/evals.md","json_url":"https://newruntime.com/topics/evals.json"},{"type":"topic","path":"/topics/models/","url":"https://newruntime.com/topics/models/","title":"Models","reason":"Explore the models topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/models.md","json_url":"https://newruntime.com/topics/models.json"},{"type":"topic","path":"/topics/open-models/","url":"https://newruntime.com/topics/open-models/","title":"Open Models","reason":"Explore the open-models topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/open-models.md","json_url":"https://newruntime.com/topics/open-models.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}