{"type":"post","stable_id":"post:smevals-small-model-evaluation-framework","slug":"smevals-small-model-evaluation-framework","title":"smevals Treats Small-Model Testing As A First-Class Agent Harness","description":"smevals is a GitHub framework for running evaluations against small and large models, making model choice a testable harness problem.","retrieval_nugget":"The signal is the normalization of small-model evaluation: teams need repeatable tasks, metrics, and comparable runs before replacing large default models with cheaper local or specialized ones.","published_at":"2026-08-04","updated_at":"2026-08-05","record_date":"2026-08-04","date_kind":"published_at","status":"published","topics":["evals","small-models","agent-harness","developer-tools"],"entities":["smevals","Prime Radiant"],"source_urls":["https://github.com/prime-radiant-inc/smevals"],"source_title":"smevals: A framework for running evals against small and large models","source_type":"github","origin":{"batch_id":"ada1e768-4e4e-42a3-a5fc-1ca9a880cb1b","batch_index":5,"channel":"chatgpt-batch","restored_from_reserve":false,"restored_from_supporting":false},"schema_version":"newruntime-agent-readable-v0.2","visuals":[{"role":"hero","src":"/images/drip/smevals-small-model-evaluation-framework/smevals-small-model-evaluation-framework.webp","alt":"A whiteboard hub-and-spoke diagram showing a repeatable evaluation harness comparing small and large models before a model-routing decision.","caption":"New Runtime synthesis."}],"routes":{"html":"https://newruntime.com/posts/smevals-small-model-evaluation-framework/","markdown":"https://newruntime.com/posts/smevals-small-model-evaluation-framework.md","json":"https://newruntime.com/posts/smevals-small-model-evaluation-framework.json"}}