---
type: "post"
id: "kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223"
slug: "kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223"
title: "Kaggle and Rehearsals launched Human Behavior Bench, scoring LLMs on predicting real human survey distributions."
description: "Human Behavior Bench makes a specific claim testable: a model can sound plausible about people yet fail to reproduce how a defined group distributes its answers."
retrieval_nugget: "Human Behavior Bench makes a specific claim testable: a model can sound plausible about people yet fail to reproduce how a defined group distributes its answers."
published_at: "2026-10-21"
updated_at: "2026-10-21"
record_date: "2026-10-21"
date_kind: "published_at"
topics: ["evals","models"]
source_urls: ["https://www.runrehearsals.com/human-behavior-bench"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:406a32649de0d50fabdac59f1c15e5e9e56d185d715fbb09f583212555174b87","reviewed_at":"2026-10-21T08:05:08.924Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Audience-level prediction needs evaluation against measured distributions, not a fluent account of what people might do.","observed_facts":[{"text":"The benchmark reports 1,199 tasks based on surveys of 1,244 U.S. adults, with 239 public and 960 private tasks.","source_urls":["https://www.runrehearsals.com/human-behavior-bench"]},{"text":"The published table lists six LLM scores between 76.2 and 80.6, and Rehearsals reports 94.1 for Engram 1.0.","source_urls":["https://www.runrehearsals.com/human-behavior-bench"]},{"text":"The source says base models ran through Kaggle's harness while Engram used Rehearsals' in-house evaluation system, with the same case and metric setup.","source_urls":["https://www.runrehearsals.com/human-behavior-bench"]}],"mechanism":"The mechanism is to compare each model's predicted answer distribution with survey ground truth for a defined cohort, rather than grade a free-form explanation.","why_now":"The benchmark makes a gap between conversational plausibility and audience-level prediction legible as a scored evaluation, including in product and marketing questions.","implications":["The first implication is that teams should not treat a model's confident audience persona or preference summary as equivalent to measured user research.","Evaluation results may identify domains, such as product choices, where model-generated audience estimates need stronger validation."],"evidence_boundary":"The evidence boundary is that These are organizer-reported results; base models used Kaggle's harness and Engram used Rehearsals' in-house harness, albeit with the same case and metric setup. The page does not establish independent replication, representativeness beyond the U.S. panel, or predictive validity for every business decision.","watch_conditions":["The first watch condition is whether independent evaluators reproduce the rankings and whether performance generalizes across new populations, survey modes, and decision domains."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied for this specific benchmark result; it establishes a distinct analytical record about measuring audience-level prediction against survey distributions."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223/","markdown":"https://newruntime.com/posts/kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223.md","json":"https://newruntime.com/posts/kaggle-and-rehearsals-launched-human-behavior-bench-scoring-0fd223.json"}
related_patterns: ["ai-engineering-evaluations-are-splitting-into-task-specific"]
---

# Kaggle and Rehearsals launched Human Behavior Bench, scoring LLMs on predicting real human survey distributions.

## Retrieval answer

Human Behavior Bench makes a specific claim testable: a model can sound plausible about people yet fail to reproduce how a defined group distributes its answers.

Human Behavior Bench makes a specific claim testable: a model can sound plausible about people yet fail to reproduce how a defined group distributes its answers. The thesis is Audience-level prediction needs evaluation against measured distributions, not a fluent account of what people might do. The benchmark reports 1,199 tasks drawn from surveys of 1,244 U.S. adults, with 239 public tasks and 960 held out. Its metric is 100 × (1 − total variation distance), comparing predicted and observed answer distributions.

The mechanism is to compare each model's predicted answer distribution with survey ground truth for a defined cohort, rather than grade a free-form explanation. Systems receive a closed-form question tied to an audience and predict percentages across fixed options. The published table lists six LLM scores between 76.2 and 80.6; Rehearsals reports 94.1 for Engram 1.0. The source says base models ran through Kaggle's harness while Engram used Rehearsals' in-house evaluation system with the same case and metric setup. Why now: The benchmark makes a gap between conversational plausibility and audience-level prediction legible as a scored evaluation, including in product and marketing questions. The organizers also report LLM top-answer accuracy of 41% on product questions and 74% on brand-perception questions.

The first implication is that teams should not treat a model's confident audience persona or preference summary as equivalent to measured user research. The evidence boundary is that These are organizer-reported results; base models used Kaggle's harness and Engram used Rehearsals' in-house harness, albeit with the same case and metric setup. The page does not establish independent replication, representativeness beyond the U.S. panel, or predictive validity for every business decision. The first watch condition is whether independent evaluators reproduce the rankings and whether performance generalizes across new populations, survey modes, and decision domains. The next useful evidence would therefore be an independent replication on additional populations and decision settings.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Evals](https://newruntime.com/topics/evals.md): Explore the evals topic hub.
- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)