{"id":"nr-hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0","slug":"hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0","title":"Hamel Husain adds five questions to his AI evals FAQ","observed_at":"2026-09-18T13:48:49+00:00","record_date":"2026-09-18","date_kind":"observed_at","novelty":"new","source_platform":"x.com","topics":["evals","llm-judges"],"entities":["Hamel Husain"],"related_patterns":[],"source_url":"https://x.com/HamelHusain/status/2099929381850468675","source_urls":["https://x.com/HamelHusain/status/2099929381850468675"],"basket":{"id":"0325e6b8-f527-4f9b-bfa3-3f30e57f380b","revision":1,"cluster_id":"5bd8e03b-4135-48a9-ad7f-aabbb53f92b6"},"type":"raw_signal","description":"Hamel Husain expanded his AI evals FAQ to 48 questions, adding entries on rubrics, sensitive traces, large traces, LLM judge context and stale gold datasets.","why_it_matters":"The new entries target operational problems that appear once evals run on real production traces. The post lists the questions only; the answers themselves are on the linked FAQ page and are not summarised here.","verification_level":"source-inspected","signal_type":"newsroom_basket_signal","schema_version":"newruntime-agent-readable-v0.2","stable_id":"signal:hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0","retrieval_nugget":"Hamel Husain expanded his AI evals FAQ to 48 questions, adding entries on rubrics, sensitive traces, large traces, LLM judge context and stale gold datasets. Hamel Husain updated his AI evals FAQ with five new questions, bringing it to 48 questions and answers. The additions cover whether a reference answer or rubric is needed before annotating data, running evals when","status":"published","visuals":[],"routes":{"html":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0/","markdown":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0.md","json":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0.json"},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"next_reads":[{"type":"topic","path":"/topics/evals/","url":"https://newruntime.com/topics/evals/","title":"Evals","reason":"Explore the evals topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/evals.md","json_url":"https://newruntime.com/topics/evals.json"},{"type":"topic","path":"/topics/llm-judges/","url":"https://newruntime.com/topics/llm-judges/","title":"Llm Judges","reason":"Explore the llm-judges topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/llm-judges.md","json_url":"https://newruntime.com/topics/llm-judges.json"},{"type":"collection","path":"/signals/","url":"https://newruntime.com/signals/","title":"Raw Signals","reason":"Open every source-linked raw signal.","media_type":"text/html","json_url":"https://newruntime.com/signals.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}