---
id: "nr-hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0"
slug: "hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0"
title: "Hamel Husain adds five questions to his AI evals FAQ"
observed_at: "2026-09-18T13:48:49+00:00"
record_date: "2026-09-18"
date_kind: "observed_at"
novelty: "new"
source_platform: "x.com"
topics: ["evals","llm-judges"]
entities: ["Hamel Husain"]
related_patterns: []
source_url: "https://x.com/HamelHusain/status/2099929381850468675"
source_urls: ["https://x.com/HamelHusain/status/2099929381850468675"]
basket: {"id":"0325e6b8-f527-4f9b-bfa3-3f30e57f380b","revision":1,"cluster_id":"5bd8e03b-4135-48a9-ad7f-aabbb53f92b6"}
type: "raw_signal"
description: "Hamel Husain expanded his AI evals FAQ to 48 questions, adding entries on rubrics, sensitive traces, large traces, LLM judge context and stale gold datasets."
why_it_matters: "The new entries target operational problems that appear once evals run on real production traces. The post lists the questions only; the answers themselves are on the linked FAQ page and are not summarised here."
verification_level: "source-inspected"
signal_type: "newsroom_basket_signal"
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0"
retrieval_nugget: "Hamel Husain expanded his AI evals FAQ to 48 questions, adding entries on rubrics, sensitive traces, large traces, LLM judge context and stale gold datasets. Hamel Husain updated his AI evals FAQ with five new questions, bringing it to 48 questions and answers. The additions cover whether a reference answer or rubric is needed before annotating data, running evals when"
status: "published"
visuals: []
routes: {"html":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0/","markdown":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0.md","json":"https://newruntime.com/signals/hamel-husain-adds-five-questions-to-his-ai-evals-5bd8e0.json"}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
---

# Hamel Husain adds five questions to his AI evals FAQ

## Retrieval answer

Hamel Husain expanded his AI evals FAQ to 48 questions, adding entries on rubrics, sensitive traces, large traces, LLM judge context and stale gold datasets. Hamel Husain updated his AI evals FAQ with five new questions, bringing it to 48 questions and answers. The additions cover whether a reference answer or rubric is needed before annotating data, running evals when

Hamel Husain updated his AI evals FAQ with five new questions, bringing it to 48 questions and answers. The additions cover whether a reference answer or rubric is needed before annotating data, running evals when traces contain sensitive data, reviewing very large traces, how much context to give an LLM judge, and what to do when a gold eval dataset becomes stale.

The new entries target operational problems that appear once evals run on real production traces. The post lists the questions only; the answers themselves are on the linked FAQ page and are not summarised here.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Evals](https://newruntime.com/topics/evals.md): Explore the evals topic hub.
- topic hub: [Llm Judges](https://newruntime.com/topics/llm-judges.md): Explore the llm-judges topic hub.
- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)