---
type: "raw_signal"
id: "strengthening-ai-safety-evaluation-through-independent-third-party-3348d0"
slug: "strengthening-ai-safety-evaluation-through-independent-third-party-3348d0"
title: "Strengthening AI safety evaluation through independent third-party review"
description: "Amazon says it partnered with the independent nonprofit AI Verification and Evaluation Research Institute (AVERI) to review a random sample of prompts from its internal safety-evaluation suite."
retrieval_nugget: "Amazon says it partnered with the independent nonprofit AI Verification and Evaluation Research Institute (AVERI) to review a random sample of prompts from its internal safety-evaluation suite."
observed_at: "2026-10-20"
record_date: "2026-10-20"
date_kind: "observed_at"
why_it_matters: "The report is a company-authored account of the collaboration. It does not provide the sampled prompts, an independently published AVERI report, quantitative results, or enough detail to assess how representative the sample was. The announcement therefore documents a process change and the company’s description of external feedback, not independently verifiable evidence that the revised evaluation"
novelty: "notable"
verification_level: "source-linked"
signal_type: "research"
source_platform: "amazon.science"
topics: []
entities: ["Amazon","Amazon Science","AVER I","Miles Brundage"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Amazon’s post adds its own approach to independent third-party AI safety evaluation, distinct from the prior OpenAI-focused discussion."
prior_refs: ["memory:e6980cfa-dd15-88a8-ad02-bc3a30281c9b","memory:6bc15a7f-c07b-87c0-a17f-f3d1f4288115"]
source_url: "https://www.amazon.science/blog/strengthening-ai-safety-evaluation-through-independent-third-party-review"
source_urls: ["https://www.amazon.science/blog/strengthening-ai-safety-evaluation-through-independent-third-party-review"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:strengthening-ai-safety-evaluation-through-independent-third-party-3348d0"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/strengthening-ai-safety-evaluation-through-independent-third-party-3348d0/","markdown":"https://newruntime.com/signals/strengthening-ai-safety-evaluation-through-independent-third-party-3348d0.md","json":"https://newruntime.com/signals/strengthening-ai-safety-evaluation-through-independent-third-party-3348d0.json"}
---

# Strengthening AI safety evaluation through independent third-party review

## Retrieval answer

Amazon says it partnered with the independent nonprofit AI Verification and Evaluation Research Institute (AVERI) to review a random sample of prompts from its internal safety-evaluation suite.

Amazon says it partnered with the independent nonprofit AI Verification and Evaluation Research Institute (AVERI) to review a random sample of prompts from its internal safety-evaluation suite. The review examined the quality of the tests themselves—including topical and linguistic coverage, diversity of formats, difficulty calibration, and methodological soundness—rather than assessing only model outputs. Amazon says AVERI conducted the review without input from Amazon’s evaluation team.

According to Amazon’s account, AVERI validated broad coverage of documented risk categories, varied testing approaches, and low contamination risk. It also identified gaps: multilingual prompts were not evenly distributed across risk categories; the suite needed more multistep adversarial prompts; some time-sensitive references had become stale; and some grading criteria could be clearer. Amazon says it responded by refreshing the dataset with harder scenarios, introducing tiered difficulty levels, expanding multilingual coverage across risk categories, and setting a quarterly review cadence.

The report is a company-authored account of the collaboration. It does not provide the sampled prompts, an independently published AVERI report, quantitative results, or enough detail to assess how representative the sample was. The announcement therefore documents a process change and the company’s description of external feedback, not independently verifiable evidence that the revised evaluations now predict safety in deployment.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)