---
type: "post"
id: "securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b"
slug: "securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b"
title: "SecurityScorecard releases open TPRM Benchmark testing AI models on third-party risk work and attack resistance"
description: "SecurityScorecard has released version 2026.09 of an open benchmark for AI used in third-party risk management."
retrieval_nugget: "SecurityScorecard has released version 2026.09 of an open benchmark for AI used in third-party risk management. It matters because it tests two things that teams often separate: whether a model can assess vendor-risk work and whether untrusted vendor material can manipulate that model’s judgment."
published_at: "2026-10-30"
updated_at: "2026-10-30"
record_date: "2026-10-30"
date_kind: "published_at"
topics: ["agent-security","sandbox"]
entities: ["SecurityScorecard","TPRM Benchmark","Model Security suite","Snyk"]
related_patterns: ["agent-security-moves-to-runtime-boundaries"]
novelty_label: "new_fact"
what_changed: "SecurityScorecard introduces an open benchmark specifically for third-party risk work and attack resistance, with rotating inputs, protected answer keys, autonomy levels, and separate A–F security grades."
prior_refs: ["memory:eafee5c5-2ddb-8391-a262-f22b328de612","memory:7e08811f-4d47-8bf1-a59e-c960db3ce107"]
source_url: "https://siliconangle.com/2026/10/29/securityscorecard-benchmark-tests-ai-on-third-party-risk-work-and-resistance-to-attack"
source_urls: ["https://siliconangle.com/2026/10/29/securityscorecard-benchmark-tests-ai-on-third-party-risk-work-and-resistance-to-attack"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b/","markdown":"https://newruntime.com/posts/securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b.md","json":"https://newruntime.com/posts/securityscorecard-releases-open-tprm-benchmark-testing-ai-models-333d1b.json"}
---

# SecurityScorecard releases open TPRM Benchmark testing AI models on third-party risk work and attack resistance

## Retrieval answer

SecurityScorecard has released version 2026.09 of an open benchmark for AI used in third-party risk management. It matters because it tests two things that teams often separate: whether a model can assess vendor-risk work and whether untrusted vendor material can manipulate that model’s judgment.

## Summary
SecurityScorecard has released version 2026.09 of an open benchmark for AI used in third-party risk management. It matters because it tests two things that teams often separate: whether a model can assess vendor-risk work and whether untrusted vendor material can manipulate that model’s judgment.

## What happened
Security ratings company SecurityScorecard has released the TPRM Benchmark, an open framework for evaluating AI models and agents in third-party risk management. The benchmark is intended for vendor-risk programs, where systems may read security questionnaires, documents, and telemetry to decide which supplier exposures matter. Those inputs are unusually difficult to trust: vendors supply some of them themselves, and the material may be incomplete or deliberately misleading.

The capability portion covers tasks including exposure triage and fourth-party technology risk. Each task begins with a snapshot of SecurityScorecard’s vendor-risk telemetry that domain experts check before a model is evaluated. Answer keys are protected, and the inputs rotate, which SecurityScorecard says is designed to reduce the value of memorizing a static test set. Results are measured against a baseline with confidence intervals. Models also receive an autonomy-readiness level from L0 to L4, based on performance over repeated trials; the stated purpose is to indicate how far a model may be trusted to operate without human oversight.

A separate Model Security suite evaluates resistance to prompt injection, jailbreaks, and related attacks, assigning grades from A to F. Some capability tasks additionally include deceptive vendor self-attestations that conflict with the underlying telemetry. The scoring then shows how much those false claims reduce accuracy. This makes the benchmark an attempt to test a realistic failure mode for document-reading agents: not only an incorrect answer, but an answer redirected by the party whose risk is being assessed.

SecurityScorecard said early runs covered closed frontier, open-weight, and international models, but the announcement did not identify individual models or publish their scores. Its chief information security officer and chief AI officer, Mike Mylrea, said open-weight models are narrowing the gap with leading frontier models on supply-chain-security tasks, while several open-weight models remain highly susceptible to adversarial attacks. The public release includes documentation, the Model Security suite, and leaderboard registration. Teams can run their own models or agents and submit results. Those public-tier results are marked “synthetic-calibrated”; SecurityScorecard says they are comparable for research and self-assessment, while certified evaluation requires working directly with the company.

## What the archive adds
This is a new event in a related evaluation thread. [Our note on Amazon’s independent third-party review](https://newruntime.com/posts/strengthening-ai-safety-evaluation-through-independent-third-party-f7e046/) described scrutiny shifting from model outputs toward the safety tests used to judge them. SecurityScorecard extends that direction into vendor-risk workflows, pairing task performance with exposure to adversarial inputs.

## Trend signal
This weakly supports [Agent security moves to runtime boundaries](https://newruntime.com/patterns/agent-security-moves-to-runtime-boundaries/). The benchmark does not test a full agent runtime, but it makes clear that task capability alone is insufficient when an agent consumes untrusted material. A usable deployment needs controls and verification around the model, not simply a strong score on a business task.

## Why it matters
Teams building agents for procurement, security review, compliance, or supplier operations face a particular form of evaluation gap. A general benchmark may show that a model can summarize a questionnaire or classify an exposure, while saying little about whether the questionnaire can steer the model away from contradictory evidence. The TPRM Benchmark creates a more relevant evaluation shape: test the work, test the hostile input, and report uncertainty and repeated-trial performance.

That does not make its L0-to-L4 rating a deployment authorization. It does, however, give teams a public framework for distinguishing a model that appears competent in a clean demo from one that retains accuracy when a source document includes misleading claims. The practical value is greatest for systems that recommend, prioritize, or automate action from vendor-provided evidence.

## What happens next
- **Organizations will run candidate models or agents through the public tier and submit comparative results.** (3 months, 60%). Watch for: new public leaderboard entries.
- **SecurityScorecard will expand or revise the benchmark’s task and attack coverage with outside participants.** (6 months, 55%). Watch for: updated benchmark versions or contributor announcements.

## Limits and uncertainty
The announcement does not name the models used in early runs, disclose their scores, or provide enough detail to independently assess individual model claims. It also does not establish that synthetic-calibrated public-tier outcomes match a certified enterprise evaluation in every deployment. The benchmark focuses on third-party risk management rather than all agentic work, so its autonomy levels should not be generalized to unrelated domains. Finally, resistance to the disclosed attack categories is not proof that an agent’s surrounding permissions, tools, data handling, or human-review process are secure.

## Skill to build now
Create a small adversarial evaluation set for one document-driven workflow. Include an underlying source of record, a plausible self-attestation that contradicts it, and a clear expected decision. Run the same case repeatedly across candidate models or agent configurations, record accuracy and variance, and keep the evaluation separate from the model’s own explanation. This mirrors the useful part of the release: evaluate whether the system can preserve an evidence-based judgment when the input tries to influence it.

## Reader takeaway
For agents that assess vendors, the critical question is not only whether they understand risk data. It is whether they can keep that judgment anchored to trusted evidence when vendor-supplied text is incomplete, misleading, or adversarial. SecurityScorecard’s release offers an open starting point for testing both properties together.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Agent Security](https://newruntime.com/topics/agent-security.md): Explore the agent-security topic hub.
- topic hub: [Sandbox](https://newruntime.com/topics/sandbox.md): Explore the sandbox topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)