---
type: "post"
id: "toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375"
slug: "toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375"
title: "Toptal ExpertCrowd and HUD announce collaboration, release ExpertCrowd-SWE coding-agent benchmark"
description: "Toptal’s ExpertCrowd and HUD have released ExpertCrowd-SWE, a benchmark for evaluating frontier language models and coding agents on software-engineering work."
retrieval_nugget: "Toptal’s ExpertCrowd and HUD have released ExpertCrowd-SWE, a benchmark for evaluating frontier language models and coding agents on software-engineering work. It combines practitioner-authored tasks and rubrics with infrastructure for running complex reinforcement-learning environments, adding a new attempt to test agents against work framed as professional practice."
published_at: "2026-11-06"
updated_at: "2026-11-06"
record_date: "2026-11-06"
date_kind: "published_at"
topics: ["coding-agents"]
entities: ["Toptal","ExpertCrowd","HUD","ExpertCrowd-SWE","DataVendor"]
related_patterns: ["verification-bandwidth-is-the-scarce-resource"]
novelty_label: "new_fact"
what_changed: "ExpertCrowd and HUD announced a collaboration and released ExpertCrowd-SWE, a new software-engineering benchmark for evaluating coding agents."
prior_refs: ["https://newruntime.com/posts/cognition-launched-frontiercode-2-0-a-coding-benchmark-c6b49a/"]
source_url: "https://en.prnasia.com/releases/apac/expertcrowd-and-hud-collaborate-to-advance-ai-training-and-evaluation-release-swe-benchmark-555191.shtml"
source_urls: ["https://en.prnasia.com/releases/apac/expertcrowd-and-hud-collaborate-to-advance-ai-training-and-evaluation-release-swe-benchmark-555191.shtml"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375/","markdown":"https://newruntime.com/posts/toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375.md","json":"https://newruntime.com/posts/toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375.json"}
markdown_url: "https://newruntime.com/posts/toptal-expertcrowd-and-hud-announce-collaboration-release-expertcrowd-0e4375.md"
---

# Toptal ExpertCrowd and HUD announce collaboration, release ExpertCrowd-SWE coding-agent benchmark

## Retrieval answer

Toptal’s ExpertCrowd and HUD have released ExpertCrowd-SWE, a benchmark for evaluating frontier language models and coding agents on software-engineering work. It combines practitioner-authored tasks and rubrics with infrastructure for running complex reinforcement-learning environments, adding a new attempt to test agents against work framed as professional practice.

## Summary
Toptal’s ExpertCrowd and HUD have released ExpertCrowd-SWE, a benchmark for evaluating frontier language models and coding agents on software-engineering work. It combines practitioner-authored tasks and rubrics with infrastructure for running complex reinforcement-learning environments, adding a new attempt to test agents against work framed as professional practice.

## What happened
Toptal announced on November 6, 2026 that its ExpertCrowd platform and HUD are collaborating on AI training and evaluation. Their first release is ExpertCrowd-SWE, a software-engineering benchmark intended to evaluate frontier-grade large language models and coding agents. Toptal describes ExpertCrowd as a platform for training and improving AI models and systems; HUD describes itself as a platform for building reinforcement-learning environments and post-training data.

The announcement says the partners built the benchmark by dividing the work between domain experts and evaluation infrastructure. Engineers from Toptal’s global network authored the tasks and their grading rubrics. HUD hosted the environments in which the tasks run and verified the benchmark through its internal quality-assurance pipelines. The companies say this approach is meant to make tasks reflect professional practice and to make the grading criteria capture successful execution rather than a narrower capability measure.

Toptal says ExpertCrowd puts practitioners at the center of defining the work AI systems should perform and judging the resulting performance. HUD says its infrastructure lets research teams create realistic reinforcement-learning training environments, run evaluations across models, and inspect model performance. In the companies’ description, ExpertCrowd translates complex workflows into tasks, reference outputs, and evaluation frameworks, while HUD supplies the environment in which those tasks can be run.

The release says Toptal’s and HUD’s AI research teams reviewed and assessed the methodology and results. It characterizes those teams as including more than a dozen PhDs and researchers with educational or professional experience at Harvard, EPFL, the University of Liege, DeepMind, Qwen, and ByteDance. The announcement does not name the models evaluated, provide their scores, state a task count, list programming languages, or describe the detailed scoring procedure.

Toptal says ExpertCrowd is supported by a network of more than 30,000 specialized domain experts in more than 140 countries, including more than 2,000 AI and big-data specialists, 12,000 professionals with advanced degrees, and 1,000 PhD-level AI specialists. HUD says its DataVendor marketplace allows AI labs to license training and evaluation data directly from the companies that created it. Neither company states a price for ExpertCrowd-SWE or terms for using the benchmark. They call it the first release in an ongoing collaboration and say further initiatives are in development, without giving dates.

## What the archive adds
New Runtime’s earlier coverage of [Cognition’s FrontierCode 2.0 coding benchmark](https://newruntime.com/posts/cognition-launched-frontiercode-2-0-a-coding-benchmark-c6b49a/) on October 27, 2026 described expanded task scope and stricter mergeability criteria. ExpertCrowd-SWE is a new benchmark event rather than a restatement: Toptal and HUD emphasize expert-authored task design and hosted evaluation environments, but do not publish comparable performance details in their announcement.

## Trend signal
This release weakly supports [verification bandwidth is the scarce resource](https://newruntime.com/patterns/verification-bandwidth-is-the-scarce-resource/). Its premise is that evaluating a coding agent requires carefully defined work, reference expectations, and criteria for successful completion, not simply inspecting generated code. The announcement does not establish whether the benchmark predicts review queues, merge latency, or production outcomes, so the connection remains provisional.

## Why now
Toptal and HUD frame the release as a response to AI systems taking on increasingly complex work. Their stated argument is that measurement must move toward tasks resembling professional practice and grading that captures successful execution. The timing is therefore tied to the need for training and evaluation environments that can represent more involved software-engineering workflows.

## Why it matters
For teams building with coding agents, benchmark construction matters as much as a headline score. A task set can favor the kinds of repositories, tools, environments, and definitions of success that its designers choose. Expert-authored rubrics and runnable environments are a credible direction because they aim to assess completed work within an operating context rather than a model response in isolation.

But this release is not yet a purchasing or deployment guide. Engineering leaders cannot infer which model to select, how much automation is safe, or what review capacity will be required from the announcement alone. Task counts, model results, scoring rules, reproducibility, and evidence of correlation with production work would all be needed before a benchmark result could meaningfully inform those decisions. For now, the practical signal is that evaluation infrastructure is being organized around real-work definitions of completion.

## Limits and uncertainty
Toptal’s announcement does not identify evaluated models, publish scores, or state how many tasks ExpertCrowd-SWE contains. It does not describe public availability, reproducibility procedures, contamination controls, the benchmark’s scoring method, or access and pricing terms. The companies report internal quality assurance and methodological review, but no independent validation is described. There is also no stated evidence that performance on ExpertCrowd-SWE corresponds to accepted changes, review effort, reliability, or other production outcomes for software teams.

## Reader takeaway
ExpertCrowd-SWE is a new coding-agent benchmark effort built around expert task authorship, rubrics, and hosted execution environments. That design focus is relevant to teams assessing agents, but the release supplies no published results or methodology detailed enough for tool selection. Treat it as an evaluation-infrastructure signal, and wait for inspectable tasks and reproducible scores.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Coding Agents](https://newruntime.com/topics/coding-agents.md): Explore the coding-agents topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)