---
type: analysis
slug: independent-tests-put-minimax-s-agent-claims-on-a-narrower-footing
title: "Independent tests put MiniMax’s agent claims on a narrower footing"
description: "MiniMax’s open weights make M3.1-Flash easier to inspect, but its headline scores are not the whole story: Vals AI’s mixed independent results complicate the company’s performance narrative without settling how the model will fare on your tasks."
published_at: 2026-10-26T16:00:00.000Z
topics: ["trends"]
source_urls: ["https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash","https://www.minimax.io/blog/minimax-m31f","https://siliconangle.com/2026/10/20/deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services","https://siliconangle.com/2026/10/18/deepseek-releases-updated-v4-1-flash-model-it-says-completes-tasks-faster","https://minimax.io/blog/minimax-m31f","https://vercel.com/changelog/minimax-m3-1-flash-now-available-on-ai-gateway-for-free"]
author: "Andrey Reshetnikov"
author_profile: https://newruntime.com/owner-profile.md
---
# Independent tests put MiniMax’s agent claims on a narrower footing

MiniMax’s open weights make M3.1-Flash easier to inspect, but its headline scores are not the whole story: Vals AI’s mixed independent results complicate the company’s performance narrative without settling how the model will fare on your tasks.

## Summary

MiniMax has released M3.1-Flash weights under its Community License, describing a multimodal model for coding and agent workflows with up to one million tokens of context. Its model card reports 76.2 on BankerToolBench and 73.7% on DeepSWE v1.1. In contrast, Vals AI’s independent tests reported mixed results, including [exact scores and ranking unavailable in the supplied evidence]. That disagreement is a reason to reproduce the comparisons, not a verdict on performance for any particular workload.

## What happened

The model card says M3.1-Flash accepts text, image, and video inputs and supports up to one million tokens of context. MiniMax says it builds on MiniMax-M3 and uses sparse attention, quantization-aware training, and DSpark speculative decoding. The weights are available on Hugging Face under the MiniMax Community License, allowing inspection and self-hosted evaluation subject to its terms. MiniMax reports 76.2 on BankerToolBench and 73.7% on DeepSWE v1.1; those are company-reported results. The editor points to Vals AI’s mixed independent results as counterevidence, but the supplied evidence pack and dossier contain no Vals scores or ranking. The evidence therefore supports a discrepancy in the performance narrative, not a precise comparison. It also does not specify deployment hardware, self-hosted cost, or performance under particular quantization settings.

## What the archive adds

The archive records MiniMax’s earlier announcement of API, M Plan, and MiniMax Code availability, with company-reported coding and computer-use results and token prices. That hosted route gives builders a baseline to compare with the newly available weights, but does not prove either option is cheaper for a given workload. The dossier’s vLLM finding says long, multi-turn agent sessions reuse context and can branch, stressing inference systems. This matters when reproducing agent results: task success alone will not capture serving behavior. Neither the archive nor dossier supplies the Vals AI scores or ranking cited in the editor’s critique, so those particulars cannot responsibly be stated here.

## Trend signal

Agent-model claims increasingly meet public or independent evaluation, but a benchmark result is only as useful as its task design and reproduction conditions. MiniMax’s release puts weights in builders’ hands; the mixed results attributed to Vals AI raise a sharper question than whether open weights are available: how robust are the headline scores across evaluators and workloads? The dossier’s account of agent-serving demands adds another layer: even a model that performs well on a test may impose different latency and operational costs in long, branching sessions.

## Why now

According to MiniMax’s model card and announcement, the release is framed as a capability and inference-efficiency upgrade, encouraging evaluation beyond hosted access. According to the dossier’s vLLM finding, long, multi-turn agent sessions with context reuse and branching create specialized serving pressures. Together, these factors make reproducible, workload-specific evaluation more relevant than a single headline score; the evidence does not establish whether cost or capability is the stronger market driver.

## Connections to other events

The important signal is not that Vals AI disproves MiniMax’s claims; the available evidence does not establish that. It is that MiniMax’s reported 76.2 on BankerToolBench and 73.7% on DeepSWE v1.1 should be read alongside the mixed independent results, not as a settled measure of agent capability. Because the supplied pack omits Vals AI’s exact scores and ranking, the gap cannot be quantified here. Our read is that the release makes reproduction more feasible, while the disagreement makes it more necessary: teams should compare methods and task-level outcomes before treating either scorecard as predictive.

## Why it matters

Builders should reproduce the published comparison as closely as possible, then extend it to representative internal tasks. Record the exact model version, prompts, harness, evaluation criteria, and serving setup so differences are interpretable. Buyers should distinguish benchmark quality from deployment suitability: the evidence does not establish self-hosted hardware needs, total cost, or commercial fit under the Community License. The open weights support hands-on evaluation, but do not remove the need to validate performance or plan for inference operations.

## What happens next

- **A public comparison will include M3.1-Flash alongside the independent Vals AI results and make its evaluation setup explicit.** (6 months, 50%). Watch for: A third-party report names the model version and method and publishes side-by-side task results.
- **Public self-hosting guidance for M3.1-Flash will document at least one concrete hardware and quantization setup, reducing uncertainty about practical deployment conditions.** (6 months, 50%). Watch for: A reproducible guide or report states hardware, quantization settings, and measured inference performance.

## Limits and uncertainty

The supplied evidence does not give Vals AI’s exact scores, ranking, test protocol, or the model version it evaluated, so this article cannot establish how large the disagreement is or why it arose. MiniMax’s scores are company-reported, and neither scorecard proves results on a reader’s own workload. The pack also lacks measured self-hosting costs, hardware requirements, and quantization-specific performance. If Vals tested a substantially different setup, the mixed results may not be directly comparable.

## Skill to build now

Build a reproducible benchmark comparison. This week, retrieve Vals AI’s original test, note its exact scores, ranking, model version, and harness, then run the same task set against M3.1-Flash and your current baseline; log failures and human interventions as well as pass rates.

## Reader takeaway

This week, reproduce one published agent or coding test against M3.1-Flash and your current baseline. Before running it, retrieve Vals AI’s original report and record its exact scores, ranking, model version, and method; then keep those conditions distinct from your own workload test. Track task success and human intervention, and check the Community License before any commercial trial.

## Open question

Do Vals AI’s mixed results reflect a real weakness in M3.1-Flash, or differences in model version, harness, and task selection that matter less on a team’s own workload?

## Sources

- [MiniMax releases open weights for MiniMax-M3.1-Flash, a 1M-context multimodal MoE model for agents and coding](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash)

- [MiniMax M3.1 Flash goes toe-to-toe with Opus and GPT on coding at ~25x lower cost.](https://www.minimax.io/blog/minimax-m31f)

- [DeepSeek open-sources DeepInfer, the inference framework behind its production services](https://siliconangle.com/2026/10/20/deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services)

- [DeepSeek releases updated V4.1-Flash model it says completes tasks faster](https://siliconangle.com/2026/10/18/deepseek-releases-updated-v4-1-flash-model-it-says-completes-tasks-faster)

- [minimax.io](https://minimax.io/blog/minimax-m31f)

- [MiniMax M3.1 Flash now available on AI Gateway for free - Vercel](https://vercel.com/changelog/minimax-m3-1-flash-now-available-on-ai-gateway-for-free)

- [MiniMax M3.1 Flash: Redefining Agent Performance and Cost — MiniMax Research](https://www.minimax.io/blog/minimax-m31f?trk=public_post_comment-text)

- [DeepInfer: High-performance, large-scale inference framework for DeepSeek models — DeepSeek-AI GitHub](https://github.com/deepseek-ai/DeepInfer)

- [vLLM x AgentX: Optimizing for Real-World Agentic Serving — vLLM Project](https://vllm-project.github.io/2026/09/08/vllm-agentx.html)

- [MiniMax-M3.1-Flash Benchmarks, Cost and Capabilities — Vals AI](https://www.vals.ai/models/minimax_MiniMax-M3.1-flash)

- [DeepSeek-V4.1-Flash-1017 — Hugging Face model card](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017)

- [DeepSeek V4.1 Flash Inference Deployment Q&A](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md)

- [Benchmark evaluation](https://github.com/deepseek-ai/DeepInfer/blob/main/eval/README.md)

Author: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Trends](https://newruntime.com/topics/trends.md): Explore the trends topic hub.
- collection: [Analysis](https://newruntime.com/analysis.json): Open the latest authorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)