---
type: "post"
id: "a-habr-author-published-a-hands-on-evaluation-404e56"
slug: "a-habr-author-published-a-hands-on-evaluation-404e56"
title: "A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0"
description: "A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0."
retrieval_nugget: "A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0."
published_at: "2026-10-23"
updated_at: "2026-10-23"
record_date: "2026-10-23"
date_kind: "published_at"
topics: ["coding-agents","evals"]
entities: ["Jev","jev-1.13.0","Typesafe","OpenAI","gpt-4.1-mini"]
related_patterns: ["verification-bandwidth-is-the-scarce-resource"]
novelty_label: "new_fact"
what_changed: "Новый масштабный практический тест Jev: 22 эксперимента и около 160 тыс. запросов к jev-1.13.0 и контрольным моделям, с проверкой точности, confidence, длинного контекста и устойчивости к грязному тексту."
prior_refs: ["memory:bdb8e41c-c608-8244-acfc-d194b4c0b224","memory:27b3f69c-5158-8819-a8e8-26c0265f06bc"]
source_url: "https://habr.com/ru/articles/1089164"
source_urls: ["https://habr.com/ru/articles/1089164"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:a-habr-author-published-a-hands-on-evaluation-404e56"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/a-habr-author-published-a-hands-on-evaluation-404e56/","markdown":"https://newruntime.com/posts/a-habr-author-published-a-hands-on-evaluation-404e56.md","json":"https://newruntime.com/posts/a-habr-author-published-a-hands-on-evaluation-404e56.json"}
---

# A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0

## Retrieval answer

A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0.

A Habr author published a hands-on evaluation of Typesafe's Jev API model, jev-1.13.0. The work ran from September 19 to 30, 2026, covered 22 experiments and about 160,000 requests to Jev and OpenAI control models, and cost about $18 through early-access API access.

Jev takes text plus structured questions and returns a selected answer, confidence value and probability distribution. The test found accuracy comparable to gpt-4.1-mini when both received class descriptions. The author reports Jev was 2.4 times faster and five times cheaper in that setup. Listed API pricing was $0.042 per million input tokens, with no output charge.

The evaluation also tested confidence thresholds, added classes, long and noisy context, languages, and multiple questions per input. At a confidence threshold above 0.9, 29% of Jev's errors and 83% of gpt-4.1-mini's errors fell above the threshold under the author's measurement method. Forty questions on one text were reported to take nearly the time of one question and cost up to 86% less than separate calls.

The source does not establish universal performance: results are tied to jev-1.13.0, selected datasets, synthetic examples in part, and the author's particular confidence calculation for GPT.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Coding Agents](https://newruntime.com/topics/coding-agents.md): Explore the coding-agents topic hub.
- topic hub: [Evals](https://newruntime.com/topics/evals.md): Explore the evals topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)