{"schema_version":"newruntime-tested-v1","type":"new-runtime-tested","updated_at":"2026-10-28","policy":"New Runtime runs the same task through several tools, checks every answer by hand and publishes the numbers. No sponsorship: no vendor paid for, saw or approved these results. Free tiers unless an entry states otherwise; each entry names its cost basis.","routes":{"html":"https://newruntime.com/tested/","json":"https://newruntime.com/tested.json"},"entries":[{"id":"search-apis-company-website-candidates-2026-10","date":"2026-10-28","published_at":"2026-10-28","title":"Five search APIs as a source of candidate websites for the same business records","category":"search-apis","tested_by":"New Runtime","cost_basis":"Free allowances on every tool: Serper's 2,500 free queries, Jina's 10,000,000 free tokens, You.com's $10 sign-up credits, Linkup's 4,000 free queries and SerpAPI's 250 free searches a month. Per-query prices in the table are the vendors' published list prices on the day of the test, not billed amounts.","disclosure":"No sponsorship; every run stayed inside a free allowance or sign-up credits. No vendor saw or approved these results.","task":"The same records as the first entry (a registered business name with a city and state), but a narrower question: when a plain web-search API is given one fixed query per record, how often does the business's verified official website appear in the first ten results, and how high? This measures each API as a source of candidates for a verifier, not as a finished answer.","result_columns":{"verified":"Official site in top 10","link_type_correct":"At rank 1 / in top 3","failures":"Misses and errors"},"method":{"sets":[{"name":"Reference set","n":55,"note":"Records from the first entry's batch whose official website one reviewer had verified by hand in that run. Every tool was also queried on the remaining records of the batch (100 queries per tool in total) for latency and cost, but only the 55 with a verified site are scored."}],"inputs":"One query per record built from one template: the registered name, the city and the state, with no quotes or operators. Ten results requested from every API.","judging":"Strict hit: a result whose host (minus www.) is the verified domain or a subdomain of it, counted at rank 1, within the top 3 and within the top 10 of the organic list. A second, looser count (in the misses column) credits a site of the same business under another domain, such as the parent company's corporate site, a product site or a domain alias, decided by hand after the run by the same reviewer. For SerpAPI the knowledge panel and local pack fields were also read, because its organic list often omitted a site that those fields carried.","repeats":"One call per record and tool, all five tools run in parallel within the same hour. Same query string, byte for byte, across tools."},"tools":[{"id":"serper","name":"Serper","vendor_url":"https://serper.dev/","tier_used":"free","mode":"Google search endpoint, 10 results, default location"},{"id":"jina-search","name":"Jina Search","vendor_url":"https://jina.ai/","tier_used":"free","mode":"s.jina.ai, JSON, page content suppressed"},{"id":"you-search","name":"You.com Web Search API","vendor_url":"https://you.com/","tier_used":"free","mode":"v1 search, count 10, snippets only"},{"id":"linkup-standard","name":"Linkup Search","vendor_url":"https://www.linkup.so/","tier_used":"free","mode":"depth standard, searchResults output, 10 results"},{"id":"serpapi-google","name":"SerpAPI","vendor_url":"https://serpapi.com/","tier_used":"free","mode":"engine google, US, English, 10 results"}],"results":[{"set":"Reference set (55)","tool":"serper","verified":"46/55","link_type_correct":"35/55 at rank 1; 40/55 in top 3","failures":"9 misses, 6 of which returned another site of the same business (52/55 loosely); 0 errors","cost":"100 of 2,500 free queries; then $1.00 per 1,000","latency":"median 1.7 s, p90 2.6 s"},{"set":"Reference set (55)","tool":"jina-search","verified":"47/55","link_type_correct":"38/55 at rank 1; 40/55 in top 3","failures":"8 misses, 6 of which returned another site of the same business (53/55 loosely); 0 errors","cost":"1,000,000 of 10,000,000 free tokens (a flat 10,000 tokens per call); then $0.05 per 1,000,000 tokens, about $0.50 per 1,000 calls","latency":"median 1.4 s, p90 2.2 s"},{"set":"Reference set (55)","tool":"you-search","verified":"36/55","link_type_correct":"14/55 at rank 1; 24/55 in top 3","failures":"19 misses, 7 of which returned another site of the same business (43/55 loosely); 0 errors","cost":"about $0.50 of the $10 sign-up credits; $5 per 1,000 calls","latency":"median 1.2 s, p90 1.6 s"},{"set":"Reference set (55)","tool":"linkup-standard","verified":"28/55","link_type_correct":"6/55 at rank 1; 12/55 in top 3","failures":"27 misses, 3 of which returned another site of the same business (31/55 loosely); 0 errors","cost":"100 of 4,000 free queries; then $5 to $6 per 1,000 standard searches","latency":"median 1.9 s, p90 2.8 s"},{"set":"Reference set (55)","tool":"serpapi-google","verified":"29/55 in the organic list; 40/55 counting the knowledge panel and local pack","link_type_correct":"19/55 at rank 1; 25/55 in top 3","failures":"26 organic misses; 11 of them had the site in the knowledge panel or local pack and 4 more another site of the same business (44/55 loosely); 0 errors, one call returned 9 results","cost":"100 of 250 free searches a month; the first paid plan is $25 per 1,000","latency":"median 1.2 s, p90 4.5 s"}],"impressions":[{"tool":"serper","pros":["Cheapest paid price per 1,000 queries here and the largest free allowance.","Returns the credits used with every response."],"cons":["Rank 1 was a directory, map or LinkedIn page on 20 of 55 records, so a verifier is still needed.","For brand and subsidiary records it returns the corporate parent's site, which may not be the site you want."]},{"tool":"jina-search","pros":["Best strict top-10 and rank-1 counts, and the cleanest JSON to consume.","The same key covers reading pages, so one vendor can do search and fetch."],"cons":["Charges a flat 10,000 tokens per search call even with page content suppressed, so the free allowance is 1,000 searches, not millions of tokens of reading.","Its lists were so close to Serper's (same first result on 78 of 100 queries) that it is not a second opinion."]},{"tool":"you-search","pros":["Fastest median here and never errored.","Snippets are longer than a Google SERP's, which helps an agent read without a second fetch."],"cons":["The official site led the list on only 14 of 55 records; map and directory sites usually came first.","No usage or cost field in the response; credits are visible only in the dashboard."]},{"tool":"linkup-standard","pros":["Every result carries a text extract, which the Google-based tools do not give at this price.","Largest free allowance after Serper."],"cons":["The weakest candidate source in this test: the official site was in the top ten on about half of the records and first on 6 of 55.","Leans towards people-data and company-directory sites for a business-name query."]},{"tool":"serpapi-google","pros":["Exposes the knowledge panel and local pack as structured fields; reading them lifted coverage from 29 to 40 of 55.","Rich result types beyond organic links, useful when the task is the SERP itself."],"cons":["Its organic list alone was the weakest of the Google-based tools and drifted to look-alike names on short business names.","The smallest free allowance (250 searches a month) and the highest list price here, 25 times Serper's."]}],"verdict":"As a source of candidate sites for your own verifier, a plain Google results API is the right building block: Serper and Jina put the verified site in the top ten on about 85% of records and first on roughly two thirds, at $0.50 to $1 per 1,000 queries. SerpAPI reaches similar coverage only if you also read its knowledge-panel and local-pack fields, at 25 times the price. You.com and Linkup are built for agents that read, not for entity lookup: they surfaced the verified site on 65% and 51% of records and rarely at rank one. None of the five replaces verification: on every tool a directory, map or data-broker page outranked the official site more often than not, and a shared miss across tools does not mean the business has no website.","scale_note":"At list prices, 1,000 queries cost about $0.50 on Jina, $1 on Serper, $5 on You.com, $5 to $6 on Linkup and $25 on SerpAPI. Every tool's free allowance covered this whole test.","caveats":["The reference comes from the first entry's hand check, and the same reviewer decided the same-business alternates after seeing the lists; this is a careful check, not a blind benchmark.","One query template and one run per record; another template (quoted name, no city, a site: operator) could reorder the tools.","Latency is the full round trip from one client on one afternoon; the vendors' own reported times were lower.","Google-based tools can return different layouts for the same query, so Serper, Jina and SerpAPI are not measuring the same page.","Prices and free allowances are those published on the day of the test and change often."],"url":"https://newruntime.com/tested/#search-apis-company-website-candidates-2026-10"},{"id":"open-models-nvidia-compact-triage-2026-10","date":"2026-10-28","published_at":"2026-10-28","title":"Open models on NVIDIA's free API, run on the newsroom's compact triage step and scored against the production operator","category":"model-apis","tested_by":"New Runtime","cost_basis":"NVIDIA's build.nvidia.com trial API: free to prototype, 40 requests a minute, no per-token billing and no production use. Every call in this entry was made on that trial, so the cost of the whole run was zero.","disclosure":"No sponsorship; NVIDIA did not see or approve these results. The models are third-party open weights that NVIDIA hosts; this entry measures them as served on the trial endpoint, not the weights themselves.","task":"The newsroom's compact triage step: given a batch of up to twenty stored material views (title, canonical URL, excerpt), return one JSON item per material with a decision (post, skip, watch or deep research), a content kind (news, release, research, profile, job, landing page, advertisement, other), a confidence, a reason code and a flag for deep analysis. In production this step runs on the newsroom's operator model (OpenAI GPT-5.6 through Codex), and its stored decisions are the reference here. The question is how far open models hosted for free get on exactly the same batches, under exactly the same prompt and schema contract.","result_columns":{"verified":"Same decision as the operator","link_type_correct":"Same post / not-post call","failures":"Batches rejected and kind agreement"},"method":{"sets":[{"name":"Production batches","n":314,"note":"Twenty intake batches from the last three weeks of production, 314 materials, replayed with the operator's stored decisions as the reference: post 174, skip 85, watch 36, deep research 19. Items from a batch that a model failed twice are not scored, so the scored count differs by model."}],"inputs":"The production classification prompt, byte for byte, with the same stored material views the operator saw. Temperature 0, 300 output tokens per item, one repair attempt after a schema failure, as in production. Reasoning was switched off where the model exposes a switch, because with it on the smaller models spent the whole output budget thinking and returned nothing.","judging":"Per material: the model's decision equals the operator's (four-way), and the model's post / not-post call equals the operator's (two-way). Per batch: accepted on the first try, accepted after one repair, or rejected by the same schema and cardinality checks production applies. Twenty-five materials on which most models disagreed with the operator were then re-read by one reviewer to decide who was right.","repeats":"One run per batch and model on one afternoon, three batches in flight at a time to stay under the trial's rate limit. Three further models in the account's callable list did not answer that day and are listed below as such."},"tools":[{"id":"nemotron-3-ultra","name":"NVIDIA Nemotron 3 Ultra (550B, 55B active)","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, reasoning on by default, temperature 0"},{"id":"minimax-m3-1-flash","name":"MiniMax M3.1 Flash","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, temperature 0"},{"id":"nemotron-3-5-bolt","name":"NVIDIA Nemotron 3.5 Bolt (4B)","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, thinking disabled, temperature 0"},{"id":"nemotron-3-5-lightning","name":"NVIDIA Nemotron 3.5 Lightning (30B, 3B active)","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, thinking disabled, temperature 0"},{"id":"glm-5-3-flash","name":"Z.ai GLM 5.3 Flash","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, temperature 0"},{"id":"step-5-preview","name":"StepFun Step 5 preview","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, temperature 0"},{"id":"gpt-oss-20b","name":"OpenAI gpt-oss-20b","vendor_url":"https://build.nvidia.com/","tier_used":"free","mode":"chat completions, reasoning effort low"}],"results":[{"set":"Production batches (20)","tool":"nemotron-3-ultra","verified":"221/314 (70%)","link_type_correct":"249/314 (79%); 33 extra posts, 32 missed posts","failures":"0 of 20 batches rejected (19 accepted first try, 1 after repair); content kind agreed on 221/314 (70%)","cost":"$0 on the trial; 140,000 input and 18,500 output tokens","latency":"median 20 s per batch, p90 35 s"},{"set":"Production batches (20)","tool":"minimax-m3-1-flash","verified":"180/265 (68%)","link_type_correct":"204/265 (77%); 40 extra posts, 21 missed posts","failures":"3 of 20 batches rejected (six empty responses with status 200, two invalid JSON, one miscounted batch); content kind agreed on 195/265 (74%)","cost":"$0 on the trial; 147,000 input and 111,000 output tokens, most of them reasoning","latency":"median 202 s per batch, p90 268 s"},{"set":"Production batches (20)","tool":"nemotron-3-5-bolt","verified":"129/224 (58%)","link_type_correct":"160/224 (71%); 50 extra posts, 14 missed posts","failures":"6 of 20 batches rejected, 14 of 20 first attempts dropped or added an item; content kind agreed on 91/224 (41%)","cost":"$0 on the trial; 195,000 input and 24,000 output tokens","latency":"median 18 s per batch, p90 23 s"},{"set":"Production batches (20)","tool":"nemotron-3-5-lightning","verified":"67/143 (47%)","link_type_correct":"79/143 (55%); 14 extra posts, 50 missed posts","failures":"9 of 20 batches rejected, almost all for dropping or adding an item; content kind agreed on 61/143 (43%)","cost":"$0 on the trial; 221,000 input and 42,000 output tokens","latency":"median 15 s per batch, p90 21 s"},{"set":"Production batches (20)","tool":"glm-5-3-flash","verified":"no answer","link_type_correct":"no answer","failures":"3 batches attempted, every call timed out at 300 s, including a one-line probe; run stopped","cost":"$0","latency":"none within 300 s"},{"set":"Production batches (20)","tool":"step-5-preview","verified":"no answer","link_type_correct":"no answer","failures":"3 batches attempted, every call timed out at 300 s; a one-line probe answered with empty content","cost":"$0","latency":"none within 300 s"},{"set":"Production batches (20)","tool":"gpt-oss-20b","verified":"not run","link_type_correct":"not run","failures":"a one-line probe timed out twice at 90 s on the test day, so no batch was sent","cost":"$0","latency":"none within 90 s"}],"impressions":[{"tool":"nemotron-3-ultra","pros":["The only model here that held the batch contract: every batch came back with the right number of items and valid JSON.","Errors are symmetric, so it is neither a spammer nor a censor; its post / not-post call matched the operator on four of five materials.","Twenty seconds for a batch of twenty, with reasoning on, is close to the operator's own speed."],"cons":["It promotes vendor marketing, pricing pages and third-party explainers to post where the operator skips them.","It almost never chooses deep research: one of nineteen cases, the rest became post or skip."]},{"tool":"minimax-m3-1-flash","pros":["Accuracy within two points of Nemotron Ultra on both measures and the best content-kind agreement of the four."],"cons":["Three to five minutes per batch and six empty responses with a success status: the slowest and least reliable of the models that answered.","Reasoning tokens outnumbered answer tokens five to one; there was no switch to turn them off."]},{"tool":"nemotron-3-5-bolt","pros":["Fastest answers and the lowest miss rate on real posts: it flagged 14 of the operator's posts as not post."],"cons":["It cannot count: fourteen first attempts returned one item too few or too many, which the schema check rejects.","Fifty extra posts, mostly materials the operator skipped as marketing or duplicates."]},{"tool":"nemotron-3-5-lightning","pros":["Fast and cheap on tokens."],"cons":["The weakest of the four: under half the batches accepted and it skipped 50 of the operator's posts.","Same counting problem as Bolt, plus one invalid reason code and one item with a missing key."]},{"tool":"glm-5-3-flash","pros":["Listed as callable for the account and answered on an earlier probe."],"cons":["Did not answer anything on the test day, which cost forty minutes of retries per batch before the run was stopped."]},{"tool":"step-5-preview","pros":["Answered an earlier short probe in under twenty seconds."],"cons":["Did not answer any production-size batch within 300 s."]},{"tool":"gpt-oss-20b","pros":["Exposes the reasoning-effort switch that production would need."],"cons":["Unreachable on the test day, so nothing could be measured."]}],"verdict":"On the newsroom's own triage prompt, Nemotron 3 Ultra served from NVIDIA's free trial reproduced the production operator's decision on 70% of materials and its post / not-post call on 79%, with every batch valid and about twenty seconds per batch. That is good enough to prototype a cheaper first pass, not to replace the operator: where the reviewer re-read the disagreements, the operator was right on 16 of 25, the open model on 2, and the rest were judgment calls. The open models' systematic errors are promoting vendor blogs, pricing pages and third-party explainers to post, and posting research and benchmark items the operator routes to deep research. The two Nemotron 3.5 models fail before judgment matters: they return a batch of twenty with nineteen or twenty-one items, which the contract rejects. MiniMax matches Ultra's accuracy but takes minutes per batch. Three of the seven callable models did not answer at all that day, which is the real shape of a free trial endpoint.","scale_note":"On the trial the cost is zero and the ceiling is 40 requests a minute: the twenty production batches took about seven minutes of wall time for Nemotron Ultra. The trial excludes production use; the same open weights are sold as downloadable containers for your own GPUs and are hosted elsewhere at per-token prices.","caveats":["The reference is the production operator's stored output, not ground truth; the hand re-read covered only the 25 materials where most models disagreed, and the same reviewer wrote the production prompt.","The prompt was tuned for the operator over months; an open model with its own prompt and examples could score higher.","Reasoning was disabled on the Nemotron 3.5 models; with it on and a larger output budget their accuracy might differ.","One run on one afternoon: three models were unreachable that day and may answer on another, and the trial's model list changes without notice.","The trial terms page could not be retrieved on the test day; these are agreement rates with our own operator on our own task, not a model benchmark, and the numbers say nothing about the models on other tasks."],"url":"https://newruntime.com/tested/#open-models-nvidia-compact-triage-2026-10"},{"id":"company-website-finder-2026-10","date":"2026-10-20","published_at":"2026-10-27","title":"Finding a company's official website from its name: six research agents compared","category":"research-agents","tested_by":"New Runtime","cost_basis":"Paid API usage for Parallel and Exa (Parallel costs are estimates from its published per-task prices, Exa costs are the amounts its API returned); Firecrawl Agent ran inside its free daily agent runs; the Codex check ran on a ChatGPT subscription and its cost was not measured.","disclosure":"No sponsorship; free tiers unless stated. This entry used paid API usage for Parallel and Exa.","task":"Given a business record (registered name, optionally a street address and a few registry fields), return the official website, the type of link between the site and the business (the business itself, one of its brands, or its parent company) and evidence pages. A site counts only when the evidence ties it to that exact business; a manufacturer of the same product or a similarly named firm does not count.","method":{"sets":[{"name":"Diagnostic set","n":10,"note":"Chosen on purpose to include ambiguous cases (brands, parents, renamed firms)."},{"name":"Hard cases","n":3,"note":"The three records no mode solved on the diagnostic set, rerun with the deepest modes."},{"name":"Residual set","n":45,"note":"Records the cheapest mode left without an accepted site in a larger batch."},{"name":"Input-field test","n":10,"note":"The diagnostic set three times on the cheapest mode: name only, name plus address, all fields."}],"inputs":"Business records with a name and, depending on the run, an address and registry fields.","judging":"One reviewer checked each returned site against its evidence pages and our own extra fetches, and accepted only links proven to the exact business. Not a blind gold standard: the reviewer saw the answers.","repeats":"One run per record and mode. The instruction and the JSON answer schema were identical, byte for byte, across tools."},"tools":[{"id":"parallel-base","name":"Parallel Task API, Base","vendor_url":"https://parallel.ai/","tier_used":"paid","mode":"processor base"},{"id":"parallel-core","name":"Parallel Task API, Core","vendor_url":"https://parallel.ai/","tier_used":"paid","mode":"processor core"},{"id":"parallel-pro","name":"Parallel Task API, Pro","vendor_url":"https://parallel.ai/","tier_used":"paid","mode":"processor pro"},{"id":"parallel-ultra8x","name":"Parallel Task API, Ultra8x","vendor_url":"https://parallel.ai/","tier_used":"paid","mode":"processor ultra8x (hard cases only)"},{"id":"parallel-search","name":"Parallel Search API","vendor_url":"https://parallel.ai/","tier_used":"paid","mode":"search, as a candidate source (residual set only)"},{"id":"exa-low","name":"Exa Agent, Low","vendor_url":"https://exa.ai/","tier_used":"paid","mode":"effort low"},{"id":"exa-medium","name":"Exa Agent, Medium","vendor_url":"https://exa.ai/","tier_used":"paid","mode":"effort medium"},{"id":"firecrawl-agent","name":"Firecrawl Agent","vendor_url":"https://www.firecrawl.dev/","tier_used":"free","mode":"spark-2, effort medium, cap 100 credits per record (hard cases only)"},{"id":"codex-search","name":"Codex built-in web search","vendor_url":"https://openai.com/codex/","tier_used":"paid","mode":"built-in search and page reading, no external research API (diagnostic set only)"}],"results":[{"set":"Diagnostic set (10)","tool":"parallel-base","verified":"7/10","link_type_correct":"7/7","failures":"3 not found","cost":"about $0.10","latency":"median 82 s per record"},{"set":"Diagnostic set (10)","tool":"parallel-core","verified":"7/10","link_type_correct":"5/7","failures":"3 not found; once left an unrelated domain filled in while saying not found","cost":"about $0.25","latency":"median 105 s per record"},{"set":"Diagnostic set (10)","tool":"parallel-pro","verified":"7/10","link_type_correct":"7/7","failures":"3 not found","cost":"about $1.00","latency":"not compared"},{"set":"Diagnostic set (10)","tool":"exa-low","verified":"7/10","link_type_correct":"5/7","failures":"3 not found","cost":"$0.25","latency":"median 25 s per record"},{"set":"Diagnostic set (10)","tool":"exa-medium","verified":"7/10","link_type_correct":"5/7","failures":"3 not found","cost":"$1.00","latency":"not compared"},{"set":"Diagnostic set (10)","tool":"codex-search","verified":"7/10","link_type_correct":"not scored","failures":"3 not found","cost":"not measured (subscription)","latency":"not measured"},{"set":"Hard cases (3)","tool":"parallel-ultra8x","verified":"0/3","link_type_correct":"n/a","failures":"3 not found, no wrong site","cost":"about $7.20","latency":"not compared"},{"set":"Hard cases (3)","tool":"firecrawl-agent","verified":"0/3","link_type_correct":"n/a","failures":"1 'resolved' we rejected (a product maker, not the business) and 2 'ambiguous'","cost":"0 credits charged (free daily agent runs); 103 credits of work in its execution traces","latency":"not compared"},{"set":"Residual set (45)","tool":"parallel-core","verified":"3 new of 45","link_type_correct":"n/a","failures":"15 not found, 3 ambiguous","cost":"about $1.10","latency":"not compared"},{"set":"Residual set (45)","tool":"parallel-pro","verified":"no new site beyond Core","link_type_correct":"n/a","failures":"21 not found","cost":"about $4.50","latency":"not compared"},{"set":"Residual set (45)","tool":"exa-medium","verified":"1 new of 45 that no other mode found","link_type_correct":"n/a","failures":"15 not found, 1 ambiguous","cost":"$4.50","latency":"not compared"},{"set":"Residual set (45)","tool":"parallel-search","verified":"0 new","link_type_correct":"n/a","failures":"mostly business directories and aggregators","cost":"not returned","latency":"not compared"},{"set":"Input-field test (3 x 10)","tool":"parallel-base","verified":"7/10 with name only, 7/10 with address, 7/10 with all fields","link_type_correct":"6/7, 7/7, 7/7","failures":"one unsupported 'resolved' in the name-plus-address variant","cost":"about $0.10 per variant","latency":"not compared"}],"impressions":[{"tool":"parallel-base","pros":["Cheapest mode with the same coverage as every deeper mode on the diagnostic set.","Named the type of link correctly on all seven finds."],"cons":["The raw output lacked our record ids, and a failed record came back as empty strings instead of null."]},{"tool":"parallel-core","pros":["Best value as a second pass: 3 of the 4 new sites on the residual set for about $1.10."],"cons":["Once filled in an unrelated domain while reporting not found, so the status alone cannot be trusted."]},{"tool":"parallel-pro","pros":["Precise link types, no stray candidates."],"cons":["Four times the cost of Core for no extra coverage in any set."]},{"tool":"parallel-ultra8x","pros":["Careful refusals: it separated look-alike firms instead of guessing."],"cons":["About $2.40 per record and no new site on the hard cases."]},{"tool":"exa-low","pros":["Fastest by far (median 25 s) and returns its own cost with each answer."],"cons":["Less precise about whether a site is the business, a brand or a parent."]},{"tool":"exa-medium","pros":["Stronger evidence pages on several records; the only mode to find one site on the residual set."],"cons":["Same price as Parallel Pro; still mislabelled some link types."]},{"tool":"firecrawl-agent","pros":["Read pages that other tools' fetches came back empty on.","Ran inside the free daily agent runs."],"cons":["Returned one confident 'resolved' that did not hold up.","Its credit reporting needs care: 0 credits charged against 103 credits of work in the traces."]},{"tool":"codex-search","pros":["Matched the paid agents on the diagnostic set without any external research API."],"cons":["Not a blind test: the session already knew earlier answers, so treat it as a reproduction, not a score."]},{"tool":"parallel-search","pros":["Confirmed already known domains."],"cons":["Returned mostly directories and aggregators; added nothing as a source of candidates."]}],"verdict":"For bulk website finding, start with the cheapest agent mode plus your own verifier, rerun only the misses on a mid mode, and keep a second vendor's mid mode for what is still missing. The deepest modes (Parallel Pro, Exa Medium, Parallel Ultra8x) cost 10 to 24 times more per record than the cheapest and added one site in total that a cheaper pass had missed. Never accept a 'resolved' status without checking that the evidence names the exact business, and do not read a shared failure across tools as proof that no website exists.","scale_note":"At the per-task prices used here, a first pass over 1,000 records would cost about $10 on Parallel Base, $25 on Parallel Core or Exa Low, $100 on Parallel Pro or Exa Medium and $2,400 on Parallel Ultra8x, before verification and retries.","caveats":["Small sets and one run per record: differences of one or two records are noise.","One instruction and one kind of record; other record types may rank the tools differently.","The reviewer saw the answers, so this is a careful check, not a blind benchmark.","Parallel costs are estimates from published prices, not billed amounts."],"url":"https://newruntime.com/tested/#company-website-finder-2026-10"}]}