# ProofBench v1.1 / overall

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall.

Automated theorem proving benchmark

Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall
Last updated: Wed, 21 Oct 2026 18:05:10 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Conquered

Automated theorem proving benchmark

Mathematics · Models tested: 34 · Score publishers: Vals AI

## Model results

| # | Model | Score | ± | Cost | Latency |  |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 100% | SE ±0 pp | $2.7 | 478.3source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.70347 \| USD \| per_test \| Lower is better \| \| `latency` \| 478.303 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-fable-5-1` |  |  |  |  |  |  |
| 1 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 100% | SE ±0 pp | $0.56 | 347.58source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.558327 \| USD \| per_test \| Lower is better \| \| `latency` \| 347.583 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals {"cache_read_tokens":33854803,"cache_write_tokens":4352571,"input_tokens":38210470,"output_tokens":3817407,"reasoning_tokens":2302419} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` |  |  |  |  |  |  |
| 1 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 100% | SE ±0 pp | $0.96 | 351.04source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.961648 \| USD \| per_test \| Lower is better \| \| `latency` \| 351.035 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals {"cache_read_tokens":18792754,"cache_write_tokens":3946449,"input_tokens":22741291,"output_tokens":3445356,"reasoning_tokens":2381878} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-opus-5-5` |  |  |  |  |  |  |
| 4 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 99% | SE ±1 pp | $1.71 | 307.87source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.71238 \| USD \| per_test \| Lower is better \| \| `latency` \| 307.867 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1 pp sourcePopulation 52 stderr 1 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-6-astra` |  |  |  |  |  |  |
| 4 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 99% | SE ±1 pp | $0.78 | 380.25source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.782978 \| USD \| per_test \| Lower is better \| \| `latency` \| 380.246 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1 pp sourcePopulation 52 stderr 1 tokenTotals {"cache_read_tokens":27510930,"cache_write_tokens":null,"input_tokens":37238434,"output_tokens":1694278,"reasoning_tokens":702938} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/google~1gemini-4-argon` |  |  |  |  |  |  |
| 4 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 99% | SE ±1 pp | $0.23 | 284.7source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.233057 \| USD \| per_test \| Lower is better \| \| `latency` \| 284.697 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1 pp sourcePopulation 52 stderr 1 tokenTotals {"cache_read_tokens":28854579,"cache_write_tokens":3985465,"input_tokens":32842768,"output_tokens":1038054,"reasoning_tokens":321606} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |
| 4 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 99% | SE ±1 pp | $2.11 | 590.89source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.10619 \| USD \| per_test \| Lower is better \| \| `latency` \| 590.886 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1 pp sourcePopulation 52 stderr 1 tokenTotals {"cache_read_tokens":45129298,"cache_write_tokens":5887376,"input_tokens":51018430,"output_tokens":4769719,"reasoning_tokens":3301232} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-opus-5` |  |  |  |  |  |  |
| 8 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 95% | SE ±2.19 pp | $5.45 | 822.74source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.451518 \| USD \| per_test \| Lower is better \| \| `latency` \| 822.739 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.19 pp sourcePopulation 52 stderr 2.19 tokenTotals {"cache_read_tokens":24973984,"cache_write_tokens":10966348,"input_tokens":35941254,"output_tokens":7112728,"reasoning_tokens":5678640} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-fable-5` |  |  |  |  |  |  |
| 9 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 89% | SE ±3.15 pp | $0.17 | 418.05source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.173874 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.047 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.15 pp sourcePopulation 52 stderr 3.145 tokenTotals {"cache_read_tokens":29799415,"cache_write_tokens":12500492,"input_tokens":42301619,"output_tokens":9672099,"reasoning_tokens":8496068} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-haiku-5-5` |  |  |  |  |  |  |
| 10 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 84% | SE ±3.69 pp | $1.31 | 929.05source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.313345 \| USD \| per_test \| Lower is better \| \| `latency` \| 929.053 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.69 pp sourcePopulation 52 stderr 3.685 tokenTotals {"cache_read_tokens":72750544,"cache_write_tokens":0,"input_tokens":83508776,"output_tokens":3687574,"reasoning_tokens":2180479} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/meta~1muse-wonder` |  |  |  |  |  |  |
| 11 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 83% | SE ±3.78 pp | $1.44 | 520.41source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.438313 \| USD \| per_test \| Lower is better \| \| `latency` \| 520.405 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.78 pp sourcePopulation 52 stderr 3.775 tokenTotals {"cache_read_tokens":115350727,"cache_write_tokens":null,"input_tokens":123160663,"output_tokens":1522264,"reasoning_tokens":541932} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-5.6-sol` |  |  |  |  |  |  |
| 11 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 83% | SE ±3.78 pp | $0.48 | 304.51source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.476321 \| USD \| per_test \| Lower is better \| \| `latency` \| 304.509 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.78 pp sourcePopulation 52 stderr 3.775 tokenTotals {"cache_read_tokens":92867287,"cache_write_tokens":6166643,"input_tokens":99039756,"output_tokens":1363035,"reasoning_tokens":593322} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |
| 13 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 77% | SE ±4.23 pp | $1.37 | 898.08source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.373684 \| USD \| per_test \| Lower is better \| \| `latency` \| 898.079 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.23 pp sourcePopulation 52 stderr 4.23 tokenTotals {"cache_read_tokens":145688844,"cache_write_tokens":12459810,"input_tokens":158153304,"output_tokens":7432445,"reasoning_tokens":5551345} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/anthropic~1claude-sonnet-5` |  |  |  |  |  |  |
| 14 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 74% | SE ±4.41 pp | $1.27 | 777.34source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.274209 \| USD \| per_test \| Lower is better \| \| `latency` \| 777.341 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.41 pp sourcePopulation 52 stderr 4.408 tokenTotals {"cache_read_tokens":222728984,"cache_write_tokens":null,"input_tokens":241131910,"output_tokens":1542468,"reasoning_tokens":704967} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-5.6-terra` |  |  |  |  |  |  |
| 15 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 70% | SE ±4.61 pp | $0.25 | 3,693.47source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.24535 \| USD \| per_test \| Lower is better \| \| `latency` \| 3693.469 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.61 pp sourcePopulation 52 stderr 4.606 tokenTotals {"cache_read_tokens":192932736,"cache_write_tokens":null,"input_tokens":214175878,"output_tokens":15653193,"reasoning_tokens":13784976} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-pro` |  |  |  |  |  |  |
| 16 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 64% | SE ±4.82 pp | $0.04 | 451.48source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.041637 \| USD \| per_test \| Lower is better \| \| `latency` \| 451.481 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.82 pp sourcePopulation 52 stderr 4.824 tokenTotals {"cache_read_tokens":117050095,"cache_write_tokens":8996692,"input_tokens":126053768,"output_tokens":2652752,"reasoning_tokens":1912396} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-6-luna` |  |  |  |  |  |  |
| 17 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 63% | SE ±4.85 pp | $0.12 | 3,769.81source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.123277 \| USD \| per_test \| Lower is better \| \| `latency` \| 3769.808 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.85 pp sourcePopulation 52 stderr 4.852 tokenTotals {"cache_read_tokens":365944512,"cache_write_tokens":null,"input_tokens":398251867,"output_tokens":24214511,"reasoning_tokens":20726471} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-flash` |  |  |  |  |  |  |
| 18 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 60% | SE ±4.92 pp | $0.12 | 513.69source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.123288 \| USD \| per_test \| Lower is better \| \| `latency` \| 513.687 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.92 pp sourcePopulation 52 stderr 4.924 tokenTotals {"cache_read_tokens":184422820,"cache_write_tokens":null,"input_tokens":203002670,"output_tokens":1885904,"reasoning_tokens":1035104} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/openai~1gpt-5.6-luna` |  |  |  |  |  |  |
| 19 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 58% | SE ±4.96 pp | $0.56 | 423.7source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.558508 \| USD \| per_test \| Lower is better \| \| `latency` \| 423.698 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.96 pp sourcePopulation 52 stderr 4.96 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/google~1gemini-3.7-flash` |  |  |  |  |  |  |
| 20 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 56% | SE ±4.99 pp | $0.04 | 933.98source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.036568 \| USD \| per_test \| Lower is better \| \| `latency` \| 933.981 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.99 pp sourcePopulation 52 stderr 4.989 tokenTotals {"cache_read_tokens":160683264,"cache_write_tokens":null,"input_tokens":168698684,"output_tokens":7445555,"reasoning_tokens":5646846} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` |  |  |  |  |  |  |
| 21 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 55% | SE ±5 pp | $0.43 | 534.06source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.433712 \| USD \| per_test \| Lower is better \| \| `latency` \| 534.06 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5 pp sourcePopulation 52 stderr 5 tokenTotals {"cache_read_tokens":96013349,"cache_write_tokens":0,"input_tokens":106926796,"output_tokens":3606435,"reasoning_tokens":2473887} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/meta~1muse_spark_1_3` |  |  |  |  |  |  |
| 22 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 54% | SE ±5.01 pp | $0.13 | 546.6source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.126437 \| USD \| per_test \| Lower is better \| \| `latency` \| 546.595 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.01 pp sourcePopulation 52 stderr 5.009 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` |  |  |  |  |  |  |
| 23 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 50% | SE ±5.03 pp | $0.06 | 732.37source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.062191 \| USD \| per_test \| Lower is better \| \| `latency` \| 732.365 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.03 pp sourcePopulation 52 stderr 5.025 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro-0813` |  |  |  |  |  |  |
| 24 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 48% | SE ±5.02 pp | $0.6 | 368.65source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.602872 \| USD \| per_test \| Lower is better \| \| `latency` \| 368.648 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.02 pp sourcePopulation 52 stderr 5.021 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/google~1gemini-3.8-flash` |  |  |  |  |  |  |
| 25 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 43% | SE ±4.98 pp | $0.44 | 469.15source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.442007 \| USD \| per_test \| Lower is better \| \| `latency` \| 469.146 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.98 pp sourcePopulation 52 stderr 4.976 tokenTotals {"cache_read_tokens":94642678,"cache_write_tokens":null,"input_tokens":104594507,"output_tokens":3716809,"reasoning_tokens":2515620} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/meta~1muse_spark_1_2` |  |  |  |  |  |  |
| 26 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 42% | SE ±4.96 pp | $0.71 | 2,313.59source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.714564 \| USD \| per_test \| Lower is better \| \| `latency` \| 2313.592 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.96 pp sourcePopulation 52 stderr 4.96 tokenTotals {"cache_read_tokens":234817792,"cache_write_tokens":null,"input_tokens":258573571,"output_tokens":13318410,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/stepfun~1step-5-preview` |  |  |  |  |  |  |
| 27 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 31% | SE ±4.65 pp | $0.57 | 394.48source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.569863 \| USD \| per_test \| Lower is better \| \| `latency` \| 394.476 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.65 pp sourcePopulation 52 stderr 4.648 tokenTotals {"cache_read_tokens":42908380,"cache_write_tokens":null,"input_tokens":66676489,"output_tokens":1338730,"reasoning_tokens":805865} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |
| 28 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 26% | SE ±4.41 pp | $0.78 | 889.06source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.784253 \| USD \| per_test \| Lower is better \| \| `latency` \| 889.059 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.41 pp sourcePopulation 52 stderr 4.408 tokenTotals {"cache_read_tokens":47066256,"cache_write_tokens":null,"input_tokens":68260388,"output_tokens":2100047,"reasoning_tokens":691664} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/google~1gemini-3.1-pro-preview` |  |  |  |  |  |  |
| 29 | [MiMo-V2.5-Pro](https://benchize.com/models/mimo-v2-5-pro.md) | 22% | SE ±4.16 pp | $0.09 | 1,177.35source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.089853 \| USD \| per_test \| Lower is better \| \| `latency` \| 1177.352 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.16 pp sourcePopulation 52 stderr 4.163 tokenTotals {"cache_read_tokens":100176384,"cache_write_tokens":null,"input_tokens":109400066,"output_tokens":5301519,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label xiaomi/mimo-v2.5-pro Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/xiaomi~1mimo-v2.5-pro` |  |  |  |  |  |  |
| 30 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 18% | SE ±3.86 pp | $0.42 | 718.66source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.419224 \| USD \| per_test \| Lower is better \| \| `latency` \| 718.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.86 pp sourcePopulation 52 stderr 3.861 tokenTotals {"cache_read_tokens":151053885,"cache_write_tokens":0,"input_tokens":162739395,"output_tokens":4876274,"reasoning_tokens":1498488} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/minimax~1MiniMax-M3` |  |  |  |  |  |  |
| 31 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-max.md)Max | 16% | SE ±3.69 pp | $0.03 | 423.45source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.033594 \| USD \| per_test \| Lower is better \| \| `latency` \| 423.45 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.69 pp sourcePopulation 52 stderr 3.685 tokenTotals {"cache_read_tokens":57067520,"cache_write_tokens":null,"input_tokens":61091499,"output_tokens":1611579,"reasoning_tokens":1200572} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` |  |  |  |  |  |  |
| 31 | [MiMo-V2.5](https://benchize.com/models/mimo-v2-5.md) | 16% | SE ±3.69 pp | $0.06 | 1,255.81source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.05707 \| USD \| per_test \| Lower is better \| \| `latency` \| 1255.813 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.69 pp sourcePopulation 52 stderr 3.685 tokenTotals {"cache_read_tokens":125621632,"cache_write_tokens":null,"input_tokens":139624413,"output_tokens":11716945,"reasoning_tokens":10908282} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label xiaomi/mimo-v2.5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/xiaomi~1mimo-v2.5` |  |  |  |  |  |  |
| 33 | [Mistral Medium 3.5](https://benchize.com/models/mistral-medium-3-5-high.md)High | 9% | SE ±2.88 pp | $1.3 | 479.05source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.302032 \| USD \| per_test \| Lower is better \| \| `latency` \| 479.048 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.88 pp sourcePopulation 52 stderr 2.876 tokenTotals {"cache_read_tokens":null,"cache_write_tokens":null,"input_tokens":66324649,"output_tokens":1838640,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label mistralai/mistral-medium-3.5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/mistralai~1mistral-medium-3.5` |  |  |  |  |  |  |
| 34 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 3% | SE ±1.71 pp | $0.04 | 201.73source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.044105 \| USD \| per_test \| Lower is better \| \| `latency` \| 201.725 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.71 pp sourcePopulation 52 stderr 1.714 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/inception~1mercury-2.5` |  |  |  |  |  |  |

---
Cite as: ProofBench v1.1 / overall — Benchize, https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall (retrieved 2026-10-21)