# IOI / ioi2024 > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024. Based on the International Olympiad in Informatics Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024 Last updated: Wed, 21 Oct 2026 18:05:10 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Conquered Based on the International Olympiad in Informatics Coding · Models tested: 29 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 100% | SE ±0 pp | $2.04 | 1,861.24source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.040245 \| USD \| per_test \| Lower is better \| \| `latency` \| 1861.244 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":3534365,"cache_write_tokens":299671,"input_tokens":3834387,"output_tokens":155914,"reasoning_tokens":120498} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-6-sol` | | | | | | | | 1 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 100% | SE ±0 pp | $8.14 | 989.27source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.136653 \| USD \| per_test \| Lower is better \| \| `latency` \| 989.274 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":23627831,"cache_write_tokens":867604,"input_tokens":24495791,"output_tokens":641287,"reasoning_tokens":543804} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-fable-5-1` | | | | | | | | 1 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 100% | SE ±0 pp | $2.32 | 702.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.322254 \| USD \| per_test \| Lower is better \| \| `latency` \| 702.579 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":5973062,"cache_write_tokens":0,"input_tokens":7390720,"output_tokens":353414,"reasoning_tokens":269441} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label google/gemini-4-argon Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/google~1gemini-4-argon` | | | | | | | | 1 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 100% | SE ±0 pp | $6.69 | 2,893.05source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.687056 \| USD \| per_test \| Lower is better \| \| `latency` \| 2893.045 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":3331853,"cache_write_tokens":226973,"input_tokens":3559180,"output_tokens":137780,"reasoning_tokens":95448} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-5.6-sol` | | | | | | | | 1 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 100% | SE ±0 pp | $9.35 | 2,484.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.345056 \| USD \| per_test \| Lower is better \| \| `latency` \| 2484.164 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":46870966,"cache_write_tokens":1447518,"input_tokens":48319012,"output_tokens":943409,"reasoning_tokens":753737} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-opus-5` | | | | | | | | 1 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 100% | SE ±0 pp | $3.36 | 1,078.13source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.359584 \| USD \| per_test \| Lower is better \| \| `latency` \| 1078.132 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":1234356,"cache_write_tokens":130132,"input_tokens":1364710,"output_tokens":64652,"reasoning_tokens":36313} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-astra Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-6-astra` | | | | | | | | 1 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 100% | SE ±0 pp | $3.66 | 853.12source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.663947 \| USD \| per_test \| Lower is better \| \| `latency` \| 853.121 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":18408980,"cache_write_tokens":1024815,"input_tokens":19434089,"output_tokens":474742,"reasoning_tokens":383878} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-opus-5-5` | | | | | | | | 1 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 100% | SE ±0 pp | $0.78 | 1,198.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.780945 \| USD \| per_test \| Lower is better \| \| `latency` \| 1198.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":1475302,"cache_write_tokens":165481,"input_tokens":1641035,"output_tokens":91097,"reasoning_tokens":62562} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-6.1-sol` | | | | | | | | 9 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 85% | SE ±13.69 pp | $5.77 | 2,893.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.767053 \| USD \| per_test \| Lower is better \| \| `latency` \| 2893.871 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±13.69 pp sourcePopulation 44 stderr 13.693 tokenTotals {"cache_read_tokens":50475623,"cache_write_tokens":4935394,"input_tokens":55411541,"output_tokens":1216766,"reasoning_tokens":1031007} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-sonnet-5-5` | | | | | | | | 9 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 85% | SE ±13.69 pp | $0.69 | 4,514.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.68829 \| USD \| per_test \| Lower is better \| \| `latency` \| 4514.485 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±13.69 pp sourcePopulation 44 stderr 13.693 tokenTotals {"cache_read_tokens":26448976,"cache_write_tokens":877395,"input_tokens":27327154,"output_tokens":405689,"reasoning_tokens":306881} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-5.6-luna` | | | | | | | | 9 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 85% | SE ±13.69 pp | $8.92 | 5,068.74source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.916335 \| USD \| per_test \| Lower is better \| \| `latency` \| 5068.741 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±13.69 pp sourcePopulation 44 stderr 13.693 tokenTotals {"cache_read_tokens":12173912,"cache_write_tokens":411829,"input_tokens":12586377,"output_tokens":239591,"reasoning_tokens":185477} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-5.6-terra` | | | | | | | | 12 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 72.83% | SE ±15.81 pp | $2.5 | 549.77source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.498747 \| USD \| per_test \| Lower is better \| \| `latency` \| 549.766 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.81 pp sourcePopulation 44 stderr 15.812 tokenTotals {"cache_read_tokens":29027083,"cache_write_tokens":0,"input_tokens":33091114,"output_tokens":605650,"reasoning_tokens":336489} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/google~1gemini-3.7-flash` | | | | | | | | 13 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 69.33% | SE ±14.06 pp | $0.28 | 2,908.98source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.280681 \| USD \| per_test \| Lower is better \| \| `latency` \| 2908.975 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.06 pp sourcePopulation 44 stderr 14.061 tokenTotals {"cache_read_tokens":32253025,"cache_write_tokens":1137051,"input_tokens":33390628,"output_tokens":703757,"reasoning_tokens":639764} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-luna Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-6-luna` | | | | | | | | 14 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 67.67% | SE ±14.67 pp | $0.59 | 502.57source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.590829 \| USD \| per_test \| Lower is better \| \| `latency` \| 502.566 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.67 pp sourcePopulation 44 stderr 14.67 tokenTotals {"cache_read_tokens":23570074,"cache_write_tokens":1553468,"input_tokens":25123832,"output_tokens":1182924,"reasoning_tokens":1089532} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-haiku-5-5` | | | | | | | | 15 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 66.83% | SE ±15.02 pp | $3.53 | 1,192.8source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.532323 \| USD \| per_test \| Lower is better \| \| `latency` \| 1192.799 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.02 pp sourcePopulation 44 stderr 15.023 tokenTotals {"cache_read_tokens":34293044,"cache_write_tokens":0,"input_tokens":34986394,"output_tokens":572338,"reasoning_tokens":465092} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label meta/muse-wonder Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/meta~1muse-wonder` | | | | | | | | 16 | [GPT-5.3 Codex](https://benchize.com/models/gpt-5-3-codex-xhigh.md)Extra high | 62.83% | SE ±17.09 pp | $3.11 | 2,616.98source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.109282 \| USD \| per_test \| Lower is better \| \| `latency` \| 2616.979 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±17.09 pp sourcePopulation 44 stderr 17.092 tokenTotals {"cache_read_tokens":16113664,"cache_write_tokens":0,"input_tokens":16836055,"output_tokens":411919,"reasoning_tokens":325681} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-5.3-codex Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-5.3-codex` | | | | | | | | 17 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 55.67% | SE ±14.62 pp | $4.2 | 938.74source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.196808 \| USD \| per_test \| Lower is better \| \| `latency` \| 938.738 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.62 pp sourcePopulation 44 stderr 14.617 tokenTotals {"cache_read_tokens":85293758,"cache_write_tokens":0,"input_tokens":89097624,"output_tokens":890798,"reasoning_tokens":523067} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/google~1gemini-3.8-flash` | | | | | | | | 18 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 55.5% | SE ±18.71 pp | $1.21 | 585.6source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.207234 \| USD \| per_test \| Lower is better \| \| `latency` \| 585.596 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±18.71 pp sourcePopulation 44 stderr 18.714 tokenTotals {"cache_read_tokens":5955720,"cache_write_tokens":0,"input_tokens":7142113,"output_tokens":306623,"reasoning_tokens":243010} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/google~1gemini-3.1-pro-preview` | | | | | | | | 19 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 50.5% | SE ±12.14 pp | $0.25 | 6,891.62source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.247075 \| USD \| per_test \| Lower is better \| \| `latency` \| 6891.616 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.14 pp sourcePopulation 44 stderr 12.142 tokenTotals {"cache_read_tokens":144385600,"cache_write_tokens":0,"input_tokens":147537013,"output_tokens":1725094,"reasoning_tokens":1533007} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/xiaomi~1mimo-v2.6-flash` | | | | | | | | 19 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 50.5% | SE ±13.75 pp | $12.25 | 3,853.25source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.245865 \| USD \| per_test \| Lower is better \| \| `latency` \| 3853.249 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±13.75 pp sourcePopulation 44 stderr 13.751 tokenTotals {"cache_read_tokens":201049345,"cache_write_tokens":4849553,"input_tokens":205900188,"output_tokens":2113886,"reasoning_tokens":1820456} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-sonnet-5` | | | | | | | | 21 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 50% | SE ±16.44 pp | $1.97 | 3,096.03source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.974595 \| USD \| per_test \| Lower is better \| \| `latency` \| 3096.029 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±16.44 pp sourcePopulation 44 stderr 16.444 tokenTotals {"cache_read_tokens":118412672,"cache_write_tokens":0,"input_tokens":118643031,"output_tokens":1592017,"reasoning_tokens":1385043} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 22 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 43.5% | SE ±17.08 pp | $0.79 | 7,363.26source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.787332 \| USD \| per_test \| Lower is better \| \| `latency` \| 7363.258 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±17.08 pp sourcePopulation 44 stderr 17.084 tokenTotals {"cache_read_tokens":23086848,"cache_write_tokens":0,"input_tokens":26943589,"output_tokens":3050999,"reasoning_tokens":3019947} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/xiaomi~1mimo-v2.6-pro` | | | | | | | | 23 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 42.67% | SE ±14.99 pp | $3.03 | 1,781.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.031404 \| USD \| per_test \| Lower is better \| \| `latency` \| 1781.576 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.99 pp sourcePopulation 44 stderr 14.985 tokenTotals {"cache_read_tokens":41843122,"cache_write_tokens":0,"input_tokens":43374206,"output_tokens":2335817,"reasoning_tokens":2233638} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/meta~1muse_spark_1_3` | | | | | | | | 23 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 42.67% | SE ±14.69 pp | $0.21 | 914.56source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.207781 \| USD \| per_test \| Lower is better \| \| `latency` \| 914.56 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.69 pp sourcePopulation 44 stderr 14.693 tokenTotals {"cache_read_tokens":27861632,"cache_write_tokens":0,"input_tokens":28008480,"output_tokens":856811,"reasoning_tokens":758852} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/deepseek~1deepseek-v4.1-flash` | | | | | | | | 25 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 40.67% | SE ±17.6 pp | $4.66 | 1,025.59source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.655748 \| USD \| per_test \| Lower is better \| \| `latency` \| 1025.59 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±17.6 pp sourcePopulation 44 stderr 17.597 tokenTotals {"cache_read_tokens":112474165,"cache_write_tokens":0,"input_tokens":116644938,"output_tokens":640960,"reasoning_tokens":416752} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/google~1gemini-3.6-flash` | | | | | | | | 26 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 34% | SE ±14.89 pp | $0.35 | 1,373.25source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.346944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1373.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.89 pp sourcePopulation 44 stderr 14.887 tokenTotals {"cache_read_tokens":56395904,"cache_write_tokens":0,"input_tokens":56595542,"output_tokens":905221,"reasoning_tokens":766588} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 27 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 26.67% | SE ±11.39 pp | $2.14 | 5,343.37source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.142166 \| USD \| per_test \| Lower is better \| \| `latency` \| 5343.371 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±11.39 pp sourcePopulation 44 stderr 11.394 tokenTotals {"cache_read_tokens":79725056,"cache_write_tokens":0,"input_tokens":82671675,"output_tokens":2027988,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label stepfun/step-5-preview Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/stepfun~1step-5-preview` | | | | | | | | 28 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 22% | SE ±12.13 pp | $2.35 | 1,236.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.353067 \| USD \| per_test \| Lower is better \| \| `latency` \| 1236.283 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.13 pp sourcePopulation 44 stderr 12.131 tokenTotals {"cache_read_tokens":54859363,"cache_write_tokens":0,"input_tokens":56226636,"output_tokens":966428,"reasoning_tokens":743553} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/meta~1muse_spark_1_2` | | | | | | | | 29 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 2.67% | SE ±1.61 pp | $0.13 | 203.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.132547 \| USD \| per_test \| Lower is better \| \| `latency` \| 203.195 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 44 stderr 1.61 tokenTotals {"cache_read_tokens":7344903,"cache_write_tokens":0,"input_tokens":9837909,"output_tokens":182920,"reasoning_tokens":175725} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/inception~1mercury-2.5` | | | | | | | --- Cite as: IOI / ioi2024 — Benchize, https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024 (retrieved 2026-10-21)