# OTIS Mock AIME 2024–2025 > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025. A collection of 45 unofficial mock AIME mathematics problems from the OTIS 2024 exam and the two 2025 exams. Each problem has an integer answer between 0 and 999; models’ extracted final answers are checked against the answer key.[Description · epoch.ai](https://epoch.ai/benchmarks/otis-mock-aime-2024-2025) Canonical URL: https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025 Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Conquered A collection of 45 unofficial mock AIME mathematics problems from the OTIS 2024 exam and the two 2025 exams. Each problem has an integer answer between 0 and 999; models’ extracted final answers are checked against the answer key.[Description · epoch.ai](https://epoch.ai/benchmarks/otis-mock-aime-2024-2025) Mathematics · Models tested: 123 · Score publishers: [epoch ai](https://epoch.ai/benchmarks/use-this-data) ## Model results | # | Model | Score | ± | | | --- | --- | --- | --- | --- | | 1 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6.1-sol_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6.1-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-29T22:00:34.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/7` | | | | | | 1 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-fable-5-1_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-fable-5-1 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-01T18:32:00.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/16` | | | | | | 1 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-5.6-sol_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-09T02:07:13.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/165` | | | | | | 1 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-4-argon_high ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-4-argon Source version OTIS Mock AIME 2024-2025 Evaluated 2026-10-13T17:14:57.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/1` | | | | | | 1 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-sol_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-22T19:59:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/11` | | | | | | 1 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-high.md)High | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion claude-fable-5_high ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-fable-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:18:19.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/136` | | | | | | 1 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-opus-5-5_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-5-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-22T18:27:39.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/12` | | | | | | 1 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-astra_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6-astra Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T14:57:43.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/36` | | | | | | 1 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-sonnet-5-5_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-5-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-29T15:15:14.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/8` | | | | | | 1 | [Qwen3.8 Max (0902)](https://benchize.com/models/qwen3-8-max-0902-xhigh.md)Extra high | 100% | SE ±0 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion qwen3.8-max-0902_xhigh ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.8-max-0902 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-02T18:04:58.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/14` | | | | | | 11 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 99.72% | SE ±0.28 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9972222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9972222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-5.6-terra_max ### Reported statistics Uncertainty Standard error: ±0.28 pp standardError 0.002777777777777778 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-terra Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-09T17:07:54.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/163` | | | | | | 11 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 99.72% | SE ±0.28 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9972222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9972222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-fable-5_max ### Reported statistics Uncertainty Standard error: ±0.28 pp standardError 0.002777777777777778 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-fable-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-06-10T20:03:17.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/171` | | | | | | 13 | [Qwen3.8 Max](https://benchize.com/models/qwen3-8-max-xhigh.md)Extra high | 99.44% | SE ±0.39 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9944444444444445 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9944444444444445 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion qwen3.8-max_xhigh ### Reported statistics Uncertainty Standard error: ±0.39 pp standardError 0.0038834738634418433 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.8-max Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-04T16:28:46.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/139` | | | | | | 14 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 98.89% | SE ±0.67 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-luna_max ### Reported statistics Uncertainty Standard error: ±0.67 pp standardError 0.006674027922425511 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6-luna Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-22T22:14:05.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/9` | | | | | | 14 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 98.89% | SE ±1.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-opus-5_max ### Reported statistics Uncertainty Standard error: ±1.11 pp standardError 0.011111111111111112 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-24T18:40:39.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/143` | | | | | | 14 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 98.89% | SE ±0.87 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.8-flash_high ### Reported statistics Uncertainty Standard error: ±0.87 pp standardError 0.008724007409153888 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.8-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-02T17:15:36.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/15` | | | | | | 14 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-xhigh.md)Extra high | 98.89% | SE ±0.54 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion claude-haiku-5-5_xhigh ### Reported statistics Uncertainty Standard error: ±0.54 pp standardError 0.005362818328685681 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-haiku-5-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-10-07T21:16:36.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/5` | | | | | | 18 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 98.61% | SE ±0.99 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9861111111111112 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9861111111111112 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion deepseek-v4-pro-0813_max ### Reported statistics Uncertainty Standard error: ±0.99 pp standardError 0.00990980650017605 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-v4-pro-0813 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-18T21:03:23.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/66` | | | | | | 19 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 98.33% | SE ±1.17 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9833333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9833333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-5.6-luna_max ### Reported statistics Uncertainty Standard error: ±1.17 pp standardError 0.011650421590325524 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-luna Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-09T13:36:49.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/164` | | | | | | 19 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-max.md)Max | 98.33% | SE ±0.64 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9833333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9833333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion deepseek-v4.1-flash_max ### Reported statistics Uncertainty Standard error: ±0.64 pp standardError 0.006405884289881728 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-v4.1-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-10-08T19:02:32.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/4` | | | | | | 21 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-xhigh.md)Extra high | 97.8% | SE ±2.2 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.978 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.978 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion claude-opus-4-7_xhigh ### Reported statistics Uncertainty Standard error: ±2.2 pp standardError 0.022 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-4-7 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-04-17T18:26:45.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/177` | | | | | | 22 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-low.md)Low | 97.78% | SE ±2.22 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9777777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9777777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion claude-fable-5_low ### Reported statistics Uncertainty Standard error: ±2.22 pp standardError 0.022222222222222223 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-fable-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:18:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/135` | | | | | | 22 | [Claude Opus 5](https://benchize.com/models/claude-opus-5.md) | 97.78% | SE ±2.22 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9777777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9777777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion claude-opus-5 ### Reported statistics Uncertainty Standard error: ±2.22 pp standardError 0.022222222222222223 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:21:15.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/131` | | | | | | 22 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-low.md)Low | 97.78% | SE ±2.22 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9777777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9777777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion claude-opus-4-8_low ### Reported statistics Uncertainty Standard error: ±2.22 pp standardError 0.022222222222222223 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-4-8 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:20:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/133` | | | | | | 25 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-high.md)High | 97.22% | SE ±1.31 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9722222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9722222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion claude-haiku-5-5_high ### Reported statistics Uncertainty Standard error: ±1.31 pp standardError 0.013109441770118595 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-haiku-5-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-10-07T21:16:25.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/6` | | | | | | 25 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 97.22% | SE ±1.86 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9722222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9722222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.7-flash_high ### Reported statistics Uncertainty Standard error: ±1.86 pp standardError 0.018586784946171872 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.7-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-14T18:29:12.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/67` | | | | | | 27 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview.md) | 95.6% | SE ±3.1 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.956 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.956 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion gemini-3.1-pro-preview ### Reported statistics Uncertainty Standard error: ±3.1 pp standardError 0.031 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.1-pro-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2026-02-20T04:39:36.998Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/183` | | | | | | 28 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-high.md)High | 95.56% | SE ±3.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion deepseek-v4-pro_high ### Reported statistics Uncertainty Standard error: ±3.11 pp standardError 0.031067790907534743 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-v4-pro Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T23:58:58.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/118` | | | | | | 28 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 95.56% | SE ±3.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.1-pro-preview_high ### Reported statistics Uncertainty Standard error: ±3.11 pp standardError 0.031067790907534736 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.1-pro-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T23:59:11.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/116` | | | | | | 28 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 95.56% | SE ±3.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3-flash-preview_high ### Reported statistics Uncertainty Standard error: ±3.11 pp standardError 0.031067790907534743 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3-flash-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:25:57.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/127` | | | | | | 28 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 95.56% | SE ±2.67 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash_high ### Reported statistics Uncertainty Standard error: ±2.67 pp standardError 0.026696111689702044 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-05-25T13:10:18.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/173` | | | | | | 28 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-low.md)Low | 95.56% | SE ±3.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gpt-5.6-sol_low ### Reported statistics Uncertainty Standard error: ±3.11 pp standardError 0.031067790907534736 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:42:21.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/85` | | | | | | 33 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-xhigh.md)Extra high | 94.72% | SE ±2.26 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9472222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9472222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion claude-sonnet-5_xhigh ### Reported statistics Uncertainty Standard error: ±2.26 pp standardError 0.022644349282618412 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-01T16:06:23.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/167` | | | | | | 34 | [Claude Opus 4.6](https://benchize.com/models/claude-opus-4-6-thinking-budget-65536.md)Thinking · 65536 tokens | 94.44% | SE ±2.82 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget 64K Reasoning effort Not reported sourceModelVersion claude-opus-4-6_64K ### Reported statistics Uncertainty Standard error: ±2.82 pp standardError 0.028247294597471075 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-4-6 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-02-06T04:32:30.354Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/185` | | | | | | 34 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-max.md)Max | 94.44% | SE ±2.09 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion deepseek-v4-flash-0731_max ### Reported statistics Uncertainty Standard error: ±2.09 pp standardError 0.020854366486931897 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-v4-flash-0731 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-02T00:22:31.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/141` | | | | | | 36 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 94.17% | SE ±3.09 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9416666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9416666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.6-flash_high ### Reported statistics Uncertainty Standard error: ±3.09 pp standardError 0.030926354359015452 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.6-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-02T00:21:27.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/142` | | | | | | 37 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-low.md)Low | 93.33% | SE ±3.76 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9333333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9333333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion claude-opus-5_low ### Reported statistics Uncertainty Standard error: ±3.76 pp standardError 0.03760507165451774 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:21:20.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/129` | | | | | | 38 | [Claude Opus 4.6](https://benchize.com/models/claude-opus-4-6-thinking-budget-32768.md)Thinking · 32768 tokens | 93.06% | SE ±3.34 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9305555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9305555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget 32K Reasoning effort Not reported sourceModelVersion claude-opus-4-6_32K ### Reported statistics Uncertainty Standard error: ±3.34 pp standardError 0.03337013961212755 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-4-6 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-02-06T04:31:07.514Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/186` | | | | | | 39 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview.md) | 92.78% | SE ±3.52 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9277777777777778 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9277777777777778 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion gemini-3-flash-preview ### Reported statistics Uncertainty Standard error: ±3.52 pp standardError 0.035196259111492545 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3-flash-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2025-12-17T22:40:30.896Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/189` | | | | | | 40 | [Qwen3.6 Max Preview](https://benchize.com/models/qwen3-6-max-preview.md) | 91.11% | SE ±4.29 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9111111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9111111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.6-max-preview ### Reported statistics Uncertainty Standard error: ±4.29 pp standardError 0.04290254662948546 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-max-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:42:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/84` | | | | | | 40 | [Qwen3.6 27B](https://benchize.com/models/qwen3-6-27b.md) | 91.11% | SE ±4.29 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9111111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9111111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.6-27b ### Reported statistics Uncertainty Standard error: ±4.29 pp standardError 0.04290254662948546 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-27b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:12:36.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/95` | | | | | | 42 | [Inkling Small](https://benchize.com/models/inkling-small-xhigh.md)Extra high | 90% | SE ±3.96 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion inkling-small_xhigh ### Reported statistics Uncertainty Standard error: ±3.96 pp standardError 0.03964807305493795 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label inkling-small Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-14T14:19:47.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/68` | | | | | | 43 | [Inkling](https://benchize.com/models/inkling-xhigh.md)Extra high | 88.89% | SE ±3.92 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8888888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8888888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort xhigh sourceModelVersion Inkling_xhigh ### Reported statistics Uncertainty Standard error: ±3.92 pp standardError 0.039171949640122654 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label Inkling Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-05T18:55:39.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/138` | | | | | | 43 | [gpt-oss-120b](https://benchize.com/models/gpt-oss-120b-high.md)High | 88.89% | SE ±4.46 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8888888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8888888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion openai/gpt-oss-120b_high ### Reported statistics Uncertainty Standard error: ±4.46 pp standardError 0.04463343655613273 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label openai/gpt-oss-120b Source version OTIS Mock AIME 2024-2025 Evaluated 2025-12-11T22:37:41.702Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/192` | | | | | | 43 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-low.md)Low | 88.89% | SE ±4.74 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8888888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8888888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gpt-5.6-terra_low ### Reported statistics Uncertainty Standard error: ±4.74 pp standardError 0.04737793696791343 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-terra Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:20:21.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/91` | | | | | | 43 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-low.md)Low | 88.89% | SE ±4.74 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8888888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8888888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gemini-3.5-flash_low ### Reported statistics Uncertainty Standard error: ±4.74 pp standardError 0.047377936967913425 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:34:49.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/123` | | | | | | 43 | [Qwen3.5 397B A17B](https://benchize.com/models/qwen3-5-397b-a17b.md) | 88.89% | SE ±4.74 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8888888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8888888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.5-397b-a17b ### Reported statistics Uncertainty Standard error: ±4.74 pp standardError 0.04737793696791343 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-397b-a17b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:12:18.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/97` | | | | | | 48 | [DeepSeek Reasoner](https://benchize.com/models/deepseek-reasoner.md) | 87.82% | SE ±4.13 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.878174603174603 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.878174603174603 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion deepseek-reasoner ### Reported statistics Uncertainty Standard error: ±4.13 pp standardError 0.041306766974863386 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-reasoner Source version OTIS Mock AIME 2024-2025 Evaluated 2025-12-16T14:32:17.922Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/190` | | | | | | 49 | [Qwen3.5 Plus](https://benchize.com/models/qwen3-5-plus.md) | 86.67% | SE ±5.12 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8666666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8666666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.5-plus ### Reported statistics Uncertainty Standard error: ±5.12 pp standardError 0.051247074319053836 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-plus Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:11:58.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/99` | | | | | | 49 | [Qwen3 235B A22B Thinking 2507](https://benchize.com/models/qwen3-235b-a22b-thinking-2507.md) | 86.67% | SE ±5.12 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8666666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8666666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion Qwen3-235B-A22B-Thinking-2507 ### Reported statistics Uncertainty Standard error: ±5.12 pp standardError 0.051247074319053836 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label Qwen3-235B-A22B-Thinking-2507 Source version OTIS Mock AIME 2024-2025 Evaluated 2025-12-10T18:03:27.753Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/196` | | | | | | 49 | [Qwen3.6 35B A3B](https://benchize.com/models/qwen3-6-35b-a3b.md) | 86.67% | SE ±5.12 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8666666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8666666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.6-35b-a3b ### Reported statistics Uncertainty Standard error: ±5.12 pp standardError 0.051247074319053836 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-35b-a3b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:03:16.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/102` | | | | | | 49 | [Qwen3.7 Flash](https://benchize.com/models/qwen3-7-flash.md) | 86.67% | SE ±5.12 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8666666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8666666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.7-flash ### Reported statistics Uncertainty Standard error: ±5.12 pp standardError 0.051247074319053836 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.7-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:02:46.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/106` | | | | | | 53 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-thinking-budget-32768.md)Thinking · 32768 tokens | 85.79% | SE ±4.54 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.857936507936508 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.857936507936508 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget 32K Reasoning effort Not reported sourceModelVersion claude-sonnet-4-6_32K ### Reported statistics Uncertainty Standard error: ±4.54 pp standardError 0.04539927553202047 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-4-6 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-02-20T16:18:28.678Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/182` | | | | | | 54 | [Gemini 2.5 Pro](https://benchize.com/models/gemini-2-5-pro.md) | 84.72% | SE ±4.81 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8472222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8416666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion gemini-2.5-pro ### Reported statistics Uncertainty Standard error: ±4.81 pp standardError 0.048143196054711346 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-2.5-pro Source version OTIS Mock AIME 2024-2025 Evaluated 2025-11-16T19:00:06.794Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/203` | | | | | | 55 | [Qwen3.6 Flash](https://benchize.com/models/qwen3-6-flash.md) | 84.44% | SE ±5.46 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.6-flash ### Reported statistics Uncertainty Standard error: ±5.46 pp standardError 0.05463890236888291 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:02:08.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/109` | | | | | | 55 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-off.md)Non-thinking | 84.44% | SE ±5.46 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion claude-opus-4-8_none ### Reported statistics Uncertainty Standard error: ±5.46 pp standardError 0.05463890236888291 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-opus-4-8 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:21:06.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/132` | | | | | | 55 | [GPT-5.5](https://benchize.com/models/gpt-5-5-low.md)Low | 84.44% | SE ±5.46 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gpt-5.5_low ### Reported statistics Uncertainty Standard error: ±5.46 pp standardError 0.05463890236888291 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:39:05.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/86` | | | | | | 58 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-medium.md)Medium | 82.22% | SE ±5.76 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8222222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8222222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort medium sourceModelVersion claude-sonnet-4-6_medium ### Reported statistics Uncertainty Standard error: ±5.76 pp standardError 0.057637747950250916 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-4-6 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-13T17:28:00.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/160` | | | | | | 58 | [Gemma 4 26B A4B](https://benchize.com/models/gemma-4-26b-a4b-it-minimal.md)Minimal | 82.22% | SE ±5.76 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8222222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8222222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemma-4-26b-a4b-it_minimal ### Reported statistics Uncertainty Standard error: ±5.76 pp standardError 0.05763774795025092 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemma-4-26b-a4b-it Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-28T12:55:56.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/44` | | | | | | 58 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-low.md)Low | 82.22% | SE ±5.76 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8222222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8222222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gemini-3.6-flash_low ### Reported statistics Uncertainty Standard error: ±5.76 pp standardError 0.057637747950250916 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.6-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:07:20.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/101` | | | | | | 58 | [Qwen3.5 397B A17B](https://benchize.com/models/qwen3-5-397b-a17b-off.md)Non-thinking | 82.22% | SE ±5.76 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8222222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8222222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.5-397b-a17b_none ### Reported statistics Uncertainty Standard error: ±5.76 pp standardError 0.057637747950250916 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-397b-a17b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:12:04.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/98` | | | | | | 62 | [Gemini 3.1 Flash Lite](https://benchize.com/models/gemini-3-1-flash-lite-high.md)High | 80% | SE ±6.03 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.1-flash-lite_high ### Reported statistics Uncertainty Standard error: ±6.03 pp standardError 0.06030226891555273 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.1-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:29:57.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/126` | | | | | | 62 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-minimal.md)Minimal | 80% | SE ±6.03 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemini-3.5-flash_minimal ### Reported statistics Uncertainty Standard error: ±6.03 pp standardError 0.06030226891555273 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-15T03:14:20.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/155` | | | | | | 62 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 80% | SE ±6.03 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-sonnet-5_max ### Reported statistics Uncertainty Standard error: ±6.03 pp standardError 0.06030226891555273 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-5 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:21:48.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/128` | | | | | | 62 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-minimal.md)Minimal | 80% | SE ±6.03 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemini-3.6-flash_minimal ### Reported statistics Uncertainty Standard error: ±6.03 pp standardError 0.06030226891555273 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.6-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:07:32.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/100` | | | | | | 66 | [Qwen3.7 Flash](https://benchize.com/models/qwen3-7-flash-off.md)Non-thinking | 77.78% | SE ±6.27 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7777777777777778 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7777777777777778 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.7-flash_none ### Reported statistics Uncertainty Standard error: ±6.27 pp standardError 0.06267511942419625 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.7-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:02:33.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/107` | | | | | | 67 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-high.md)High | 75.56% | SE ±6.48 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7555555555555555 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7555555555555555 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion claude-sonnet-4-6_high ### Reported statistics Uncertainty Standard error: ±6.48 pp standardError 0.06478835438717 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label claude-sonnet-4-6 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-13T17:28:15.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/157` | | | | | | 68 | [Gemma 4 31B](https://benchize.com/models/gemma-4-31b-it-minimal.md)Minimal | 73.33% | SE ±6.67 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7333333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7333333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemma-4-31b-it_minimal ### Reported statistics Uncertainty Standard error: ±6.67 pp standardError 0.06666666666666667 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemma-4-31b-it Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T23:59:45.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/115` | | | | | | 69 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 71.11% | SE ±6.83 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7111111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7111111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion MiniMax-M3 ### Reported statistics Uncertainty Standard error: ±6.83 pp standardError 0.0683294324254051 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label MiniMax-M3 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-10T17:46:09.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/73` | | | | | | 69 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 71.11% | SE ±6.83 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7111111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7111111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash-lite_high ### Reported statistics Uncertainty Standard error: ±6.83 pp standardError 0.0683294324254051 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:35:36.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/122` | | | | | | 71 | [Qwen3 30B A3B Thinking 2507](https://benchize.com/models/qwen3-30b-a3b-thinking-2507.md) | 70.28% | SE ±6.04 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7027777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7027777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3-30b-a3b-thinking-2507 ### Reported statistics Uncertainty Standard error: ±6.04 pp standardError 0.060418480757624834 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3-30b-a3b-thinking-2507 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:44:45.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/23` | | | | | | 72 | [Qwen3.5-35B-A3B](https://benchize.com/models/qwen3-5-35b-a3b-off.md)Non-thinking | 70% | SE ±6 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.7 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.5-35b-a3b_none ### Reported statistics Uncertainty Standard error: ±6 pp standardError 0.06003996985174109 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-35b-a3b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:45:56.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/19` | | | | | | 73 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-off.md)Non-thinking | 68.89% | SE ±6.98 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion gpt-5.6-sol_none ### Reported statistics Uncertainty Standard error: ±6.98 pp standardError 0.0697920592732311 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:54:12.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/78` | | | | | | 73 | [Qwen3.6 35B A3B](https://benchize.com/models/qwen3-6-35b-a3b-off.md)Non-thinking | 68.89% | SE ±6.98 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.6-35b-a3b_none ### Reported statistics Uncertainty Standard error: ±6.98 pp standardError 0.06979205927323111 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-35b-a3b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:03:06.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/103` | | | | | | 75 | [Qwen3 32B](https://benchize.com/models/qwen3-32b.md) | 66.94% | SE ±6.04 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6694444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6694444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion Qwen3-32B ### Reported statistics Uncertainty Standard error: ±6.04 pp standardError 0.06040106323530329 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label Qwen3-32B Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:44:49.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/22` | | | | | | 76 | [Qwen3.6 27B](https://benchize.com/models/qwen3-6-27b-off.md)Non-thinking | 66.67% | SE ±7.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6666666666666666 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6666666666666666 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.6-27b_none ### Reported statistics Uncertainty Standard error: ±7.11 pp standardError 0.07106690545187015 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.6-27b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:12:23.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/96` | | | | | | 76 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-low.md)Low | 66.67% | SE ±7.11 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6666666666666666 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6666666666666666 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gpt-5.6-luna_low ### Reported statistics Uncertainty Standard error: ±7.11 pp standardError 0.07106690545187014 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-luna Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:02:15.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/108` | | | | | | 78 | [R1 0528](https://benchize.com/models/deepseek-r1-0528.md) | 66.39% | SE ±4.91 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6638888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6638888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion DeepSeek-R1-0528 ### Reported statistics Uncertainty Standard error: ±4.91 pp standardError 0.04911177955575991 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label DeepSeek-R1-0528 Source version OTIS Mock AIME 2024-2025 Evaluated 2025-05-29T10:59:10.779Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/225` | | | | | | 78 | [Qwen3 14B](https://benchize.com/models/qwen3-14b.md) | 66.39% | SE ±6.19 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6638888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6638888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3-14b ### Reported statistics Uncertainty Standard error: ±6.19 pp standardError 0.06190655406811223 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3-14b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-28T16:54:14.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/42` | | | | | | 80 | [gpt-oss-20b](https://benchize.com/models/gpt-oss-20b-medium.md)Medium | 65.28% | SE ±5.93 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6527777777777778 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6527777777777778 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort medium sourceModelVersion gpt-oss-20b_medium ### Reported statistics Uncertainty Standard error: ±5.93 pp standardError 0.0593111883162336 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-oss-20b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:12:40.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/62` | | | | | | 81 | [Qwen3 30B A3B](https://benchize.com/models/qwen3-30b-a3b.md) | 62.78% | SE ±6.21 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6277777777777778 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6277777777777778 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion Qwen3-30B-A3B ### Reported statistics Uncertainty Standard error: ±6.21 pp standardError 0.062118645754659016 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label Qwen3-30B-A3B Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-28T17:03:13.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/41` | | | | | | 82 | [Qwen3 30B A3B Instruct 2507](https://benchize.com/models/qwen3-30b-a3b-instruct-2507.md) | 62.22% | SE ±6.51 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6222222222222222 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6222222222222222 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3-30b-a3b-instruct-2507 ### Reported statistics Uncertainty Standard error: ±6.51 pp standardError 0.06509619058885623 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3-30b-a3b-instruct-2507 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:44:11.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/24` | | | | | | 83 | [Qwen3.5-9B](https://benchize.com/models/qwen3-5-9b-off.md)Non-thinking | 61.67% | SE ±6 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6166666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6166666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion qwen3.5-9B_none ### Reported statistics Uncertainty Standard error: ±6 pp standardError 0.06003996985174109 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-9B Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:31:08.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/56` | | | | | | 84 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-low.md)Low | 60% | SE ±7.39 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.6 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.6 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gemini-3.5-flash-lite_low ### Reported statistics Uncertainty Standard error: ±7.39 pp standardError 0.07385489458759963 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:36:02.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/120` | | | | | | 85 | [GLM-4.7 Flash](https://benchize.com/models/glm-4-7-flash.md) | 58.33% | SE ±6.56 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5833333333333334 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5833333333333334 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion glm-4.7-flash ### Reported statistics Uncertainty Standard error: ±6.56 pp standardError 0.06564078093085213 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label glm-4.7-flash Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:37:43.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/35` | | | | | | 86 | [Qwen3 8B](https://benchize.com/models/qwen3-8b.md) | 56.11% | SE ±6.45 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5611111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5611111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3-8b ### Reported statistics Uncertainty Standard error: ±6.45 pp standardError 0.0645035213369854 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3-8b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:21:53.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/59` | | | | | | 87 | [Qwen3.5-35B-A3B](https://benchize.com/models/qwen3-5-35b-a3b.md) | 54.44% | SE ±6.19 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.5-35b-a3b ### Reported statistics Uncertainty Standard error: ±6.19 pp standardError 0.061949030495634445 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-35b-a3b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-30T22:45:36.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/20` | | | | | | 88 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-off.md)Non-thinking | 53.33% | SE ±7.52 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5333333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5333333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion gpt-5.6-terra_none ### Reported statistics Uncertainty Standard error: ±7.52 pp standardError 0.07521014330903548 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-terra Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:42:38.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/82` | | | | | | 88 | [DeepSeek R1](https://benchize.com/models/deepseek-r1.md) | 53.33% | SE ±7.52 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5333333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5333333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion DeepSeek-R1 ### Reported statistics Uncertainty Standard error: ±7.52 pp standardError 0.07521014330903548 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label DeepSeek-R1 Source version OTIS Mock AIME 2024-2025 Evaluated 2025-02-26T16:56:45.594Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/270` | | | | | | 90 | [R1 Distill Llama 70B](https://benchize.com/models/deepseek-r1-distill-llama-70b.md) | 51.39% | SE ±6.05 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5138888888888888 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5138888888888888 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion DeepSeek-R1-Distill-Llama-70B ### Reported statistics Uncertainty Standard error: ±6.05 pp standardError 0.060540263104731575 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label DeepSeek-R1-Distill-Llama-70B Source version OTIS Mock AIME 2024-2025 Evaluated 2025-03-07T23:03:59.002Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/258` | | | | | | 91 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-minimal.md)Minimal | 51.11% | SE ±7.54 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5111111111111111 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5111111111111111 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemini-3.5-flash-lite_minimal ### Reported statistics Uncertainty Standard error: ±7.54 pp standardError 0.07535922203472521 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.5-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:35:57.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/121` | | | | | | 92 | [gpt-oss-20b](https://benchize.com/models/gpt-oss-20b-high.md)High | 50.83% | SE ±6.71 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5083333333333333 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5083333333333333 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gpt-oss-20b_high ### Reported statistics Uncertainty Standard error: ±6.71 pp standardError 0.06711496995394688 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-oss-20b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:13:32.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/61` | | | | | | 93 | [DeepSeek V3](https://benchize.com/models/deepseek-chat.md) | 48.89% | SE ±7.54 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4888888888888889 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.4888888888888889 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion deepseek-chat ### Reported statistics Uncertainty Standard error: ±7.54 pp standardError 0.0753592220347252 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label deepseek-chat Source version OTIS Mock AIME 2024-2025 Evaluated 2026-07-16T00:10:43.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/147` | | | | | | 94 | [Gemini 3.1 Flash Lite](https://benchize.com/models/gemini-3-1-flash-lite-low.md)Low | 44.44% | SE ±7.49 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4444444444444444 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.4444444444444444 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gemini-3.1-flash-lite_low ### Reported statistics Uncertainty Standard error: ±7.49 pp standardError 0.07491109582924914 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.1-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:30:07.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/125` | | | | | | 95 | [Qwen3.5-9B](https://benchize.com/models/qwen3-5-9b.md) | 42.78% | SE ±6.53 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.42777777777777776 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.42777777777777776 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion qwen3.5-9B ### Reported statistics Uncertainty Standard error: ±6.53 pp standardError 0.06533819469304575 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label qwen3.5-9B Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:30:39.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/57` | | | | | | 96 | [gpt-oss-20b](https://benchize.com/models/gpt-oss-20b-low.md)Low | 40.28% | SE ±6.32 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4027777777777778 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.4027777777777778 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort low sourceModelVersion gpt-oss-20b_low ### Reported statistics Uncertainty Standard error: ±6.32 pp standardError 0.06317675132490254 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-oss-20b Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-27T14:12:25.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/63` | | | | | | 97 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-off.md)Non-thinking | 40% | SE ±7.39 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.4 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion gpt-5.6-luna_none ### Reported statistics Uncertainty Standard error: ±7.39 pp standardError 0.07385489458759964 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-5.6-luna Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-07T00:20:15.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/92` | | | | | | 98 | [Gemini 3.1 Flash Lite](https://benchize.com/models/gemini-3-1-flash-lite-minimal.md)Minimal | 37.78% | SE ±7.31 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.37777777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.37777777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort minimal sourceModelVersion gemini-3.1-flash-lite_minimal ### Reported statistics Uncertainty Standard error: ±7.31 pp standardError 0.07309112127323451 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3.1-flash-lite Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:30:17.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/124` | | | | | | 98 | [DeepSeek V3 0324](https://benchize.com/models/deepseek-chat-v3-0324.md) | 37.78% | SE ±6.24 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.37777777777777777 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.37777777777777777 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort Not reported sourceModelVersion DeepSeek-V3-0324 ### Reported statistics Uncertainty Standard error: ±6.24 pp standardError 0.062372203800719374 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label DeepSeek-V3-0324 Source version OTIS Mock AIME 2024-2025 Evaluated 2025-04-01T14:33:01.704Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/250` | | | | | | 100 | [MiniMax M3](https://benchize.com/models/minimax-m3-off.md)Non-thinking | 26.67% | SE ±6.67 pp | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.26666666666666666 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.26666666666666666 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort none sourceModelVersion MiniMax-M3_none ### Reported statistics Uncertainty Standard error: ±6.67 pp standardError 0.06666666666666668 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label MiniMax-M3 Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-10T17:46:50.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/72` | | | | | [Next page](https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025.md?cursor=%7B%22score%22%3A26.666666666666668%2C%22id%22%3A%22e733811606490e309dd9c7bbaebef9cd8003b0d925b03da66945fc3cf73f46dd%22%7D) --- Cite as: OTIS Mock AIME 2024–2025 — Benchize, https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025 (retrieved 2026-10-21)