# GPT-4o-mini (2024-07-18)High > Source: Benchize (https://benchize.com) — https://benchize.com/models/gpt-4o-mini-2024-07-18-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gpt-4o-mini-2024-07-18-high. OpenAI Canonical URL: https://benchize.com/models/gpt-4o-mini-2024-07-18-high Last updated: Thu, 22 Oct 2026 08:20:15 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. OpenAI ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [MGSM / mgsm zh](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_zh.md) | 86% | SE ±2.19 pp | 8 / 8 | $0.000175 | 3.81source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000175 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.807 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.19 pp sourcePopulation 75 stderr 2.194 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_zh/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / unbiased](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Aunbiased.md) | 73.3% | SE ±1.94 pp | 11 / 12 | $0.000154 | 2.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.3 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000154 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.583 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 95 stderr 1.938 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/unbiased/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / history](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahistory.md) | 54.59% | SE ±2.55 pp | 40 / 40 | $0.00042 | 2.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.593 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00042 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.482 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.55 pp sourcePopulation 138 stderr 2.551 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/history/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [CorpFin v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aoverall.md) | 45.46% | SE ±0.98 pp | 38 / 39 | $0.00885 | 0.04source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.455 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.008848 \| USD \| per_test \| Lower is better \| \| `latency` \| 0.041 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.98 pp sourcePopulation 134 stderr 0.977 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm te](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_te.md) | 76.4% | SE ±2.69 pp | 7 / 8 | $0.000225 | 5.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.4 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000225 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.178 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.69 pp sourcePopulation 75 stderr 2.686 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_te/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [TaxEval v2 / overall](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Aoverall.md) | 60.55% | SE ±0.96 pp | 38 / 40 | $0.000323 | 8.9source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.548 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000323 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.899 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.96 pp sourcePopulation 145 stderr 0.963 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / black](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Ablack.md) | 72.47% | SE ±1.96 pp | 11 / 12 | $0.000157 | 2.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.467 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000157 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.267 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 95 stderr 1.959 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/black/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / philosophy](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphilosophy.md) | 50.7% | SE ±2.24 pp | 40 / 40 | $0.000319 | 4.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.701 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000319 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.171 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.24 pp sourcePopulation 138 stderr 2.238 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/philosophy/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm es](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_es.md) | 91.2% | SE ±1.79 pp | 6 / 8 | $0.000169 | 2.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000169 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.791 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.79 pp sourcePopulation 75 stderr 1.792 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_es/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm fr](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_fr.md) | 86.8% | SE ±2.14 pp | 3 / 8 | $0.000196 | 5.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000196 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.279 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.14 pp sourcePopulation 75 stderr 2.141 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_fr/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [LiveCodeBench / hard](https://benchize.com/benchmarks/vals-ai%3Alcb%3Ahard.md) | 1.43% | SE ±0.63 pp | 43 / 43 | $0.000362 | 8.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.429 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000362 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.949 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.63 pp sourcePopulation 143 stderr 0.634 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/hard/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / economics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aeconomics.md) | 73.34% | SE ±1.52 pp | 40 / 40 | $0.000352 | 4.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.341 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000352 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.476 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.52 pp sourcePopulation 138 stderr 1.522 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/economics/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / indigenous](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Aindigenous.md) | 72.2% | SE ±1.96 pp | 11 / 12 | $0.000158 | 2.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000158 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.58 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 95 stderr 1.962 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/indigenous/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 16.97% | SE ±1.92 pp | 43 / 43 | $0.000295 | 7.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.971 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000295 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.076 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 143 stderr 1.918 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm de](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_de.md) | 88% | SE ±2.06 pp | 8 / 8 | $0.000206 | 4.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000206 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.14 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.06 pp sourcePopulation 75 stderr 2.055 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_de/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MortgageTax / overall](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Aoverall.md) | 54.49% | SE ±1.93 pp | 29 / 31 | $0.000822 | 6.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.492 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000822 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.059 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.93 pp sourcePopulation 98 stderr 1.93 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:13.924Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm ja](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_ja.md) | 84% | SE ±2.32 pp | 8 / 8 | $0.000201 | 5.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000201 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.042 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.32 pp sourcePopulation 75 stderr 2.319 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_ja/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / health](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahealth.md) | 67.48% | SE ±1.64 pp | 40 / 40 | $0.000298 | 3.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.482 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000298 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 138 stderr 1.638 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/health/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [CorpFin v2 / shared max context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Ashared_max_context.md) | 43.12% | SE ±1.69 pp | 36 / 39 | $0.00925 | 0.06source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.124 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009254 \| USD \| per_test \| Lower is better \| \| `latency` \| 0.056 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 134 stderr 1.691 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/shared_max_context/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [AIME / overall](https://benchize.com/benchmarks/vals-ai%3Aaime%3Aoverall.md) | 11.46% | SE ±1.03 pp | 14 / 14 | $0.00101 | 28.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.458 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00101 \| USD \| per_test \| Lower is better \| \| `latency` \| 28.771 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.03 pp sourcePopulation 96 stderr 1.03 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:17.157Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / computer science](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Acomputer_science.md) | 64.39% | SE ±2.37 pp | 40 / 40 | $0.000462 | 5.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.39 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000462 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.662 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.37 pp sourcePopulation 138 stderr 2.365 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/computer_science/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [GPQA Diamond / diamond zero shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_zero_shot_cot.md) | 43.43% | SE ±3.52 pp | 38 / 40 | $0.000403 | 15.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000403 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.6 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.52 pp sourcePopulation 138 stderr 3.523 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:20.661Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_zero_shot_cot/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / law](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Alaw.md) | 37.69% | SE ±1.46 pp | 40 / 40 | $0.000363 | 3.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.693 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000363 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.593 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.46 pp sourcePopulation 138 stderr 1.461 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/law/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MortgageTax / numerical extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Anumerical_extraction.md) | 60.18% | SE ±2.7 pp | 28 / 31 | $0.000727 | 5.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.175 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000727 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.931 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 98 stderr 2.701 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:13.924Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/numerical_extraction/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / business](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abusiness.md) | 70.34% | SE ±1.63 pp | 39 / 40 | $0.000405 | 5.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.342 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000405 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.379 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 138 stderr 1.626 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/business/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm th](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_th.md) | 85.2% | SE ±2.25 pp | 8 / 8 | $0.000173 | 3.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000173 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.264 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.25 pp sourcePopulation 75 stderr 2.246 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_th/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm sw](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_sw.md) | 86% | SE ±2.19 pp | 7 / 8 | $0.000156 | 3.81source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000156 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.814 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.19 pp sourcePopulation 75 stderr 2.194 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_sw/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / asian](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Aasian.md) | 72% | SE ±1.97 pp | 11 / 12 | $0.000157 | 2.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000157 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.246 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.97 pp sourcePopulation 95 stderr 1.966 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/asian/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU-Pro](https://benchize.com/benchmarks/mmlu-pro.md) | 62.74% | SE ±0.46 pp | 40 / 40 | $0.000412 | 5.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.735 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000412 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.086 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.46 pp sourcePopulation 138 stderr 0.461 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / biology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abiology.md) | 80.2% | SE ±1.49 pp | 38 / 40 | $0.000416 | 4.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.195 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000416 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.753 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 138 stderr 1.488 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/biology/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm ru](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_ru.md) | 90.4% | SE ±1.86 pp | 8 / 8 | $0.000175 | 3.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.4 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000175 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.727 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.86 pp sourcePopulation 75 stderr 1.863 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_ru/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MATH-500](https://benchize.com/benchmarks/math-500.md) | 72.6% | SE ±3.9 pp | 5 / 5 | $0.00035 | 6.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_math500.BvFbu92X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.6 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00035 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.304 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 8192 provider OpenAI reasoning Not reported Reasoning effort high Temperature 0 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.9 pp sourcePopulation 60 stderr 3.898 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_math500.BvFbu92X.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:17.658Z Evidence SHA-256 `435ec4da1112362c0a6bb4afeab53bf7b0faf2000e1ddbe1a878735dca1fe35b` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / math](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Amath.md) | 74.24% | SE ±1.19 pp | 37 / 40 | $0.000567 | 6.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.241 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000567 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.517 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 138 stderr 1.19 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/math/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 44.95% | SE ±3.54 pp | 37 / 40 | $0.00067 | 15.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.949 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00067 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.039 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.54 pp sourcePopulation 138 stderr 3.535 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:20.661Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / hispanic](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Ahispanic.md) | 72.3% | SE ±1.96 pp | 11 / 12 | $0.000157 | 2.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.3 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000157 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.477 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 95 stderr 1.96 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/hispanic/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm en](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_en.md) | 94.8% | SE ±1.4 pp | 8 / 8 | $0.000185 | 3.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000185 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.59 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 75 stderr 1.404 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_en/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [LiveCodeBench / overall](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aoverall.md) | 26.42% | SE ±1.13 pp | 43 / 43 | $0.000298 | 7.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.423 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000298 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.364 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.13 pp sourcePopulation 143 stderr 1.129 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / overall](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Aoverall.md) | 72.44% | SE ±0.8 pp | 11 / 12 | $0.000157 | 2.32source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.436 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000157 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.322 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.8 pp sourcePopulation 95 stderr 0.799 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [AIME / aime 2024](https://benchize.com/benchmarks/vals-ai%3Aaime%3Aaime_2024.md) | 8.75% | SE ±1.54 pp | 14 / 14 | $0.00134 | 40.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001342 \| USD \| per_test \| Lower is better \| \| `latency` \| 40.44 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.54 pp sourcePopulation 96 stderr 1.535 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:17.157Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/aime_2024/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [LiveCodeBench / easy](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aeasy.md) | 60.87% | SE ±2.72 pp | 43 / 43 | $0.000237 | 6.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.87 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000237 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.066 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.72 pp sourcePopulation 143 stderr 2.72 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/easy/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [TaxEval v2 / correctness](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Acorrectness.md) | 49.55% | SE ±1.43 pp | 37 / 40 | $0.000323 | 8.9source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000323 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.899 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.43 pp sourcePopulation 145 stderr 1.43 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/correctness/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MortgageTax / semantic extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Asemantic_extraction.md) | 48.81% | SE ±2.76 pp | 30 / 31 | $0.000916 | 6.19source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.808 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000916 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.187 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.76 pp sourcePopulation 98 stderr 2.758 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:13.924Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/semantic_extraction/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / psychology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Apsychology.md) | 72.93% | SE ±1.57 pp | 40 / 40 | $0.000277 | 2.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.932 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000277 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.934 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.57 pp sourcePopulation 138 stderr 1.573 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/psychology/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / chemistry](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Achemistry.md) | 62.28% | SE ±1.44 pp | 40 / 40 | $0.000573 | 8.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.279 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000573 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.068 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.44 pp sourcePopulation 138 stderr 1.441 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/chemistry/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / mgsm bn](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_bn.md) | 79.2% | SE ±2.57 pp | 5 / 8 | $0.000199 | 3.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000199 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.735 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 75 stderr 2.567 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_bn/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [CorpFin v2 / exact pages](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aexact_pages.md) | 52.21% | SE ±1.71 pp | 38 / 39 | $0.000268 | 0.01source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.214 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000268 \| USD \| per_test \| Lower is better \| \| `latency` \| 0.012 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.71 pp sourcePopulation 134 stderr 1.705 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/exact_pages/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [TaxEval v2 / stepwise reasoning](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Astepwise_reasoning.md) | 71.55% | SE ±1.29 pp | 38 / 40 | $0.000323 | 8.9source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.545 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000323 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.899 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 145 stderr 1.29 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/stepwise_reasoning/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [GPQA Diamond](https://benchize.com/benchmarks/gpqa-diamond.md) | 44.19% | SE ±2.5 pp | 37 / 40 | $0.000536 | 15.32source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.192 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000536 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.32 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.5 pp sourcePopulation 138 stderr 2.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:20.661Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MGSM / overall](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Aoverall.md) | 86.18% | SE ±2.13 pp | 6 / 8 | $0.000187 | 4.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.182 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000187 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.033 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 75 stderr 2.133 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:18.177Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 64.43% | SE ±1.33 pp | 40 / 40 | $0.000508 | 7.72source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000508 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.722 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.33 pp sourcePopulation 138 stderr 1.328 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / other](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aother.md) | 66.45% | SE ±1.55 pp | 40 / 40 | $0.000253 | 2.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.45 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000253 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.835 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.55 pp sourcePopulation 138 stderr 1.553 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/other/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMLU Pro / engineering](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aengineering.md) | 39.22% | SE ±1.57 pp | 40 / 40 | $0.000561 | 8.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 39.216 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000561 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.948 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.57 pp sourcePopulation 138 stderr 1.568 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/engineering/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [AIME 2025](https://benchize.com/benchmarks/aime-2025.md) | 14.17% | SE ±1.37 pp | 12 / 14 | $0.000678 | 17.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000678 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.37 pp sourcePopulation 96 stderr 1.373 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:17.157Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/aime_2025/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MedQA / white](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Awhite.md) | 72.35% | SE ±1.96 pp | 11 / 12 | $0.000157 | 1.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.35 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000157 \| USD \| per_test \| Lower is better \| \| `latency` \| 1.776 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 95 stderr 1.959 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:15.908Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/white/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [MMMU Pro / overall](https://benchize.com/benchmarks/vals-ai%3Ammmu%3Aoverall.md) | 56.56% | SE ±1.19 pp | 28 / 28 | $0.00037 | 45.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmmu.BBln8x5x.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.557 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00037 \| USD \| per_test \| Lower is better \| \| `latency` \| 45.016 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 93 stderr 1.191 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmmu.BBln8x5x.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 1 Observed 2026-10-22T02:10:21.664Z Evidence SHA-256 `29395186eddd2c70601228c584a14b3c1d8f9dba4b68256561b46b0dd03c635a` Source locator `/tasks/overall/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | | [CorpFin v2 / max fitting context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Amax_fitting_context.md) | 41.03% | SE ±1.68 pp | 38 / 39 | $0.02 | 0.06source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.026 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.017023 \| USD \| per_test \| Lower is better \| \| `latency` \| 0.055 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.68 pp sourcePopulation 134 stderr 1.679 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label openai/gpt-4o-mini-2024-07-18 Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/max_fitting_context/openai~1gpt-4o-mini-2024-07-18` | | | | | | | | --- Cite as: GPT-4o-mini (2024-07-18)High — Benchize, https://benchize.com/models/gpt-4o-mini-2024-07-18-high (retrieved 2026-10-22)