# GPT-6 SolMax

> Source: Benchize (https://benchize.com) — https://benchize.com/models/gpt-6-sol-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gpt-6-sol-max.

OpenAI

Canonical URL: https://benchize.com/models/gpt-6-sol-max
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

OpenAI

## Benchmarks

| Benchmark | Score | ± | Rank | Cost | Latency | Source |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| [τ³-Banking](https://benchize.com/benchmarks/mercor%3Atau3-banking.md) | 43% | — | 10 / 24 | — | — | [mercor](https://www.mercor.com/apex/partner-benchmarks/tau-3-banking-leaderboard/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 43 \| percent \| task_completion_pass_at_1 \| Higher is better \| ### Configuration datasetUrl Not reported datasetVersion Not reported harness Not reported Reasoning effort max ### Reported statistics reportedErrors {"pass_1":8.6} sampleSize 291 ### Source [Original source](https://www.mercor.com/apex/partner-benchmarks/tau-3-banking-leaderboard/) Source mercor Original model label GPT-6 Sol Observed 2026-10-21T16:33:27.812Z Evidence SHA-256 `78850adeb8ad047fcb1dd65cc04ffaca61c86fc67e7035f9973805b02483ed07` Source locator `/__NEXT_DATA__/props/pageProps/leaderboard/rows/7` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / general quantitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_quantitative_analysis.md) | 70% | SE ±2.03 pp | 32 / 50 | $1.78 | 403.98source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.002 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.779785 \| USD \| per_test \| Lower is better \| \| `latency` \| 403.982 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.03 pp sourcePopulation 80 stderr 2.026 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_quantitative_analysis/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 92.13% | SE ±1.46 pp | 15 / 49 | $4.67 | 1,000.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.128 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.665221 \| USD \| per_test \| Lower is better \| \| `latency` \| 1000.01 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.46 pp sourcePopulation 81 stderr 1.456 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 45.46% | SE ±5.25 pp | 9 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.455 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.25 pp sourcePopulation 52 stderr 5.249 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label openai/gpt-6-sol Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 57.95% | SE ±0.63 pp | 34 / 50 | $1.25 | 412.4source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.951 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.250149 \| USD \| per_test \| Lower is better \| \| `latency` \| 412.402 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.63 pp sourcePopulation 80 stderr 0.633 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [IOI / ioi2024](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024.md) | 100% | SE ±0 pp | 1 / 29 | $2.04 | 1,861.24source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.040245 \| USD \| per_test \| Lower is better \| \| `latency` \| 1861.244 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 44 stderr 0 tokenTotals {"cache_read_tokens":3534365,"cache_write_tokens":299671,"input_tokens":3834387,"output_tokens":155914,"reasoning_tokens":120498} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 30% | SE ±5.52 pp | 7 / 30 | $5.82 | 4,930.43source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.821477 \| USD \| per_test \| Lower is better \| \| `latency` \| 4930.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.52 pp sourcePopulation 41 stderr 5.517 tokenTotals {"cache_read_tokens":1090637986,"cache_write_tokens":15843344,"input_tokens":2212998922,"output_tokens":14625198,"reasoning_tokens":5482665} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label openai/gpt-6-sol Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 75% | SE ±4.99 pp | 14 / 36 | $0.18 | 27.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.176051 \| USD \| per_test \| Lower is better \| \| `latency` \| 27.696 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.99 pp sourcePopulation 91 stderr 4.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [AutomationBench](https://benchize.com/benchmarks/zapier%3Aautomationbench.md) | 32% | — | 13 / 113 | $0.34 | — | [zapier](https://framerusercontent.com/sites/4WTSl4BNjd1q9QFEFibC6h/lOvLnhItj8S8-1vyjH_s33Jn-WmUXPmiJ5Diajl0Q38.CMutaHYl.mjs) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 32 \| percent \| reported \| Higher is better \| \| `cost` \| 0.34 \| USD \| per_task \| Lower is better \| ### Configuration benchmarkVersion 1.0.6 costFootnote false Fallback policy Not reported Reasoning effort max Model configuration GPT-6 Sol (Max) thinkingEnabled true ### Reported statistics rank 13 ### Source [Original source](https://framerusercontent.com/sites/4WTSl4BNjd1q9QFEFibC6h/lOvLnhItj8S8-1vyjH_s33Jn-WmUXPmiJ5Diajl0Q38.CMutaHYl.mjs) Source zapier Original model label GPT-6 Sol (Max) Observed 2026-10-21T16:34:56.783Z Evidence SHA-256 `43978e38f018789fa59c11de5328abe9e5905bfa9a5a4d601fa3aa4d9c153b0d` Source locator `/rows/12` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Areal_estate.md) | 0% | SE ±0 pp | 6 / 49 | $4.67 | 1,000.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.665221 \| USD \| per_test \| Lower is better \| \| `latency` \| 1000.01 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/real_estate/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 0% | SE ±0 pp | 6 / 49 | $1.55 | 616.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.548993 \| USD \| per_test \| Lower is better \| \| `latency` \| 616.432 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / korean](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Akorean.md) | 1,417.15 | — | 39 / 124 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1417.1471878765865 \| rating \| reported \| Higher is better \| ### Configuration category korean config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1375.876020247597 rank 63 upper 1458.418355505576 variance 443.4016755000356 votes 228 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/8296/row` |  |  |  |  |  |  |  |
| [Arena text style control / non english](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Anon_english.md) | 1,446.35 | — | 43 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1446.3458603966142 \| rating \| reported \| Higher is better \| ### Configuration category non_english config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1439.0007025055222 rank 62 upper 1453.6910182877061 variance 14.044493762221503 votes 8405 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/9792/row` |  |  |  |  |  |  |  |
| [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 72.92% | SE ±5.02 pp | 6 / 47 | $0.93 | 552.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.923 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.926683 \| USD \| per_test \| Lower is better \| \| `latency` \| 552.8 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.02 pp sourcePopulation 76 stderr 5.016 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 81.32% | SE ±2.3 pp | 23 / 49 | $3.84 | 805.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.315 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.843686 \| USD \| per_test \| Lower is better \| \| `latency` \| 805.67 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.3 pp sourcePopulation 81 stderr 2.297 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [ARC-AGI-3](https://benchize.com/benchmarks/arc-prize%3Av3_Semi_Private.md) | 4.62% | — | 18 / 53 | 5,554.49usd | — | [arc prize · v3](https://arcprize.org/media/data/leaderboard/v3.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.04623568186131146 \| score_fraction \| v3_Semi_Private \| Higher is better \| \| `total_cost` \| 5554.493172 \| usd \| v3_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v3_Semi_Private harness standard modelGroup openai-gpt-6-sol modelType CoT Reasoning effort max sourceModelVersion openai-gpt-6-sol-max thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v3.json) Source arc prize Original model label GPT-6 Sol (Max) Source version v3 Observed 2026-10-21T16:34:21.112Z Evidence SHA-256 `1aaee2199f8c6d9b5a589aab2fd2480a729e9132f8cf8fbf7b49f81dbd5b2c49` Source locator `/evaluations/57` |  |  |  |  |  |  |  |
| [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 2% | SE ±0.99 pp | 9 / 40 | $11.67 | 2,771.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.672651 \| USD \| per_test \| Lower is better \| \| `latency` \| 2771.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 64 stderr 0.992 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [APEX-Agents](https://benchize.com/benchmarks/mercor%3Aapex-agents%3A1.1.md) | 54.3% | — | 19 / 38 | — | — | [mercor · 1.1](https://www.mercor.com/apex/apex-agents-leaderboard/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 54.3 \| percent \| task_completion_pass_at_1 \| Higher is better \| \| `mean_score` \| 68.7 \| percent \| rubric_mean \| Higher is better \| ### Configuration datasetUrl https://huggingface.co/datasets/mercor/apex-agents-v1.1 datasetVersion 1.1 harness loop_truncated_tools_agent Reasoning effort max ### Reported statistics reportedErrors {"mean_score":4.7,"pass_1":5.6} sampleSize 951 ### Source [Original source](https://www.mercor.com/apex/apex-agents-leaderboard/) Source mercor Original model label GPT-6 Sol Source version 1.1 Observed 2026-10-21T16:33:27.509Z Evidence SHA-256 `359c569c3b0b62ab375895eabf3c45846fa45551d419d5c9d6a4c9117db78184` Source locator `/__NEXT_DATA__/props/pageProps/leaderboardData/43` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $3.03 | 884.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.027072 \| USD \| per_test \| Lower is better \| \| `latency` \| 884.656 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 79.69% | SE ±2.7 pp | 32 / 50 | $5.5 | 1,562.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.693 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.495004 \| USD \| per_test \| Lower is better \| \| `latency` \| 1562.595 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 82 stderr 2.698 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench-Science / engineering sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aengineering_sciences.md) | 22.22% | SE ±14.7 pp | 5 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.222 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.7 pp sourcePopulation 41 stderr 14.699 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label openai/gpt-6-sol Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/engineering_sciences/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Furniture Assembly Benchmark](https://benchize.com/benchmarks/epoch-ai%3Afurniture_assembly.md) | 58.33% | SE ±6.42 pp | 8 / 21 | — | — | [epoch ai · Furniture Assembly](https://epoch.ai/data/benchmark_data.zip#file=furniture_assembly.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5833333333333334 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.5833333333333334 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Furniture Assembly reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-sol_max ### Reported statistics Uncertainty Standard error: ±6.42 pp standardError 0.06418399868645064 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=furniture_assembly.csv) Source epoch ai Original model label gpt-6-sol Source version Furniture Assembly Evaluated 2026-09-23T20:34:23.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `d62ab28005a1b7ab0e57b227d04b0cf86c70a5d8a1981addb26d2ac41c116865` Source locator `/rows/11` |  |  |  |  |  |  |  |
| [IOI / ioi2025](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2025.md) | 79.33% | SE ±11.93 pp | 9 / 29 | $3.36 | 1,975.57source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.361427 \| USD \| per_test \| Lower is better \| \| `latency` \| 1975.572 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±11.93 pp sourcePopulation 44 stderr 11.934 tokenTotals {"cache_read_tokens":4091306,"cache_write_tokens":408977,"input_tokens":4500601,"output_tokens":127414,"reasoning_tokens":98238} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2025/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / german](https://benchize.com/benchmarks/lmarena%3Atext%3Agerman.md) | 1,378.94 | — | 81 / 129 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1378.9412378167735 \| rating \| reported \| Higher is better \| ### Configuration category german config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1339.310773753521 rank 144 upper 1418.5717018800265 variance 408.84824104009925 votes 229 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/3280/row` |  |  |  |  |  |  |  |
| [Arena vision / homework (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Ahomework--config-vision.md) | 1,286.35 | — | 41 / 73 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1286.34636227072 \| rating \| reported \| Higher is better \| ### Configuration category homework config vision Reasoning effort max thinkingEnabled true ### Reported statistics lower 1260.597984617661 rank 48 upper 1312.0947399237791 variance 172.5852033589559 votes 518 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/855/row` |  |  |  |  |  |  |  |
| [Arena text / multi turn](https://benchize.com/benchmarks/lmarena%3Atext%3Amulti_turn.md) | 1,413.83 | — | 76 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1413.8317218916363 \| rating \| reported \| Higher is better \| ### Configuration category multi_turn config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1400.5261859432737 rank 131 upper 1427.1372578399987 variance 46.085952013713104 votes 2185 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/9446/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 78.62% | SE ±2.47 pp | 34 / 49 | $2.23 | 724.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.623 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.232444 \| USD \| per_test \| Lower is better \| \| `latency` \| 724.042 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 81 stderr 2.472 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 59.45% | SE ±5.64 pp | 13 / 47 | $1.42 | 694.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.445 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.423011 \| USD \| per_test \| Lower is better \| \| `latency` \| 694.513 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.64 pp sourcePopulation 76 stderr 5.635 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [SRE Bench / overall](https://benchize.com/benchmarks/vals-ai%3Asrebench%3Aoverall.md) | 30.53% | SE ±2.85 pp | 5 / 24 | $20.65 | 3,807.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sre_bench.Bd0uBJxH.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.534 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 20.651048 \| USD \| per_test \| Lower is better \| \| `latency` \| 3807.782 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.85 pp sourcePopulation 35 stderr 2.851 tokenTotals {"cache_read_tokens":13407848991,"cache_write_tokens":113335637,"input_tokens":13521569335,"output_tokens":35917420,"reasoning_tokens":22503283} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sre_bench.Bd0uBJxH.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:15.724Z Evidence SHA-256 `a10b2c964f5878b0fc21c888703f311a5ef7d7a74fc3c529cab3ddd9fa474e2b` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / industry writing and literature and language](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_writing_and_literature_and_language.md) | 1,437.56 | — | 47 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1437.5609890704102 \| rating \| reported \| Higher is better \| ### Configuration category industry_writing_and_literature_and_language config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1426.9747155033085 rank 73 upper 1448.1472626375119 variance 29.173601297973807 votes 3774 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/7190/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $2.27 | 865.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.274689 \| USD \| per_test \| Lower is better \| \| `latency` \| 865.958 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 100% | SE ±0 pp | 1 / 43 | $8.3 | 6,650source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.299794 \| USD \| per_test \| Lower is better \| \| `latency` \| 6649.999 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 76 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 18.52% | SE ±7.62 pp | 38 / 50 | $5.21 | 1,441.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.519 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.211283 \| USD \| per_test \| Lower is better \| \| `latency` \| 1441.302 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.62 pp sourcePopulation 82 stderr 7.618 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / polish](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Apolish.md) | 1,451.65 | — | 45 / 117 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1451.6505749640135 \| rating \| reported \| Higher is better \| ### Configuration category polish config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1410.174155378732 rank 69 upper 1493.1269945492947 variance 447.82293964652484 votes 196 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/10216/row` |  |  |  |  |  |  |  |
| [Public Benefits Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall.md) | 56.63% | SE ±1.29 pp | 28 / 34 | $2.49 | 1,862.49source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.631 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.491613 \| USD \| per_test \| Lower is better \| \| `latency` \| 1862.494 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 50 stderr 1.289 tokenTotals {"cache_read_tokens":1494255406,"cache_write_tokens":57401341,"input_tokens":1552257632,"output_tokens":7014313,"reasoning_tokens":3446365} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / chinese](https://benchize.com/benchmarks/lmarena%3Atext%3Achinese.md) | 1,411.78 | — | 97 / 146 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1411.7773506017397 \| rating \| reported \| Higher is better \| ### Configuration category chinese config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1393.4290357650284 rank 173 upper 1430.1256654384513 variance 87.63875211507728 votes 1135 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/589/row` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__forms_filings.md) | 53.68% | SE ±8.22 pp | 31 / 46 | $3.19 | 1,481.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.675 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.18507 \| USD \| per_test \| Lower is better \| \| `latency` \| 1481.667 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.22 pp sourcePopulation 73 stderr 8.222 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__forms_filings/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $6.26 | 828.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.264231 \| USD \| per_test \| Lower is better \| \| `latency` \| 828.34 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 26.32% | SE ±5.88 pp | 31 / 50 | $5.5 | 1,562.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.316 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.495004 \| USD \| per_test \| Lower is better \| \| `latency` \| 1562.595 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.88 pp sourcePopulation 82 stderr 5.884 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / polish](https://benchize.com/benchmarks/lmarena%3Atext%3Apolish.md) | 1,372.3 | — | 83 / 117 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1372.3049610680143 \| rating \| reported \| Higher is better \| ### Configuration category polish config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1330.4158287114792 rank 148 upper 1414.1940934245495 variance 456.7794401779525 votes 196 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/10295/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 84.73% | SE ±2.17 pp | 24 / 49 | $1.55 | 616.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.727 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.548993 \| USD \| per_test \| Lower is better \| \| `latency` \| 616.432 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.17 pp sourcePopulation 81 stderr 2.173 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 49.93% | SE ±7.23 pp | 33 / 46 | $2.09 | 939.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.931 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.094899 \| USD \| per_test \| Lower is better \| \| `latency` \| 939.613 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.23 pp sourcePopulation 73 stderr 7.233 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / multi turn](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Amulti_turn.md) | 1,468.37 | — | 42 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1468.365236418399 \| rating \| reported \| Higher is better \| ### Configuration category multi_turn config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1455.0345777632533 rank 65 upper 1481.6958950735448 variance 46.26014971778398 votes 2185 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/9380/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 89.05% | SE ±1.86 pp | 28 / 49 | $2.68 | 924.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.046 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.678832 \| USD \| per_test \| Lower is better \| \| `latency` \| 924.138 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.86 pp sourcePopulation 81 stderr 1.86 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $2.23 | 724.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.232444 \| USD \| per_test \| Lower is better \| \| `latency` \| 724.042 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $2.96 | 751.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.96028 \| USD \| per_test \| Lower is better \| \| `latency` \| 751.328 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / exclude ties](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aexclude_ties.md) | 1,463.93 | — | 41 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1463.9272161347058 \| rating \| reported \| Higher is better \| ### Configuration category exclude_ties config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1455.7325870464967 rank 61 upper 1472.1218452229148 variance 17.480844915366553 votes 10194 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/2114/row` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 79.18% | SE ±1.56 pp | 38 / 50 | $4.9 | 1,449.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.181 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.897717 \| USD \| per_test \| Lower is better \| \| `latency` \| 1449.451 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 82 stderr 1.564 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / industry life and physical and social science](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_life_and_physical_and_social_science.md) | 1,472.78 | — | 48 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1472.7826050286733 \| rating \| reported \| Higher is better \| ### Configuration category industry_life_and_physical_and_social_science config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1459.8268714887188 rank 71 upper 1485.7383385686276 variance 43.69460650055067 votes 2325 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/5568/row` |  |  |  |  |  |  |  |
| [ArXivMath 08/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--august.md) | 92.98% | 95% CI 86.35–99.61% | 2 / 11 | 4.82usd per problem | — | [matharena · arxiv--august](https://matharena.ai/competition_tables/arxiv--august) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.98 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 4.82 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--august predicted false Reasoning effort max sourceModelName GPT-6 Sol (max) sourceModelVersion openai/gpt-6-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 86.35–99.61% confidenceInterval95 6.63 problems 57 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--august) Source matharena Original model label GPT-6 Sol (max) Source version arxiv--august Observed 2026-10-21T16:52:30.283Z Evidence SHA-256 `5924e5ecc15ff087ec1504b1e4ac44408b005c4f70e81e0d2184f930383a963c` Source locator `/table/rows/1` |  |  |  |  |  |  |  |
| [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 81.91% | SE ±1.5 pp | 7 / 40 | $11.67 | 2,771.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.909 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.672651 \| USD \| per_test \| Lower is better \| \| `latency` \| 2771.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.5 pp sourcePopulation 64 stderr 1.502 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / creative writing](https://benchize.com/benchmarks/lmarena%3Atext%3Acreative_writing.md) | 1,377.56 | — | 75 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1377.5590711286704 \| rating \| reported \| Higher is better \| ### Configuration category creative_writing config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1365.6370545344307 rank 135 upper 1389.4810877229102 variance 37.000131019924474 votes 2926 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/1356/row` |  |  |  |  |  |  |  |
| [Arena text / industry medicine and healthcare](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_medicine_and_healthcare.md) | 1,366.17 | — | 110 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1366.1725777973402 \| rating \| reported \| Higher is better \| ### Configuration category industry_medicine_and_healthcare config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1346.9747437511671 rank 194 upper 1385.3704118435132 variance 95.94189324091352 votes 1051 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/6506/row` |  |  |  |  |  |  |  |
| [Arena text / spanish](https://benchize.com/benchmarks/lmarena%3Atext%3Aspanish.md) | 1,386.59 | — | 94 / 134 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1386.5944542999337 \| rating \| reported \| Higher is better \| ### Configuration category spanish config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1360.673832263662 rank 162 upper 1412.5150763362053 variance 174.90195212501203 votes 552 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/10926/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $2.11 | 762.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.10689 \| USD \| per_test \| Lower is better \| \| `latency` \| 762.124 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 19.3% | SE ±0.93 pp | 32 / 50 | $1.62 | 485.21source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.298 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.624618 \| USD \| per_test \| Lower is better \| \| `latency` \| 485.214 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.93 pp sourcePopulation 80 stderr 0.93 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [IOI / ioi2026](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2026.md) | 68.5% | SE ±18.23 pp | 9 / 29 | $2.71 | 1,675.84source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.708373 \| USD \| per_test \| Lower is better \| \| `latency` \| 1675.844 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±18.23 pp sourcePopulation 44 stderr 18.233 tokenTotals {"cache_read_tokens":3583111,"cache_write_tokens":266327,"input_tokens":3849795,"output_tokens":158823,"reasoning_tokens":125568} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2026/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aimmigration.md) | 36.36% | SE ±15.21 pp | 9 / 50 | $6.66 | 1,914.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.364 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.6645 \| USD \| per_test \| Lower is better \| \| `latency` \| 1914.142 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.21 pp sourcePopulation 82 stderr 15.212 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/immigration/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / spanish](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aspanish.md) | 1,447.03 | — | 47 / 134 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1447.0273181547122 \| rating \| reported \| Higher is better \| ### Configuration category spanish config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1421.0348433599593 rank 77 upper 1473.0197929494652 variance 175.87296323894833 votes 552 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/10841/row` |  |  |  |  |  |  |  |
| [Arena webdev / webdev-react](https://benchize.com/benchmarks/lmarena%3Awebdev%3Awebdev-react.md) | 1,685.26 | — | 10 / 81 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1685.2566972013929 \| rating \| reported \| Higher is better \| ### Configuration category webdev-react config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1672.1441569171839 rank 11 upper 1698.3692374856018 variance 44.75870254778267 votes 2719 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/517/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 88.74% | SE ±1.85 pp | 19 / 49 | $2.32 | 799.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.737 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.317085 \| USD \| per_test \| Lower is better \| \| `latency` \| 799.56 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 81 stderr 1.85 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / industry business and management and financial operations](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_business_and_management_and_financial_operations.md) | 1,380.05 | — | 97 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1380.05170902791 \| rating \| reported \| Higher is better \| ### Configuration category industry_business_and_management_and_financial_operations config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1367.6657346882007 rank 170 upper 1392.437683367619 variance 39.93596378477212 votes 2513 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/4452/row` |  |  |  |  |  |  |  |
| [BrokenArXiv 05/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--may.md) | 87% | 95% CI 80.41–93.59% | 5 / 20 | 0.33usd per problem | — | [matharena · arxiv_false--may](https://matharena.ai/competition_tables/arxiv_false--may) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.33 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--may predicted false Reasoning effort max sourceModelName GPT-6 Sol (max) sourceModelVersion openai/gpt-6-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 80.41–93.59% confidenceInterval95 6.59 problems 50 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--may) Source matharena Original model label GPT-6 Sol (max) Source version arxiv_false--may Observed 2026-10-21T16:52:24.785Z Evidence SHA-256 `b317fca30109e8c78ed5d2b835fe227b8cfbee3b6f9cb6d7aaeadca866660969` Source locator `/table/rows/4` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $4.21 | 754.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.210018 \| USD \| per_test \| Lower is better \| \| `latency` \| 754.62 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 28.85% | SE ±3.15 pp | 36 / 50 | $4.9 | 1,449.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.846 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.897717 \| USD \| per_test \| Lower is better \| \| `latency` \| 1449.451 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.15 pp sourcePopulation 82 stderr 3.149 tokenTotals {"cache_read_tokens":634566483,"cache_write_tokens":73881332,"input_tokens":709313788,"output_tokens":26332564,"reasoning_tokens":22035035} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 81.75% | SE ±2.44 pp | 32 / 49 | $2.96 | 751.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.746 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.96028 \| USD \| per_test \| Lower is better \| \| `latency` \| 751.328 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 81 stderr 2.438 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena vision style control / ocr](https://benchize.com/benchmarks/lmarena%3Avision_style_control%3Aocr.md) | 1,285.47 | — | 31 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1285.4738873981016 \| rating \| reported \| Higher is better \| ### Configuration category ocr config vision_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1272.5263568024943 rank 35 upper 1298.4214179937092 variance 43.63929339060235 votes 2285 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `38568936995f36625aae49e9ea05557d55c1b1df9b2954f5c50cee523f26fe92` Source locator `/rows/1053/row` |  |  |  |  |  |  |  |
| [SpireClimb](https://benchize.com/benchmarks/epoch-ai%3Aspireclimb.md) | 7.5% | SE ±2.14 pp | 3 / 7 | — | — | [epoch ai · SpireClimb](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.075 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.075 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion SpireClimb reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-sol_max ### Reported statistics Uncertainty Standard error: ±2.14 pp standardError 0.021443759811069392 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) Source epoch ai Original model label gpt-6-sol Source version SpireClimb Evaluated 2026-09-24T05:13:21.047Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `8bb2bda1fd232b776c132133a46df08f18ceabdc5552e9d61eb5802dc85f4a5e` Source locator `/rows/5` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass.md) | 53.05% | SE ±3.29 pp | 34 / 46 | $2.28 | 1,052.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.045 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.277613 \| USD \| per_test \| Lower is better \| \| `latency` \| 1052.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.29 pp sourcePopulation 73 stderr 3.29 tokenTotals {"cache_read_tokens":316266542,"cache_write_tokens":69503073,"input_tokens":386182808,"output_tokens":18761287,"reasoning_tokens":15767772} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 81.16% | SE ±2.29 pp | 32 / 49 | $4.21 | 754.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.164 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.210018 \| USD \| per_test \| Lower is better \| \| `latency` \| 754.62 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.29 pp sourcePopulation 81 stderr 2.292 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aprecedents.md) | 28.5% | SE ±1.71 pp | 19 / 50 | $3.9 | 839.99source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.499 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.902195 \| USD \| per_test \| Lower is better \| \| `latency` \| 839.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.71 pp sourcePopulation 80 stderr 1.713 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/precedents/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 41.67% | SE ±8.33 pp | 9 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 52 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label openai/gpt-6-sol Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 89.58% | — | 17 / 212 | 0.44usd per task | — | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8958333333333334 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.43889828333333336 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup openai-gpt-6-sol modelType CoT Reasoning effort max sourceModelVersion openai-gpt-6-sol-max thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label GPT-6 Sol (Max) Source version v2 Observed 2026-10-21T16:34:20.750Z Evidence SHA-256 `bca68e12cc9a01954ae21b641844c1ecbabe146cb24b87f0f8e9b1db27433c29` Source locator `/evaluations/247` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 53.7% | SE ±1.85 pp | 7 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.704 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 52 stderr 1.852 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label openai/gpt-6-sol Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [EMB / ma](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ama.md) | 83.02% | SE ±2.86 pp | 5 / 47 | $1.17 | 677.53source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.021 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.171055 \| USD \| per_test \| Lower is better \| \| `latency` \| 677.534 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.86 pp sourcePopulation 76 stderr 2.861 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/ma/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / industry mathematical](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_mathematical.md) | 1,465.82 | — | 49 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1465.8217139461512 \| rating \| reported \| Higher is better \| ### Configuration category industry_mathematical config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1445.9441238323584 rank 68 upper 1485.699304059944 variance 102.8563905471087 votes 873 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/5980/row` |  |  |  |  |  |  |  |
| [Vals Science Index / terminal bench science](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Aterminal_bench_science.md) | 30% | SE ±5.52 pp | 6 / 17 | $5.82 | 4,930.43source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.821477 \| USD \| per_test \| Lower is better \| \| `latency` \| 4930.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.52 pp sourcePopulation 21 stderr 5.517 tokenTotals {"cache_read_tokens":1090637986,"cache_write_tokens":15843344,"input_tokens":2212998922,"output_tokens":14625198,"reasoning_tokens":5482665} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/terminal_bench_science/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [MysteryMechanism / overall](https://benchize.com/benchmarks/vals-ai%3Amysterymechanism%3Aoverall.md) | 30.18% | SE ±3.09 pp | 11 / 19 | $0.46 | 373.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.18 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.460931 \| USD \| per_test \| Lower is better \| \| `latency` \| 373.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.09 pp sourcePopulation 27 stderr 3.088 tokenTotals {"cache_read_tokens":64736689,"cache_write_tokens":8491566,"input_tokens":73446321,"output_tokens":6771421,"reasoning_tokens":5398618} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:04.766Z Evidence SHA-256 `1ef09cd0a0ee8690b0c07b2272057461242359b1680e24975ea42971e03acea9` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 40.74% | SE ±3.7 pp | 5 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.741 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 52 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label openai/gpt-6-sol Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 2.1 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Ahard.md) | 78.89% | SE ±2.94 pp | 6 / 43 | $1.11 | 886.67source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.889 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.106639 \| USD \| per_test \| Lower is better \| \| `latency` \| 886.667 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.94 pp sourcePopulation 76 stderr 2.94 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/hard/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text style control / industry software and it services](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_software_and_it_services.md) | 1,505.61 | — | 35 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1505.6059457434187 \| rating \| reported \| Higher is better \| ### Configuration category industry_software_and_it_services config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1496.4910204709543 rank 45 upper 1514.7208710158834 variance 21.627685366571516 votes 4975 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/6745/row` |  |  |  |  |  |  |  |
| [Arena text / hard prompts](https://benchize.com/benchmarks/lmarena%3Atext%3Ahard_prompts.md) | 1,421.56 | — | 77 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1421.555193185152 \| rating \| reported \| Higher is better \| ### Configuration category hard_prompts config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1414.245784484171 rank 131 upper 1428.8646018861332 variance 13.908116174553726 votes 8512 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/3581/row` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 53.62% | SE ±1.82 pp | 38 / 50 | $2.9 | 782.02source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.615 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.899374 \| USD \| per_test \| Lower is better \| \| `latency` \| 782.017 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.82 pp sourcePopulation 80 stderr 1.824 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 82.03% | SE ±1.94 pp | 28 / 49 | $0.08 | 81.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.034 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.083138 \| USD \| per_test \| Lower is better \| \| `latency` \| 81.135 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 109 stderr 1.942 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:00.004Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / english](https://benchize.com/benchmarks/lmarena%3Atext%3Aenglish.md) | 1,396 | — | 96 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1395.9985419262316 \| rating \| reported \| Higher is better \| ### Configuration category english config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1387.1793662929829 rank 169 upper 1404.8177175594803 variance 20.246958897774675 votes 5361 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/1805/row` |  |  |  |  |  |  |  |
| [OTIS Mock AIME 2024–2025](https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025.md) | 100% | SE ±0 pp | 1 / 123 | — | — | [epoch ai · OTIS Mock AIME 2024-2025](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-sol_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-22T19:59:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/11` |  |  |  |  |  |  |  |
| [Arena text / instruction following](https://benchize.com/benchmarks/lmarena%3Atext%3Ainstruction_following.md) | 1,413.48 | — | 63 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1413.4842471995496 \| rating \| reported \| Higher is better \| ### Configuration category instruction_following config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1404.2260503443463 rank 100 upper 1422.7424440547527 variance 22.31293188617513 votes 4626 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/7633/row` |  |  |  |  |  |  |  |
| [BrokenArXiv 06/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--june.md) | 95.37% | 95% CI 91.41–99.33% | 5 / 21 | 0.29usd per problem | — | [matharena · arxiv_false--june](https://matharena.ai/competition_tables/arxiv_false--june) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.37 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.29 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--june predicted false Reasoning effort max sourceModelName GPT-6 Sol (max) sourceModelVersion openai/gpt-6-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 91.41–99.33% confidenceInterval95 3.96 problems 54 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--june) Source matharena Original model label GPT-6 Sol (max) Source version arxiv_false--june Observed 2026-10-21T16:52:25.323Z Evidence SHA-256 `f6d7e2280d05706643d46ae7786bd3fad6d912e4893781c3f62c73deebf38459` Source locator `/table/rows/4` |  |  |  |  |  |  |  |
| [IOI / overall](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aoverall.md) | 82.61% | SE ±9.24 pp | 10 / 29 | $2.7 | 1,837.55source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.611 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.703348 \| USD \| per_test \| Lower is better \| \| `latency` \| 1837.553 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.24 pp sourcePopulation 44 stderr 9.24 tokenTotals {"cache_read_tokens":3736261,"cache_write_tokens":324992,"input_tokens":4061594,"output_tokens":147384,"reasoning_tokens":114768} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/overall/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Arena text / industry writing and literature and language](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_writing_and_literature_and_language.md) | 1,389.14 | — | 79 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1389.1388607443394 \| rating \| reported \| Higher is better \| ### Configuration category industry_writing_and_literature_and_language config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1378.62133777825 rank 133 upper 1399.6563837104288 variance 28.795906582757315 votes 3774 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/7250/row` |  |  |  |  |  |  |  |
| [Arena text style control / longer query](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Alonger_query.md) | 1,467.47 | — | 44 / 150 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1467.4749768966876 \| rating \| reported \| Higher is better \| ### Configuration category longer_query config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1458.766451090253 rank 64 upper 1476.1835027031223 variance 19.74208894620788 votes 5776 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/8581/row` |  |  |  |  |  |  |  |
| [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 60% | SE ±9.1 pp | 20 / 50 | $3.79 | 1,026.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.793295 \| USD \| per_test \| Lower is better \| \| `latency` \| 1026.417 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.1 pp sourcePopulation 82 stderr 9.097 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [FrontierMath Tiers 1–3 v2 (private)](https://benchize.com/benchmarks/epoch-ai%3Afrontiermath_tiers_1_3_v2.md) | 89.82% | SE ±1.79 pp | 5 / 49 | — | — | [epoch ai · FrontierMath-Tiers-1-3-v2-Private](https://epoch.ai/data/benchmark_data.zip#file=frontiermath_tiers_1_3_v2.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8982456140350877 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8982456140350877 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion FrontierMath-Tiers-1-3-v2-Private reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6-sol_max ### Reported statistics Uncertainty Standard error: ±1.79 pp standardError 0.01793969072128637 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=frontiermath_tiers_1_3_v2.csv) Source epoch ai Original model label gpt-6-sol Source version FrontierMath-Tiers-1-3-v2-Private Evaluated 2026-09-22T19:59:24.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `6e31ece7503f8f89dffe8af33010ed7d1d6f0aa3a80cff14fa79f9b34a7b258f` Source locator `/rows/10` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 78.72% | SE ±2.21 pp | 28 / 49 | $3.03 | 884.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.717 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.027072 \| USD \| per_test \| Lower is better \| \| `latency` \| 884.656 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.21 pp sourcePopulation 81 stderr 2.213 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [SAGE / PDEs](https://benchize.com/benchmarks/vals-ai%3Asage%3APDEs.md) | 36.51% | SE ±6.05 pp | 35 / 41 | $0.06 | 29.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.508 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.055244 \| USD \| per_test \| Lower is better \| \| `latency` \| 29.991 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.05 pp sourcePopulation 91 stderr 6.052 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/PDEs/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [ArXivMath 05/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--may.md) | 90% | 95% CI 84.63–95.37% | 3 / 21 | 0.27usd per problem | — | [matharena · arxiv--may](https://matharena.ai/competition_tables/arxiv--may) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.27 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--may predicted false Reasoning effort max sourceModelName GPT-6 Sol (max) sourceModelVersion openai/gpt-6-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 84.63–95.37% confidenceInterval95 5.37 problems 40 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--may) Source matharena Original model label GPT-6 Sol (max) Source version arxiv--may Observed 2026-10-21T16:52:29.369Z Evidence SHA-256 `758e1bba77e36f4ced2a51b4bc0d46bd956ed94ef6f57e93e1a974b75e76a4ce` Source locator `/table/rows/2` |  |  |  |  |  |  |  |
| [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 63.14% | SE ±7.39 pp | 12 / 47 | $0.94 | 548.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.142 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.938299 \| USD \| per_test \| Lower is better \| \| `latency` \| 548.146 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.39 pp sourcePopulation 76 stderr 7.389 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 91.69% | SE ±2.54 pp | 23 / 50 | $3.79 | 1,026.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.692 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.793295 \| USD \| per_test \| Lower is better \| \| `latency` \| 1026.417 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.54 pp sourcePopulation 82 stderr 2.536 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 32.5% | SE ±3.32 pp | 9 / 40 | $11.67 | 2,771.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 32.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.672651 \| USD \| per_test \| Lower is better \| \| `latency` \| 2771.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.32 pp sourcePopulation 64 stderr 3.32 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/openai~1gpt-6-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 0% | SE ±0 pp | 41 / 46 | $3.19 | 1,481.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.18507 \| USD \| per_test \| Lower is better \| \| `latency` \| 1481.667 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 73 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/openai~1gpt-6-sol` |  |  |  |  |  |  |  |

[Next page](https://benchize.com/models/gpt-6-sol-max.md?cursor=68b9ce79a86779762d17407fdb337a23fa3fc5f28985bf8e4ded2ac678126f05)

---
Cite as: GPT-6 SolMax — Benchize, https://benchize.com/models/gpt-6-sol-max (retrieved 2026-10-21)