# GPT-6.1 SolMax

> Source: Benchize (https://benchize.com) — https://benchize.com/models/gpt-6-1-sol-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gpt-6-1-sol-max.

OpenAI

Canonical URL: https://benchize.com/models/gpt-6-1-sol-max
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

OpenAI

## Benchmarks

| Benchmark | Score | ± | Rank | Cost | Latency | Source |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| [Finance Agent v2.1 / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Ageneral_qualitative_analysis.md) | 64.51% | SE ±5.89 pp | 14 / 18 | $2.21 | 1,752.83source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.511 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.213508 \| USD \| per_test \| Lower is better \| \| `latency` \| 1752.826 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.89 pp sourcePopulation 28 stderr 5.892 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/general_qualitative_analysis/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $2.11 | 2,309.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.114472 \| USD \| per_test \| Lower is better \| \| `latency` \| 2309.216 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Finance Agent v2.1 / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Afinancial_modeling.md) | 52.6% | SE ±6.8 pp | 2 / 18 | $1.2 | 950.53source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.602 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.199828 \| USD \| per_test \| Lower is better \| \| `latency` \| 950.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.8 pp sourcePopulation 28 stderr 6.803 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/financial_modeling/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / industry legal and government](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_legal_and_government.md) | 1,500.48 | — | 13 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1500.4751070286632 \| rating \| reported \| Higher is better \| ### Configuration category industry_legal_and_government config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1476.6372286483052 rank 14 upper 1524.312985409021 variance 147.92412783799043 votes 636 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/5121/row` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 12 / 14 | $3.76 | 2,352.11source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.758665 \| USD \| per_test \| Lower is better \| \| `latency` \| 2352.11 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/data_privacy_cybersecurity/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Finance Agent v2.1 / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aadjustments.md) | 54.72% | SE ±6.97 pp | 17 / 18 | $1.55 | 1,019.86source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.722 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.548504 \| USD \| per_test \| Lower is better \| \| `latency` \| 1019.859 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.97 pp sourcePopulation 28 stderr 6.97 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/adjustments/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 3 / 14 | $3.29 | 2,681.31source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.292441 \| USD \| per_test \| Lower is better \| \| `latency` \| 2681.314 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/environmental_esg/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Robo-bench / balance scale](https://benchize.com/benchmarks/vals-ai%3Arobo_bench%3Abalance_scale.md) | 100% | SE ±0 pp | 1 / 5 | $0.2 | 46.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_robo_bench.DzDVSjnk.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.2038 \| USD \| per_test \| Lower is better \| \| `latency` \| 46.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness codex maxOutputTokens Not reported provider openai reasoning true Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 7 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_robo_bench.DzDVSjnk.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:05.362Z Evidence SHA-256 `79018880e0e64e5ee8ea196976a86b4ef42ed28e4ec2284db5d302d8aa6226a5` Source locator `/tasks/balance_scale/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__energy_natural_resources.md) | 96.54% | SE ±1.05 pp | 6 / 14 | $5.97 | 2,751.12source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.535 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.971747 \| USD \| per_test \| Lower is better \| \| `latency` \| 2751.118 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.05 pp sourcePopulation 24 stderr 1.052 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [ARC-AGI-1](https://benchize.com/benchmarks/arc-prize%3Av1_Semi_Private.md) | 96.5% | — | 23 / 213 | 0.1usd per task | — | [arc prize · v1](https://arcprize.org/media/data/leaderboard/v1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.965 \| score_fraction \| v1_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.09543634000000001 \| usd_per_task \| v1_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v1_Semi_Private harness Not reported modelGroup openai-gpt-6-1-sol modelType CoT Reasoning effort max sourceModelVersion openai-gpt-6-1-sol-max thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v1.json) Source arc prize Original model label GPT-6.1 Sol (Max) Source version v1 Observed 2026-10-21T16:34:20.378Z Evidence SHA-256 `302d8267c0522bf450ec1deeac4d060e2cf2fb444ba0a4269014d9535de0bd1f` Source locator `/evaluations/253` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 8 / 14 | $5.1 | 2,687.46source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.097847 \| USD \| per_test \| Lower is better \| \| `latency` \| 2687.456 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/arbitration_international_dispute_resolution/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [OTIS Mock AIME 2024–2025](https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025.md) | 100% | SE ±0 pp | 1 / 123 | — | — | [epoch ai · OTIS Mock AIME 2024-2025](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1.0 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 1.0 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6.1-sol_max ### Reported statistics Uncertainty Standard error: ±0 pp standardError 0 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gpt-6.1-sol Source version OTIS Mock AIME 2024-2025 Evaluated 2026-09-29T22:00:34.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/7` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aimmigration.md) | 0% | SE ±0 pp | 8 / 14 | $1.54 | 1,745.38source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.539912 \| USD \| per_test \| Lower is better \| \| `latency` \| 1745.378 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/immigration/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / expert](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aexpert.md) | 1,517.03 | — | 18 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1517.031453779368 \| rating \| reported \| Higher is better \| ### Configuration category expert config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1494.9253133134077 rank 20 upper 1539.1375942453283 variance 127.21246513647857 votes 668 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/2490/row` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 2 / 14 | $3.05 | 2,123.43source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.053435 \| USD \| per_test \| Lower is better \| \| `latency` \| 2123.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/white_collar_defense_investigations/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__constitutional_civil_rights.md) | 83.99% | SE ±2.7 pp | 28 / 50 | $2.69 | 1,775.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.985 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.692913 \| USD \| per_test \| Lower is better \| \| `latency` \| 1775.958 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 82 stderr 2.699 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__constitutional_civil_rights/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Ahealthcare_life_sciences.md) | 0% | SE ±0 pp | 14 / 14 | $3.63 | 2,990.29source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.628876 \| USD \| per_test \| Lower is better \| \| `latency` \| 2990.288 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/healthcare_life_sciences/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 3.33% | SE ±3.33 pp | 38 / 46 | $1.95 | 1,823.19source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.949409 \| USD \| per_test \| Lower is better \| \| `latency` \| 1823.192 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.33 pp sourcePopulation 73 stderr 3.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena vision style control / diagram](https://benchize.com/benchmarks/lmarena%3Avision_style_control%3Adiagram.md) | 1,318.95 | — | 16 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1318.953262509024 \| rating \| reported \| Higher is better \| ### Configuration category diagram config vision_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1293.950268554255 rank 16 upper 1343.956256463793 variance 162.73757858199642 votes 573 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `38568936995f36625aae49e9ea05557d55c1b1df9b2954f5c50cee523f26fe92` Source locator `/rows/482/row` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__structured_finance_securitization.md) | 90.81% | SE ±1.64 pp | 7 / 14 | $3.67 | 2,638.61source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.806 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.667821 \| USD \| per_test \| Lower is better \| \| `latency` \| 2638.612 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 24 stderr 1.644 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text / coding](https://benchize.com/benchmarks/lmarena%3Atext%3Acoding.md) | 1,477.78 | — | 41 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1477.7770417609768 \| rating \| reported \| Higher is better \| ### Configuration category coding config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1462.022726245962 rank 59 upper 1493.5313572759917 variance 64.61046933773753 votes 1462 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/868/row` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acorporate_governance.md) | 10% | SE ±10 pp | 7 / 14 | $4.1 | 2,408.28source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.104081 \| USD \| per_test \| Lower is better \| \| `latency` \| 2408.28 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/corporate_governance/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $8.05 | 4,209.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.046596 \| USD \| per_test \| Lower is better \| \| `latency` \| 4209.854 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 85.19% | SE ±2.07 pp | 17 / 49 | $2.3 | 1,588.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.185 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.297079 \| USD \| per_test \| Lower is better \| \| `latency` \| 1588.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.07 pp sourcePopulation 81 stderr 2.065 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 84.46% | SE ±2.24 pp | 13 / 50 | $3.38 | 2,341.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.458 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.38256 \| USD \| per_test \| Lower is better \| \| `latency` \| 2341.734 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.24 pp sourcePopulation 82 stderr 2.235 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aenergy_natural_resources.md) | 30% | SE ±20 pp | 6 / 14 | $5.97 | 2,751.12source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.971747 \| USD \| per_test \| Lower is better \| \| `latency` \| 2751.118 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/energy_natural_resources/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Finance Agent v2.1 / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aearnings_analysis.md) | 83.85% | SE ±4.95 pp | 12 / 18 | $0.48 | 466.59source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.851 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.481907 \| USD \| per_test \| Lower is better \| \| `latency` \| 466.585 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.95 pp sourcePopulation 28 stderr 4.946 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/earnings_analysis/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__funds_asset_management.md) | 89.94% | SE ±1.63 pp | 4 / 14 | $3.95 | 2,848.4source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.942 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.952966 \| USD \| per_test \| Lower is better \| \| `latency` \| 2848.396 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 24 stderr 1.626 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__funds_asset_management/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Astructured_finance_securitization.md) | 0% | SE ±0 pp | 9 / 14 | $3.67 | 2,638.61source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.667821 \| USD \| per_test \| Lower is better \| \| `latency` \| 2638.612 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/structured_finance_securitization/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 64.8% | SE ±7.93 pp | 11 / 46 | $1.3 | 1,316.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.799 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.303605 \| USD \| per_test \| Lower is better \| \| `latency` \| 1316.356 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.93 pp sourcePopulation 73 stderr 7.933 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 20% | SE ±7.43 pp | 35 / 46 | $1.02 | 945.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.02288 \| USD \| per_test \| Lower is better \| \| `latency` \| 945.988 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.43 pp sourcePopulation 73 stderr 7.428 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Humanity's Sixth Sense](https://benchize.com/benchmarks/scale%3Ahss.md) | 46.6source score | CI 42.8–50.4 | 3 / 24 | — | — | [scale](https://labs.scale.com/leaderboard/hss) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 46.6 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort max Model configuration GPT-6.1-Sol (max) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 42.8–50.4 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 3.8 ### Source [Original source](https://labs.scale.com/leaderboard/hss) Source scale Original model label GPT-6.1-Sol (max) Observed 2026-10-21T16:35:26.953Z Evidence SHA-256 `0b7d37655e8886c99f8325ed125eafaa4f89312cd44e1a783ac0c49b45db842b` Source locator `/rows/2` |  |  |  |  |  |  |  |
| [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 21.75% | SE ±2.98 pp | 31 / 46 | $1.32 | 1,213.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.754 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.323601 \| USD \| per_test \| Lower is better \| \| `latency` \| 1213.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.98 pp sourcePopulation 73 stderr 2.977 tokenTotals {"cache_read_tokens":103439338,"cache_write_tokens":45805894,"input_tokens":149308416,"output_tokens":12717430,"reasoning_tokens":9488118} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text / expert](https://benchize.com/benchmarks/lmarena%3Atext%3Aexpert.md) | 1,488.75 | — | 33 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1488.7516184950098 \| rating \| reported \| Higher is better \| ### Configuration category expert config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1466.700483439098 rank 39 upper 1510.8027535509214 variance 126.58018215230989 votes 668 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/2509/row` |  |  |  |  |  |  |  |
| [ClaimBench / non supporting citation](https://benchize.com/benchmarks/vals-ai%3Aclaimbench%3Anon_supporting_citation.md) | 100% | SE ±0 pp | 1 / 5 | $0.03 | 12.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.029973 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.071575 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 19 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:56.452Z Evidence SHA-256 `c0b96e43a97ed7a9dbb35a06b2ff59446cf8fd70f6d50005b8c58aa860f229d8` Source locator `/tasks/non_supporting_citation/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 84.55% | SE ±1.95 pp | 13 / 49 | $4.3 | 3,159.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.548 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.299795 \| USD \| per_test \| Lower is better \| \| `latency` \| 3159.604 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.95 pp sourcePopulation 81 stderr 1.954 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Abanking_finance.md) | 10% | SE ±10 pp | 3 / 14 | $3.43 | 2,181.91source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.430828 \| USD \| per_test \| Lower is better \| \| `latency` \| 2181.91 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/banking_finance/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 91.73% | SE ±1.69 pp | 12 / 14 | $4.97 | 2,621.26source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.729 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.966411 \| USD \| per_test \| Lower is better \| \| `latency` \| 2621.26 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 24 stderr 1.692 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [SRE Bench / overall](https://benchize.com/benchmarks/vals-ai%3Asrebench%3Aoverall.md) | 50.76% | SE ±3.1 pp | 2 / 24 | $2.69 | 1,952.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sre_bench.Bd0uBJxH.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.763 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.685175 \| USD \| per_test \| Lower is better \| \| `latency` \| 1952.046 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.1 pp sourcePopulation 35 stderr 3.095 tokenTotals {"cache_read_tokens":2003867049,"cache_write_tokens":56528756,"input_tokens":2060874823,"output_tokens":11701048,"reasoning_tokens":6536861} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sre_bench.Bd0uBJxH.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:15.724Z Evidence SHA-256 `a10b2c964f5878b0fc21c888703f311a5ef7d7a74fc3c529cab3ddd9fa474e2b` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 40% | SE ±11.55 pp | 7 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±11.55 pp sourcePopulation 52 stderr 11.547 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Vals Index / vibe code bench](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Avibe_code_bench.md) | 88.93% | SE ±1.89 pp | 6 / 14 | $6.23 | 4,489.76source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.931 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.234794 \| USD \| per_test \| Lower is better \| \| `latency` \| 4489.755 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.89 pp sourcePopulation 23 stderr 1.893 tokenTotals {"cache_read_tokens":1056860976,"cache_write_tokens":16623336,"input_tokens":1073502468,"output_tokens":10186788,"reasoning_tokens":5383848} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/vibe_code_bench/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 52.86% | SE ±6.01 pp | 2 / 30 | $3.44 | 5,003.99source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.857 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.436948 \| USD \| per_test \| Lower is better \| \| `latency` \| 5003.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.01 pp sourcePopulation 41 stderr 6.009 tokenTotals {"cache_read_tokens":317832491,"cache_write_tokens":19972804,"input_tokens":675620471,"output_tokens":20718878,"reasoning_tokens":9185469} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__real_estate.md) | 96.21% | SE ±1.03 pp | 3 / 14 | $5.11 | 2,926.52source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.21 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.105426 \| USD \| per_test \| Lower is better \| \| `latency` \| 2926.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.03 pp sourcePopulation 24 stderr 1.033 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__real_estate/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Vals Index / terminal bench 4](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Aterminal_bench_4.md) | 55.05% | SE ±1.82 pp | 5 / 14 | $1.72 | 2,711.67source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.051 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.72496 \| USD \| per_test \| Lower is better \| \| `latency` \| 2711.667 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.82 pp sourcePopulation 23 stderr 1.821 tokenTotals {"cache_read_tokens":172615332,"cache_write_tokens":9543063,"input_tokens":182186678,"output_tokens":6309120,"reasoning_tokens":4640611} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/terminal_bench_4/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 46.54% | SE ±3.46 pp | 25 / 41 | $0.06 | 50.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.538 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.061245 \| USD \| per_test \| Lower is better \| \| `latency` \| 50.219 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.46 pp sourcePopulation 91 stderr 3.462 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__banking_finance.md) | 93.28% | SE ±1.53 pp | 7 / 14 | $3.43 | 2,181.91source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.284 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.430828 \| USD \| per_test \| Lower is better \| \| `latency` \| 2181.91 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.53 pp sourcePopulation 24 stderr 1.532 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__banking_finance/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [ProgramBench / strict](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Astrict.md) | 3% | SE ±1.21 pp | 5 / 40 | $2.87 | 4,223.19source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.868128 \| USD \| per_test \| Lower is better \| \| `latency` \| 4223.187 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 64 stderr 1.209 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/strict/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Areal_estate.md) | 30% | SE ±20 pp | 1 / 14 | $5.11 | 2,926.52source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.105426 \| USD \| per_test \| Lower is better \| \| `latency` \| 2926.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/real_estate/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / overall](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aoverall.md) | 1,479.36 | — | 25 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1479.3637389057449 \| rating \| reported \| Higher is better \| ### Configuration category overall config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1471.7655739862869 rank 28 upper 1486.961903825203 variance 15.02869426382478 votes 7453 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/27/row` |  |  |  |  |  |  |  |
| [ArXivMath 09/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--september.md) | 91.51% | 95% CI 86.2–96.82% | 1 / 6 | 0.74usd per problem | — | [matharena · arxiv--september](https://matharena.ai/competition_tables/arxiv--september) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.51 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.74 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--september predicted false Reasoning effort max sourceModelName GPT-6.1 Sol (max) sourceModelVersion openai/gpt-61-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 86.2–96.82% confidenceInterval95 5.31 problems 53 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--september) Source matharena Original model label GPT-6.1 Sol (max) Source version arxiv--september Observed 2026-10-21T16:52:30.640Z Evidence SHA-256 `8973b7bb8ea675057eb4c03cedc79d40de49d0a1b15da24d387eaaad2d82b3be` Source locator `/table/rows/0` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Abankruptcy_restructuring.md) | 10% | SE ±10 pp | 4 / 14 | $3.87 | 2,414.9source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.872355 \| USD \| per_test \| Lower is better \| \| `latency` \| 2414.9 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/bankruptcy_restructuring/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text / industry mathematical](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_mathematical.md) | 1,513.36 | — | 11 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1513.3604000598543 \| rating \| reported \| Higher is better \| ### Configuration category industry_mathematical config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1485.2547548884572 rank 11 upper 1541.4660452312514 variance 205.63211201044354 votes 405 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/5923/row` |  |  |  |  |  |  |  |
| [Arena vision style control / ocr](https://benchize.com/benchmarks/lmarena%3Avision_style_control%3Aocr.md) | 1,296.73 | — | 24 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1296.7343184533854 \| rating \| reported \| Higher is better \| ### Configuration category ocr config vision_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1281.8714962546267 rank 25 upper 1311.5971406521442 variance 57.50510262452774 votes 1654 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `38568936995f36625aae49e9ea05557d55c1b1df9b2954f5c50cee523f26fe92` Source locator `/rows/1043/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 87.78% | SE ±0.39 pp | 16 / 49 | $3.93 | 2,594.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.784 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.933528 \| USD \| per_test \| Lower is better \| \| `latency` \| 2594.18 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.39 pp sourcePopulation 81 stderr 0.394 tokenTotals {"cache_read_tokens":1274733716,"cache_write_tokens":36236558,"input_tokens":1310994009,"output_tokens":13514557,"reasoning_tokens":5869694} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Code Migration v1.1 / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acode_quality.md) | 3.42% | SE ±0.04 pp | 9 / 17 | $9.51 | 5,954.4source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.419 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.505623 \| USD \| per_test \| Lower is better \| \| `latency` \| 5954.402 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.04 pp sourcePopulation 26 stderr 0.041 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/code_quality/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [SWE-bench Multimodal / lighthouse](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Alighthouse.md) | 24.07% | SE ±5.87 pp | 7 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 24.074 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.87 pp sourcePopulation 20 stderr 5.873 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/lighthouse/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 65.31% | SE ±7.53 pp | 9 / 47 | $0.76 | 784.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.313 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.759726 \| USD \| per_test \| Lower is better \| \| `latency` \| 784.133 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.53 pp sourcePopulation 76 stderr 7.529 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aoverall.md) | 6.25% | SE ±1.83 pp | 12 / 14 | $3.54 | 2,425.63source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.536302 \| USD \| per_test \| Lower is better \| \| `latency` \| 2425.625 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 24 stderr 1.83 tokenTotals {"cache_read_tokens":1191414163,"cache_write_tokens":36281197,"input_tokens":1227717408,"output_tokens":11158751,"reasoning_tokens":3901277} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [ClaimBench / numeric discrepancy](https://benchize.com/benchmarks/vals-ai%3Aclaimbench%3Anumeric_discrepancy.md) | 100% | SE ±0 pp | 1 / 5 | $0.03 | 10.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.029731 \| USD \| per_test \| Lower is better \| \| `latency` \| 10.514776 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 19 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:56.452Z Evidence SHA-256 `c0b96e43a97ed7a9dbb35a06b2ff59446cf8fd70f6d50005b8c58aa860f229d8` Source locator `/tasks/numeric_discrepancy/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text / math](https://benchize.com/benchmarks/lmarena%3Atext%3Amath.md) | 1,488.9 | — | 19 / 149 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1488.8988057953748 \| rating \| reported \| Higher is better \| ### Configuration category math config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1460.5534996474128 rank 22 upper 1517.244111943337 variance 209.15397460293977 votes 390 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/8934/row` |  |  |  |  |  |  |  |
| [Code Migration v1.1 / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acli.md) | 74.77% | SE ±2.14 pp | 4 / 17 | $12.22 | 7,270.97source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.765 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.223041 \| USD \| per_test \| Lower is better \| \| `latency` \| 7270.974 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.14 pp sourcePopulation 26 stderr 2.139 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/cli/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 0% | SE ±0 pp | 18 / 49 | $3.98 | 3,143.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.981076 \| USD \| per_test \| Lower is better \| \| `latency` \| 3143.702 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 61.71% | SE ±7.02 pp | 15 / 46 | $1.2 | 1,028.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.713 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.20312 \| USD \| per_test \| Lower is better \| \| `latency` \| 1028.913 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.02 pp sourcePopulation 73 stderr 7.024 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena vision / homework (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Ahomework--config-vision.md) | 1,355.61 | — | 3 / 73 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1355.605381138743 \| rating \| reported \| Higher is better \| ### Configuration category homework config vision Reasoning effort max thinkingEnabled true ### Reported statistics lower 1324.5838015371398 rank 3 upper 1386.6269607403465 variance 250.51378809384866 votes 368 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/810/row` |  |  |  |  |  |  |  |
| [Arena vision style control / creative writing vision](https://benchize.com/benchmarks/lmarena%3Avision_style_control%3Acreative_writing_vision.md) | 1,311.47 | — | 3 / 70 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1311.469515688962 \| rating \| reported \| Higher is better \| ### Configuration category creative_writing_vision config vision_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1271.1503837316532 rank 3 upper 1351.7886476462706 variance 423.1810043188588 votes 229 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `38568936995f36625aae49e9ea05557d55c1b1df9b2954f5c50cee523f26fe92` Source locator `/rows/365/row` |  |  |  |  |  |  |  |
| [APEX-Agents](https://benchize.com/benchmarks/mercor%3Aapex-agents%3A1.1.md) | 60% | — | 11 / 38 | — | — | [mercor · 1.1](https://www.mercor.com/apex/apex-agents-leaderboard/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 60 \| percent \| task_completion_pass_at_1 \| Higher is better \| \| `mean_score` \| 73.2 \| percent \| rubric_mean \| Higher is better \| ### Configuration datasetUrl https://huggingface.co/datasets/mercor/apex-agents-v1.1 datasetVersion 1.1 harness loop_truncated_tools_agent Reasoning effort max ### Reported statistics reportedErrors {"mean_score":4.6,"pass_1":5.6} sampleSize 954 ### Source [Original source](https://www.mercor.com/apex/apex-agents-leaderboard/) Source mercor Original model label GPT-6.1 Sol Source version 1.1 Observed 2026-10-21T16:33:27.509Z Evidence SHA-256 `359c569c3b0b62ab375895eabf3c45846fa45551d419d5c9d6a4c9117db78184` Source locator `/__NEXT_DATA__/props/pageProps/leaderboardData/50` |  |  |  |  |  |  |  |
| [SWE-bench Multimodal / prettier](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Aprettier.md) | 61.54% | SE ±14.04 pp | 4 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.538 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.04 pp sourcePopulation 20 stderr 14.044 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/prettier/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Terminal-Bench-Science / engineering sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aengineering_sciences.md) | 33.33% | SE ±16.67 pp | 3 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.333 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±16.67 pp sourcePopulation 41 stderr 16.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/engineering_sciences/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__capital_markets.md) | 94.31% | SE ±1.45 pp | 7 / 14 | $2.74 | 1,917.71source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.314 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.741122 \| USD \| per_test \| Lower is better \| \| `latency` \| 1917.712 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.45 pp sourcePopulation 24 stderr 1.453 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__capital_markets/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aantitrust_competition.md) | 10% | SE ±10 pp | 4 / 14 | $2.77 | 2,421.53source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.7654 \| USD \| per_test \| Lower is better \| \| `latency` \| 2421.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/antitrust_competition/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__antitrust_competition.md) | 90.76% | SE ±1.75 pp | 9 / 14 | $2.77 | 2,421.53source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.761 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.7654 \| USD \| per_test \| Lower is better \| \| `latency` \| 2421.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 24 stderr 1.746 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__antitrust_competition/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Finance Agent v2.1 / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aall_pass.md) | 48.2% | SE ±2.24 pp | 12 / 18 | $1.63 | 1,145.08source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.195 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.626543 \| USD \| per_test \| Lower is better \| \| `latency` \| 1145.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.24 pp sourcePopulation 28 stderr 2.238 tokenTotals {"cache_read_tokens":70737183,"cache_write_tokens":182747233,"input_tokens":253535763,"output_tokens":21994277,"reasoning_tokens":16385214} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/all_pass/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $3.84 | 2,139.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.837556 \| USD \| per_test \| Lower is better \| \| `latency` \| 2139.946 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Afunds_asset_management.md) | 0% | SE ±0 pp | 10 / 14 | $3.95 | 2,848.4source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.952966 \| USD \| per_test \| Lower is better \| \| `latency` \| 2848.396 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/funds_asset_management/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / french](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Afrench.md) | 1,505.62 | — | 18 / 132 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1505.6249501716288 \| rating \| reported \| Higher is better \| ### Configuration category french config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1471.1603508290373 rank 21 upper 1540.0895495142208 variance 309.20768991368436 votes 303 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/2858/row` |  |  |  |  |  |  |  |
| [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 63.5% | SE ±2.81 pp | 5 / 47 | $8.28 | 7,039.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.496 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.28157 \| USD \| per_test \| Lower is better \| \| `latency` \| 7039.569 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.81 pp sourcePopulation 79 stderr 2.811 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Vals Science Index / terminal bench science](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Aterminal_bench_science.md) | 52.86% | SE ±6.01 pp | 2 / 17 | $3.44 | 5,003.99source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.857 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.436948 \| USD \| per_test \| Lower is better \| \| `latency` \| 5003.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.01 pp sourcePopulation 21 stderr 6.009 tokenTotals {"cache_read_tokens":317832491,"cache_write_tokens":19972804,"input_tokens":675620471,"output_tokens":20718878,"reasoning_tokens":9185469} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/terminal_bench_science/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__employment_labor.md) | 81.27% | SE ±2.39 pp | 19 / 49 | $2.44 | 2,260.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.273 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.442027 \| USD \| per_test \| Lower is better \| \| `latency` \| 2260.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.39 pp sourcePopulation 81 stderr 2.392 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__employment_labor/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena webdev / image to webdev](https://benchize.com/benchmarks/lmarena%3Awebdev%3Aimage_to_webdev.md) | 1,701.74 | — | 5 / 42 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1701.7353202057543 \| rating \| reported \| Higher is better \| ### Configuration category image_to_webdev config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1686.6346767723207 rank 5 upper 1716.8359636391883 variance 59.360113630609725 votes 1916 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/152/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__structured_finance_securitization.md) | 83.81% | SE ±2.08 pp | 18 / 49 | $3.98 | 2,423.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.81 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.983856 \| USD \| per_test \| Lower is better \| \| `latency` \| 2423.536 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.08 pp sourcePopulation 81 stderr 2.079 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / spanish](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aspanish.md) | 1,471.66 | — | 24 / 134 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1471.6580805563358 \| rating \| reported \| Higher is better \| ### Configuration category spanish config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1435.4428954253217 rank 31 upper 1507.8732656873503 variance 341.4170749425503 votes 257 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/10795/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 92.95% | SE ±1.45 pp | 12 / 49 | $4.42 | 3,062.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.949 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.419748 \| USD \| per_test \| Lower is better \| \| `latency` \| 3062.552 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.45 pp sourcePopulation 81 stderr 1.452 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text / hard prompts](https://benchize.com/benchmarks/lmarena%3Atext%3Ahard_prompts.md) | 1,462.87 | — | 39 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1462.868047560597 \| rating \| reported \| Higher is better \| ### Configuration category hard_prompts config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1453.2704346307823 rank 51 upper 1472.4656604904117 variance 23.978956498067816 votes 4318 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/3501/row` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Atax.md) | 0% | SE ±0 pp | 7 / 14 | $4.7 | 2,757.38source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.69923 \| USD \| per_test \| Lower is better \| \| `latency` \| 2757.38 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/tax/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena vision / creative writing vision (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Acreative_writing_vision--config-vision.md) | 1,300.66 | — | 18 / 70 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1300.664434956268 \| rating \| reported \| Higher is better \| ### Configuration category creative_writing_vision config vision Reasoning effort max thinkingEnabled true ### Reported statistics lower 1260.811670694409 rank 19 upper 1340.517199218127 variance 413.4478315257123 votes 229 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/381/row` |  |  |  |  |  |  |  |
| [Arena text style control / english](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aenglish.md) | 1,491.19 | — | 16 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1491.1858253518124 \| rating \| reported \| Higher is better \| ### Configuration category english config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1479.4772835957801 rank 18 upper 1502.8943671078446 variance 35.68695031018953 votes 2828 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/1654/row` |  |  |  |  |  |  |  |
| [Arena vision / overall (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Aoverall--config-vision.md) | 1,287.73 | — | 27 / 82 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1287.7313467775475 \| rating \| reported \| Higher is better \| ### Configuration category overall config vision Reasoning effort max thinkingEnabled true ### Reported statistics lower 1274.871343958036 rank 29 upper 1300.5913495970588 variance 43.051267822248334 votes 2330 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/28/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $2.3 | 1,588.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.297079 \| USD \| per_test \| Lower is better \| \| `latency` \| 1588.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Acomparables.md) | 42.62% | SE ±1.16 pp | 31 / 50 | $2.4 | 1,376.35source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.619 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.399668 \| USD \| per_test \| Lower is better \| \| `latency` \| 1376.349 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.16 pp sourcePopulation 80 stderr 1.155 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/comparables/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 89.49% | SE ±1.85 pp | 13 / 49 | $3.5 | 2,421.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.493 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.497776 \| USD \| per_test \| Lower is better \| \| `latency` \| 2421.988 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 81 stderr 1.849 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [BrokenArXiv 09/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--september.md) | 82.7% | 95% CI 75.5–89.9% | 1 / 6 | 1.1usd per problem | — | [matharena · arxiv_false--september](https://matharena.ai/competition_tables/arxiv_false--september) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.70 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 1.10 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--september predicted false Reasoning effort max sourceModelName GPT-6.1 Sol (max) sourceModelVersion openai/gpt-61-sol thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 75.5–89.9% confidenceInterval95 7.20 problems 53 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--september) Source matharena Original model label GPT-6.1 Sol (max) Source version arxiv_false--september Observed 2026-10-21T16:52:26.084Z Evidence SHA-256 `4431a852d035438c3451a917deda38fbe7f7e38a2da0903d97e836c3b8776de1` Source locator `/table/rows/0` |  |  |  |  |  |  |  |
| [SpireClimb](https://benchize.com/benchmarks/epoch-ai%3Aspireclimb.md) | 24.67% | SE ±3.71 pp | 2 / 7 | — | — | [epoch ai · SpireClimb](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.24666666666666667 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.24666666666666667 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion SpireClimb reasoningBudget Not reported Reasoning effort max sourceModelVersion gpt-6.1-sol_max ### Reported statistics Uncertainty Standard error: ±3.71 pp standardError 0.03712501046458055 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) Source epoch ai Original model label gpt-6.1-sol Source version SpireClimb Evaluated 2026-09-30T01:02:59.538Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `8bb2bda1fd232b776c132133a46df08f18ceabdc5552e9d61eb5802dc85f4a5e` Source locator `/rows/0` |  |  |  |  |  |  |  |
| [Arena text / industry writing and literature and language](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_writing_and_literature_and_language.md) | 1,443.85 | — | 37 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1443.8479382683356 \| rating \| reported \| Higher is better \| ### Configuration category industry_writing_and_literature_and_language config text Reasoning effort max thinkingEnabled true ### Reported statistics lower 1430.2445485455385 rank 49 upper 1457.4513279911325 variance 48.172379449550654 votes 2155 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/7166/row` |  |  |  |  |  |  |  |
| [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 70.77% | SE ±2.71 pp | 12 / 47 | $1.14 | 1,107.81source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.773 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.143199 \| USD \| per_test \| Lower is better \| \| `latency` \| 1107.811 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.71 pp sourcePopulation 76 stderr 2.712 tokenTotals {"cache_read_tokens":112922114,"cache_write_tokens":16703171,"input_tokens":129631093,"output_tokens":6056928,"reasoning_tokens":3495567} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [HLAB-Vals-1.1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__immigration.md) | 91.46% | SE ±1.69 pp | 8 / 14 | $1.54 | 1,745.38source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.455 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.539912 \| USD \| per_test \| Lower is better \| \| `latency` \| 1745.378 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 24 stderr 1.689 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__immigration/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 89.75% | SE ±1.81 pp | 29 / 49 | $3.58 | 2,246.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.753 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.575179 \| USD \| per_test \| Lower is better \| \| `latency` \| 2246.312 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.81 pp sourcePopulation 81 stderr 1.806 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__current_temporal_analysis.md) | 63.37% | SE ±8.38 pp | 23 / 46 | $2.13 | 1,908.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.366 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.13313 \| USD \| per_test \| Lower is better \| \| `latency` \| 1908.876 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.38 pp sourcePopulation 73 stderr 8.378 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__current_temporal_analysis/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Arena text style control / russian](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Arussian.md) | 1,488.51 | — | 24 / 150 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1488.5087810145387 \| rating \| reported \| Higher is better \| ### Configuration category russian config text_style_control Reasoning effort max thinkingEnabled true ### Reported statistics lower 1470.1769323735484 rank 25 upper 1506.8406296555286 variance 87.48152467125723 votes 1071 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label GPT-6.1 Sol (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/10408/row` |  |  |  |  |  |  |  |
| [Legal Research Bench / family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Afamily.md) | 18.18% | SE ±8.42 pp | 18 / 50 | $3.77 | 2,357.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.182 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.772956 \| USD \| per_test \| Lower is better \| \| `latency` \| 2357.557 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.42 pp sourcePopulation 82 stderr 8.417 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/family/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__business_commercial.md) | 84.16% | SE ±3.13 pp | 28 / 50 | $3.46 | 2,357.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.159 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.455558 \| USD \| per_test \| Lower is better \| \| `latency` \| 2357.851 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.13 pp sourcePopulation 82 stderr 3.13 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__business_commercial/openai~1gpt-6.1-sol` |  |  |  |  |  |  |  |

[Next page](https://benchize.com/models/gpt-6-1-sol-max.md?cursor=48cbd0cc9615a7cf469f00812c66f937f853d3d5bf2980ccee4880b268ef793e)

---
Cite as: GPT-6.1 SolMax — Benchize, https://benchize.com/models/gpt-6-1-sol-max (retrieved 2026-10-21)