# Public Benefits Bench v1.1 / overall > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall. Can AI help people navigate SNAP benefits? Canonical URL: https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Can AI help people navigate SNAP benefits? Professional work · Models tested: 34 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 76.93% | SE ±1.1 pp | $3.95 | 1,808.88source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.928 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.949099 \| USD \| per_test \| Lower is better \| \| `latency` \| 1808.881 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.1 pp sourcePopulation 50 stderr 1.096 tokenTotals {"cache_read_tokens":662589765,"cache_write_tokens":42269235,"input_tokens":704875690,"output_tokens":12509272,"reasoning_tokens":6498299} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-opus-5` | | | | | | | | 2 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 74.9% | SE ±1.13 pp | $7.48 | 2,882.4source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.481573 \| USD \| per_test \| Lower is better \| \| `latency` \| 2882.401 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.13 pp sourcePopulation 50 stderr 1.128 tokenTotals {"cache_read_tokens":685891147,"cache_write_tokens":78776117,"input_tokens":764685044,"output_tokens":11288197,"reasoning_tokens":5916781} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-fable-5-1` | | | | | | | | 3 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 70.64% | SE ±1.19 pp | $5.95 | 4,771.81source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.636 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.952407 \| USD \| per_test \| Lower is better \| \| `latency` \| 4771.81 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 50 stderr 1.185 tokenTotals {"cache_read_tokens":766997389,"cache_write_tokens":139155673,"input_tokens":906177236,"output_tokens":18318977,"reasoning_tokens":14012794} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-opus-5-5` | | | | | | | | 4 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 70.43% | SE ±1.19 pp | $4.59 | 1,276.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.433 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.585446 \| USD \| per_test \| Lower is better \| \| `latency` \| 1276.576 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 50 stderr 1.187 tokenTotals {"cache_read_tokens":236807281,"cache_write_tokens":26426081,"input_tokens":263243176,"output_tokens":9748421,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-fable-5` | | | | | | | | 5 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 70.23% | SE ±1.19 pp | $2.46 | 1,184.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.23 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.457181 \| USD \| per_test \| Lower is better \| \| `latency` \| 1184.49 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 50 stderr 1.189 tokenTotals {"cache_read_tokens":1106854548,"cache_write_tokens":0,"input_tokens":1156134294,"output_tokens":7333084,"reasoning_tokens":3034066} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/meta~1muse-wonder` | | | | | | | | 6 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 69.76% | SE ±0 pp | $2.79 | 1,566.79source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.756 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.793902 \| USD \| per_test \| Lower is better \| \| `latency` \| 1566.785 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 50 stderr 0 tokenTotals {"cache_read_tokens":559760847,"cache_write_tokens":null,"input_tokens":658538120,"output_tokens":6776808,"reasoning_tokens":3500703} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-4-argon` | | | | | | | | 7 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 68.95% | SE ±1.2 pp | $0.08 | 2,234.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.945 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.081936 \| USD \| per_test \| Lower is better \| \| `latency` \| 2234.344 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.2 pp sourcePopulation 50 stderr 1.204 tokenTotals {"cache_read_tokens":302628352,"cache_write_tokens":null,"input_tokens":322998308,"output_tokens":10223982,"reasoning_tokens":7494264} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-pro` | | | | | | | | 8 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 68.47% | SE ±1.21 pp | $0.59 | 535.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.471 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.587608 \| USD \| per_test \| Lower is better \| \| `latency` \| 535.162 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 50 stderr 1.209 tokenTotals {"cache_read_tokens":340605576,"cache_write_tokens":null,"input_tokens":388336719,"output_tokens":5740007,"reasoning_tokens":1080003} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/meta~1muse_spark_1_2` | | | | | | | | 9 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 68.13% | SE ±1.21 pp | $1.89 | 767.85source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.133 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.889795 \| USD \| per_test \| Lower is better \| \| `latency` \| 767.85 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 50 stderr 1.212 tokenTotals {"cache_read_tokens":130953865,"cache_write_tokens":21982282,"input_tokens":152942625,"output_tokens":9270173,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | 10 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 67.59% | SE ±1.22 pp | $0.03 | 1,255.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.591 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.031798 \| USD \| per_test \| Lower is better \| \| `latency` \| 1255.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.22 pp sourcePopulation 50 stderr 1.217 tokenTotals {"cache_read_tokens":332096768,"cache_write_tokens":null,"input_tokens":357795247,"output_tokens":9949392,"reasoning_tokens":6091752} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-flash` | | | | | | | | 11 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 67.19% | SE ±1.22 pp | $4.25 | 4,527.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.185 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.245484 \| USD \| per_test \| Lower is better \| \| `latency` \| 4527.815 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.22 pp sourcePopulation 50 stderr 1.221 tokenTotals {"cache_read_tokens":1108467518,"cache_write_tokens":185872526,"input_tokens":1294364352,"output_tokens":29003787,"reasoning_tokens":23360865} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | 12 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 66.51% | SE ±1.23 pp | $8.85 | 3,677.63source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.509 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.847882 \| USD \| per_test \| Lower is better \| \| `latency` \| 3677.634 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.23 pp sourcePopulation 50 stderr 1.228 tokenTotals {"cache_read_tokens":1350622720,"cache_write_tokens":null,"input_tokens":1506316388,"output_tokens":8001721,"reasoning_tokens":4848671} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-5.6-sol` | | | | | | | | 13 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 66.04% | SE ±1.23 pp | $1.29 | 1,268.85source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.035 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.290695 \| USD \| per_test \| Lower is better \| \| `latency` \| 1268.852 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.23 pp sourcePopulation 50 stderr 1.232 tokenTotals {"cache_read_tokens":331877261,"cache_write_tokens":41387480,"input_tokens":373274847,"output_tokens":12699538,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-sonnet-5` | | | | | | | | 14 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 65.29% | SE ±1.24 pp | $1.01 | 201.42source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.291 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.011271 \| USD \| per_test \| Lower is better \| \| `latency` \| 201.419 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.24 pp sourcePopulation 50 stderr 1.238 tokenTotals {"cache_read_tokens":456416879,"cache_write_tokens":null,"input_tokens":539398130,"output_tokens":5287711,"reasoning_tokens":3569491} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-3.8-flash` | | | | | | | | 15 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-max.md)Max | 64.28% | SE ±1.25 pp | $0.07 | 598.96source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.276 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.073234 \| USD \| per_test \| Lower is better \| \| `latency` \| 598.96 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.25 pp sourcePopulation 50 stderr 1.246 tokenTotals {"cache_read_tokens":305366398,"cache_write_tokens":null,"input_tokens":324083542,"output_tokens":7830427,"reasoning_tokens":5092304} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | 16 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 64.14% | SE ±1.25 pp | $0.25 | 691.44source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.141 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.2543 \| USD \| per_test \| Lower is better \| \| `latency` \| 691.436 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.25 pp sourcePopulation 50 stderr 1.247 tokenTotals {"cache_read_tokens":256756972,"cache_write_tokens":0,"input_tokens":284714117,"output_tokens":4543318,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/minimax~1MiniMax-M3` | | | | | | | | 17 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 63.19% | SE ±1.25 pp | $0.53 | 2,192.48source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.194 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.525402 \| USD \| per_test \| Lower is better \| \| `latency` \| 2192.484 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.25 pp sourcePopulation 50 stderr 1.254 tokenTotals {"cache_read_tokens":443798683,"cache_write_tokens":78101633,"input_tokens":521915572,"output_tokens":37160233,"reasoning_tokens":32909550} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-haiku-5-5` | | | | | | | | 18 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-max.md)Max | 62.92% | SE ±1.26 pp | $0.46 | 922.4source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.923 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.460453 \| USD \| per_test \| Lower is better \| \| `latency` \| 922.395 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.26 pp sourcePopulation 50 stderr 1.256 tokenTotals {"cache_read_tokens":396119296,"cache_write_tokens":null,"input_tokens":416776131,"output_tokens":3598874,"reasoning_tokens":1104352} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | 19 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-max.md)Max | 62.45% | SE ±1.26 pp | $1.11 | 883source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.449 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.105228 \| USD \| per_test \| Lower is better \| \| `latency` \| 882.995 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning true Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.26 pp sourcePopulation 50 stderr 1.26 tokenTotals {"cache_read_tokens":287033952,"cache_write_tokens":21907318,"input_tokens":308948823,"output_tokens":5727806,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-sonnet-4-6` | | | | | | | | 20 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 62.38% | SE ±1.26 pp | $1.2 | 1,171.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.382 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.199723 \| USD \| per_test \| Lower is better \| \| `latency` \| 1171.486 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.26 pp sourcePopulation 50 stderr 1.26 tokenTotals {"cache_read_tokens":594265088,"cache_write_tokens":null,"input_tokens":641043129,"output_tokens":5293935,"reasoning_tokens":2859562} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-5.6-terra` | | | | | | | | 21 | [Muse Spark 1.1](https://benchize.com/models/muse-spark-1-1-xhigh.md)Extra high | 61.3% | SE ±1.27 pp | $0.28 | 395.05source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.299 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.278428 \| USD \| per_test \| Lower is better \| \| `latency` \| 395.054 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.27 pp sourcePopulation 50 stderr 1.267 tokenTotals {"cache_read_tokens":199788960,"cache_write_tokens":null,"input_tokens":213360122,"output_tokens":4024960,"reasoning_tokens":929960} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | 22 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 61.16% | SE ±1.27 pp | $0.27 | 2,319.75source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.164 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.270182 \| USD \| per_test \| Lower is better \| \| `latency` \| 2319.748 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.27 pp sourcePopulation 50 stderr 1.268 tokenTotals {"cache_read_tokens":1407739326,"cache_write_tokens":null,"input_tokens":1526164237,"output_tokens":8585020,"reasoning_tokens":5566100} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-5.6-luna` | | | | | | | | 23 | [GPT-5.5](https://benchize.com/models/gpt-5-5-xhigh.md)Extra high | 60.89% | SE ±1.27 pp | $3.97 | 2,377.31source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.893 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.971165 \| USD \| per_test \| Lower is better \| \| `latency` \| 2377.308 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.27 pp sourcePopulation 50 stderr 1.269 tokenTotals {"cache_read_tokens":790680064,"cache_write_tokens":null,"input_tokens":859486603,"output_tokens":5799841,"reasoning_tokens":3539368} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-5.5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-5.5` | | | | | | | | 24 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 59.47% | SE ±1.28 pp | $1.11 | 338.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.472 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.113099 \| USD \| per_test \| Lower is better \| \| `latency` \| 338.872 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 50 stderr 1.277 tokenTotals {"cache_read_tokens":489377006,"cache_write_tokens":null,"input_tokens":574078082,"output_tokens":6172733,"reasoning_tokens":4296597} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-3.5-flash` | | | | | | | | 25 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 59.34% | SE ±1.28 pp | $0.69 | 3,024.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.337 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.690795 \| USD \| per_test \| Lower is better \| \| `latency` \| 3024.455 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 50 stderr 1.278 tokenTotals {"cache_read_tokens":294009060,"cache_write_tokens":20960904,"input_tokens":315084952,"output_tokens":7684981,"reasoning_tokens":5458507} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-6.1-sol` | | | | | | | | 26 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 58.12% | SE ±1.28 pp | $0.5 | 1,639.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.119 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.504407 \| USD \| per_test \| Lower is better \| \| `latency` \| 1639.157 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 50 stderr 1.283 tokenTotals {"cache_read_tokens":708651520,"cache_write_tokens":null,"input_tokens":756809486,"output_tokens":12008540,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/stepfun~1step-5-preview` | | | | | | | | 27 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 57.65% | SE ±1.29 pp | $0.22 | 3,178.91source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.645 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.224168 \| USD \| per_test \| Lower is better \| \| `latency` \| 3178.908 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 50 stderr 1.285 tokenTotals {"cache_read_tokens":1725599246,"cache_write_tokens":99654947,"input_tokens":1825774915,"output_tokens":17789521,"reasoning_tokens":13913959} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-6-luna` | | | | | | | | 28 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 56.63% | SE ±1.29 pp | $2.49 | 1,862.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.631 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.491613 \| USD \| per_test \| Lower is better \| \| `latency` \| 1862.494 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 50 stderr 1.289 tokenTotals {"cache_read_tokens":1494255406,"cache_write_tokens":57401341,"input_tokens":1552257632,"output_tokens":7014313,"reasoning_tokens":3446365} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/openai~1gpt-6-sol` | | | | | | | | 29 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 56.56% | SE ±1.29 pp | $0.39 | 121.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.563 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.385496 \| USD \| per_test \| Lower is better \| \| `latency` \| 121.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 50 stderr 1.289 tokenTotals {"cache_read_tokens":85117031,"cache_write_tokens":null,"input_tokens":119300944,"output_tokens":3282758,"reasoning_tokens":2147005} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-3.6-flash` | | | | | | | | 30 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 53.79% | SE ±1.3 pp | $0.38 | 185.55source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.789 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.376768 \| USD \| per_test \| Lower is better \| \| `latency` \| 185.553 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 50 stderr 1.297 tokenTotals {"cache_read_tokens":53466726,"cache_write_tokens":null,"input_tokens":77067942,"output_tokens":2396732,"reasoning_tokens":1547863} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-3.1-pro-preview` | | | | | | | | 31 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 51.83% | SE ±1.3 pp | $0.1 | 69.26source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.827 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.09565 \| USD \| per_test \| Lower is better \| \| `latency` \| 69.26 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 50 stderr 1.3 tokenTotals {"cache_read_tokens":30028794,"cache_write_tokens":null,"input_tokens":84048754,"output_tokens":1957055,"reasoning_tokens":1249909} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/google~1gemini-3.5-flash-lite` | | | | | | | | 32 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 45.26% | SE ±1.3 pp | $0.07 | 68.33source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.264 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.067576 \| USD \| per_test \| Lower is better \| \| `latency` \| 68.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 50 stderr 1.295 tokenTotals {"cache_read_tokens":176220251,"cache_write_tokens":null,"input_tokens":227432781,"output_tokens":2367427,"reasoning_tokens":1625567} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | 33 | [Laguna M 1](https://benchize.com/models/laguna-m-1.md) | 43.84% | SE ±1.29 pp | $0.2 | 334.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.843 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.201829 \| USD \| per_test \| Lower is better \| \| `latency` \| 334.832 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 50 stderr 1.291 tokenTotals {"cache_read_tokens":365818068,"cache_write_tokens":null,"input_tokens":410949945,"output_tokens":2031370,"reasoning_tokens":485323} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label poolside/laguna-m.1 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/poolside~1laguna-m.1` | | | | | | | | 34 | [Laguna Xs 2](https://benchize.com/models/laguna-xs-2.md) | 41.41% | SE ±1.28 pp | $0.11 | 305.41source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.407 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.113029 \| USD \| per_test \| Lower is better \| \| `latency` \| 305.41 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 50 stderr 1.281 tokenTotals {"cache_read_tokens":318477696,"cache_write_tokens":null,"input_tokens":415335868,"output_tokens":1934540,"reasoning_tokens":591448} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label poolside/laguna-xs.2 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/poolside~1laguna-xs.2` | | | | | | | --- Cite as: Public Benefits Bench v1.1 / overall — Benchize, https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall (retrieved 2026-10-21)