# CyberBench v1.1 / poc > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Canonical URL: https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Safety & robustness · Models tested: 28 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 75% | SE ±5.59 pp | $5.72 | 1,076source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.721935 \| USD \| per_test \| Lower is better \| \| `latency` \| 1075.999 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.59 pp sourcePopulation 47 stderr 5.59 tokenTotals {"cache_read_tokens":1248055208,"cache_write_tokens":0,"input_tokens":1260598106,"output_tokens":2176673,"reasoning_tokens":1260474} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/meta~1muse-wonder` | | | | | | | | 2 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 70% | SE ±5.92 pp | $5.23 | 1,006.18source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.22941 \| USD \| per_test \| Lower is better \| \| `latency` \| 1006.177 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.92 pp sourcePopulation 47 stderr 5.916 tokenTotals {"cache_read_tokens":507918819,"cache_write_tokens":null,"input_tokens":549780427,"output_tokens":2236721,"reasoning_tokens":1960849} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/google~1gemini-4-argon` | | | | | | | | 3 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 66.67% | SE ±6.09 pp | $1.94 | 782.38source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.93889 \| USD \| per_test \| Lower is better \| \| `latency` \| 782.382 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.09 pp sourcePopulation 47 stderr 6.086 tokenTotals {"cache_read_tokens":308566663,"cache_write_tokens":7166263,"input_tokens":315942272,"output_tokens":1837590,"reasoning_tokens":619068} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-6-sol` | | | | | | | | 4 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 65% | SE ±6.16 pp | $0.07 | 2,117.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.06607 \| USD \| per_test \| Lower is better \| \| `latency` \| 2117.341 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.16 pp sourcePopulation 47 stderr 6.158 tokenTotals {"cache_read_tokens":505599872,"cache_write_tokens":null,"input_tokens":515875137,"output_tokens":3964234,"reasoning_tokens":3132769} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/xiaomi~1mimo-v2.6-flash` | | | | | | | | 4 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 65% | SE ±6.16 pp | $0.23 | 1,277.38source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.231144 \| USD \| per_test \| Lower is better \| \| `latency` \| 1277.384 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.16 pp sourcePopulation 47 stderr 6.158 tokenTotals {"cache_read_tokens":668440014,"cache_write_tokens":9760678,"input_tokens":678263901,"output_tokens":3824645,"reasoning_tokens":1961289} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-6-luna` | | | | | | | | 6 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 63.33% | SE ±6.22 pp | $5.04 | 748.95source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.041798 \| USD \| per_test \| Lower is better \| \| `latency` \| 748.947 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.22 pp sourcePopulation 47 stderr 6.221 tokenTotals {"cache_read_tokens":428829703,"cache_write_tokens":6869793,"input_tokens":435759534,"output_tokens":900060,"reasoning_tokens":450179} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-5.6-sol` | | | | | | | | 6 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 63.33% | SE ±6.22 pp | $0.75 | 1,452.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.746458 \| USD \| per_test \| Lower is better \| \| `latency` \| 1452.463 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.22 pp sourcePopulation 47 stderr 6.221 tokenTotals {"cache_read_tokens":1116211902,"cache_write_tokens":11891009,"input_tokens":1128169501,"output_tokens":2502985,"reasoning_tokens":967423} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-5.6-luna` | | | | | | | | 8 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 61.67% | SE ±6.28 pp | $1.52 | 2,149.07source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.522459 \| USD \| per_test \| Lower is better \| \| `latency` \| 2149.069 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.28 pp sourcePopulation 47 stderr 6.277 tokenTotals {"cache_read_tokens":1459355823,"cache_write_tokens":15473963,"input_tokens":1474844338,"output_tokens":8290468,"reasoning_tokens":6113266} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-haiku-5-5` | | | | | | | | 8 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 61.67% | SE ±6.28 pp | $0.09 | 1,104.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085565 \| USD \| per_test \| Lower is better \| \| `latency` \| 1104.199 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.28 pp sourcePopulation 47 stderr 6.277 tokenTotals {"cache_read_tokens":617766272,"cache_write_tokens":null,"input_tokens":622988275,"output_tokens":2636167,"reasoning_tokens":1934806} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/deepseek~1deepseek-v4.1-flash` | | | | | | | | 10 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 60% | SE ±6.33 pp | $0.9 | 1,954.21source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.899129 \| USD \| per_test \| Lower is better \| \| `latency` \| 1954.213 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.33 pp sourcePopulation 47 stderr 6.325 tokenTotals {"cache_read_tokens":819103744,"cache_write_tokens":null,"input_tokens":825046633,"output_tokens":2541046,"reasoning_tokens":1517436} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 10 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 60% | SE ±6.33 pp | $0.12 | 1,994.63source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.123983 \| USD \| per_test \| Lower is better \| \| `latency` \| 1994.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.33 pp sourcePopulation 47 stderr 6.325 tokenTotals {"cache_read_tokens":337432064,"cache_write_tokens":null,"input_tokens":345055619,"output_tokens":3342485,"reasoning_tokens":2781362} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/xiaomi~1mimo-v2.6-pro` | | | | | | | | 12 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 56.67% | SE ±6.4 pp | $7.41 | 2,406.95source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.406486 \| USD \| per_test \| Lower is better \| \| `latency` \| 2406.946 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.4 pp sourcePopulation 47 stderr 6.397 tokenTotals {"cache_read_tokens":2598173464,"cache_write_tokens":0,"input_tokens":2623794883,"output_tokens":5326198,"reasoning_tokens":3464191} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/meta~1muse_spark_1_3` | | | | | | | | 12 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 56.67% | SE ±6.4 pp | $5.78 | 1,353.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.778512 \| USD \| per_test \| Lower is better \| \| `latency` \| 1353.461 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.4 pp sourcePopulation 47 stderr 6.397 tokenTotals {"cache_read_tokens":876351355,"cache_write_tokens":10221037,"input_tokens":886640596,"output_tokens":1515597,"reasoning_tokens":796065} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-5.6-terra` | | | | | | | | 12 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 56.67% | SE ±6.4 pp | $0.28 | 1,248.67source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.280231 \| USD \| per_test \| Lower is better \| \| `latency` \| 1248.673 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.4 pp sourcePopulation 47 stderr 6.397 tokenTotals {"cache_read_tokens":776538752,"cache_write_tokens":null,"input_tokens":781847954,"output_tokens":2732010,"reasoning_tokens":1975938} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 15 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 55% | SE ±6.42 pp | $4.76 | 3,411.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.756365 \| USD \| per_test \| Lower is better \| \| `latency` \| 3411.342 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.42 pp sourcePopulation 47 stderr 6.423 tokenTotals {"cache_read_tokens":2126479900,"cache_write_tokens":0,"input_tokens":2166622902,"output_tokens":2549380,"reasoning_tokens":244908} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/minimax~1MiniMax-M3` | | | | | | | | 16 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 53.33% | SE ±6.44 pp | $4.41 | 1,082.73source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.410709 \| USD \| per_test \| Lower is better \| \| `latency` \| 1082.729 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.44 pp sourcePopulation 47 stderr 6.441 tokenTotals {"cache_read_tokens":317828784,"cache_write_tokens":7640171,"input_tokens":325475123,"output_tokens":2317849,"reasoning_tokens":1654262} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-fable-5-1` | | | | | | | | 16 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 53.33% | SE ±6.44 pp | $3.31 | 1,783.43source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.311236 \| USD \| per_test \| Lower is better \| \| `latency` \| 1783.432 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.44 pp sourcePopulation 47 stderr 6.441 tokenTotals {"cache_read_tokens":1006754333,"cache_write_tokens":0,"input_tokens":1030551066,"output_tokens":4215318,"reasoning_tokens":2676940} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/meta~1muse_spark_1_2` | | | | | | | | 18 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 45% | SE ±6.42 pp | $0.49 | 1,452.84source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.486741 \| USD \| per_test \| Lower is better \| \| `latency` \| 1452.835 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.42 pp sourcePopulation 47 stderr 6.423 tokenTotals {"cache_read_tokens":324526199,"cache_write_tokens":null,"input_tokens":377107947,"output_tokens":1477655,"reasoning_tokens":1071214} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/google~1gemini-3.5-flash-lite` | | | | | | | | 18 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 45% | SE ±6.42 pp | $3.73 | 1,100.72source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.731305 \| USD \| per_test \| Lower is better \| \| `latency` \| 1100.724 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.42 pp sourcePopulation 47 stderr 6.423 tokenTotals {"cache_read_tokens":308202284,"cache_write_tokens":4909598,"input_tokens":313117780,"output_tokens":2235503,"reasoning_tokens":1550456} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-opus-5` | | | | | | | | 20 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 41.67% | SE ±6.37 pp | $2.1 | 1,027.02source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.096951 \| USD \| per_test \| Lower is better \| \| `latency` \| 1027.016 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.37 pp sourcePopulation 47 stderr 6.365 tokenTotals {"cache_read_tokens":418344013,"cache_write_tokens":7361982,"input_tokens":425713259,"output_tokens":2372875,"reasoning_tokens":1746473} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-sonnet-5` | | | | | | | | 21 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 31.67% | SE ±6.01 pp | $2.32 | 988.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.320498 \| USD \| per_test \| Lower is better \| \| `latency` \| 988.925 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.01 pp sourcePopulation 47 stderr 6.005 tokenTotals {"cache_read_tokens":409215714,"cache_write_tokens":5769851,"input_tokens":414993169,"output_tokens":2226472,"reasoning_tokens":1660391} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-sonnet-5-5` | | | | | | | | 22 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 26.67% | SE ±5.71 pp | $0.12 | 1,616.78source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.122978 \| USD \| per_test \| Lower is better \| \| `latency` \| 1616.777 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.71 pp sourcePopulation 47 stderr 5.709 tokenTotals {"cache_read_tokens":131523068,"cache_write_tokens":null,"input_tokens":151807555,"output_tokens":921785,"reasoning_tokens":861715} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/inception~1mercury-2.5` | | | | | | | | 23 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 25% | SE ±5.59 pp | $2.62 | 806.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.619409 \| USD \| per_test \| Lower is better \| \| `latency` \| 806.336 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.59 pp sourcePopulation 47 stderr 5.59 tokenTotals {"cache_read_tokens":178561509,"cache_write_tokens":4399125,"input_tokens":182966246,"output_tokens":1638628,"reasoning_tokens":1132881} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-opus-5-5` | | | | | | | | 24 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 5% | SE ±2.81 pp | $1.35 | 595.99source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.353677 \| USD \| per_test \| Lower is better \| \| `latency` \| 595.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.81 pp sourcePopulation 47 stderr 2.814 tokenTotals {"cache_read_tokens":332876499,"cache_write_tokens":null,"input_tokens":347255054,"output_tokens":1296179,"reasoning_tokens":482847} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/google~1gemini-3.6-flash` | | | | | | | | 25 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 0% | SE ±0 pp | — | 0source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 47 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/google~1gemini-3.7-flash` | | | | | | | | 25 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 0% | SE ±0 pp | — | 0source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 47 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-6-astra` | | | | | | | | 25 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 0% | SE ±0 pp | $0.02 | 179.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01784 \| USD \| per_test \| Lower is better \| \| `latency` \| 179.865 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 47 stderr 0 tokenTotals {"cache_read_tokens":320223249,"cache_write_tokens":null,"input_tokens":324018011,"output_tokens":279216,"reasoning_tokens":236844} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/google~1gemini-3.8-flash` | | | | | | | | 25 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 0% | SE ±0 pp | — | 0source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 47 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/openai~1gpt-6.1-sol` | | | | | | | --- Cite as: CyberBench v1.1 / poc — Benchize, https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc (retrieved 2026-10-21)