# CyberBench v1.1 / patch > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Canonical URL: https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Safety & robustness · Models tested: 28 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 89.29% | SE ±4.13 pp | $1.05 | 544.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.054807 \| USD \| per_test \| Lower is better \| \| `latency` \| 544.579 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.13 pp sourcePopulation 47 stderr 4.133 tokenTotals {"cache_read_tokens":74366882,"cache_write_tokens":3717011,"input_tokens":78127183,"output_tokens":528210,"reasoning_tokens":316000} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-5.6-sol` | | | | | | | | 1 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 89.29% | SE ±4.13 pp | $0.97 | 1,769.29source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.973744 \| USD \| per_test \| Lower is better \| \| `latency` \| 1769.293 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.13 pp sourcePopulation 47 stderr 4.133 tokenTotals {"cache_read_tokens":778762528,"cache_write_tokens":13230568,"input_tokens":792003986,"output_tokens":7580926,"reasoning_tokens":5771159} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-haiku-5-5` | | | | | | | | 1 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 89.29% | SE ±4.13 pp | $0.74 | 686.53source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.738255 \| USD \| per_test \| Lower is better \| \| `latency` \| 686.527 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.13 pp sourcePopulation 47 stderr 4.133 tokenTotals {"cache_read_tokens":90276605,"cache_write_tokens":4404805,"input_tokens":94724760,"output_tokens":1218826,"reasoning_tokens":587117} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-6-sol` | | | | | | | | 4 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 87.5% | SE ±4.42 pp | $6.51 | 3,076.33source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.514246 \| USD \| per_test \| Lower is better \| \| `latency` \| 3076.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":1250717995,"cache_write_tokens":15482728,"input_tokens":1266214579,"output_tokens":7588569,"reasoning_tokens":5454199} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-sonnet-5-5` | | | | | | | | 4 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 87.5% | SE ±4.42 pp | $0.84 | 694.4source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.837953 \| USD \| per_test \| Lower is better \| \| `latency` \| 694.402 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":112516436,"cache_write_tokens":5363115,"input_tokens":117928626,"output_tokens":893821,"reasoning_tokens":597103} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-5.6-terra` | | | | | | | | 4 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 87.5% | SE ±4.42 pp | $0.11 | 606.96source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.105488 \| USD \| per_test \| Lower is better \| \| `latency` \| 606.957 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":168260608,"cache_write_tokens":null,"input_tokens":171650340,"output_tokens":1560761,"reasoning_tokens":1214217} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 4 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 87.5% | SE ±4.42 pp | $3.61 | 1,141.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.610668 \| USD \| per_test \| Lower is better \| \| `latency` \| 1141.461 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":195550188,"cache_write_tokens":4819249,"input_tokens":200375111,"output_tokens":1858688,"reasoning_tokens":1120338} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-fable-5-1` | | | | | | | | 4 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 87.5% | SE ±4.42 pp | $2.15 | 907.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.146121 \| USD \| per_test \| Lower is better \| \| `latency` \| 907.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":331520510,"cache_write_tokens":null,"input_tokens":368043780,"output_tokens":2089305,"reasoning_tokens":1659807} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/google~1gemini-3.7-flash` | | | | | | | | 4 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 87.5% | SE ±4.42 pp | $2.59 | 997.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.592537 \| USD \| per_test \| Lower is better \| \| `latency` \| 997.832 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":580481719,"cache_write_tokens":null,"input_tokens":616087630,"output_tokens":3645533,"reasoning_tokens":3223721} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/google~1gemini-3.8-flash` | | | | | | | | 4 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 87.5% | SE ±4.42 pp | $0.04 | 622.21source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.036459 \| USD \| per_test \| Lower is better \| \| `latency` \| 622.21 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":72887486,"cache_write_tokens":4036841,"input_tokens":76967293,"output_tokens":1578776,"reasoning_tokens":623938} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-6-luna` | | | | | | | | 11 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 85.71% | SE ±4.68 pp | $5 | 1,294.56source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.998186 \| USD \| per_test \| Lower is better \| \| `latency` \| 1294.559 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":370161604,"cache_write_tokens":null,"input_tokens":408936582,"output_tokens":2538309,"reasoning_tokens":2199322} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/google~1gemini-4-argon` | | | | | | | | 11 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 85.71% | SE ±4.68 pp | $1.6 | 573.62source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.595649 \| USD \| per_test \| Lower is better \| \| `latency` \| 573.622 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":217411788,"cache_write_tokens":0,"input_tokens":222983450,"output_tokens":1179366,"reasoning_tokens":761461} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/meta~1muse-wonder` | | | | | | | | 11 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 85.71% | SE ±4.68 pp | $0.29 | 752.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.286927 \| USD \| per_test \| Lower is better \| \| `latency` \| 752.164 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":161342336,"cache_write_tokens":null,"input_tokens":164763464,"output_tokens":1124488,"reasoning_tokens":772476} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 11 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 85.71% | SE ±4.68 pp | $2.01 | 1,230.08source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.005228 \| USD \| per_test \| Lower is better \| \| `latency` \| 1230.084 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":187975583,"cache_write_tokens":5014307,"input_tokens":192995538,"output_tokens":1802901,"reasoning_tokens":1157748} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-opus-5` | | | | | | | | 11 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 85.71% | SE ±4.68 pp | $0.06 | 1,328.02source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.06428 \| USD \| per_test \| Lower is better \| \| `latency` \| 1328.015 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":86926592,"cache_write_tokens":null,"input_tokens":90775069,"output_tokens":1853606,"reasoning_tokens":1534749} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/xiaomi~1mimo-v2.6-pro` | | | | | | | | 11 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 85.71% | SE ±4.68 pp | $0.03 | 1,286.65source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.026483 \| USD \| per_test \| Lower is better \| \| `latency` \| 1286.649 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":100661120,"cache_write_tokens":null,"input_tokens":105465030,"output_tokens":1887952,"reasoning_tokens":1565847} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/xiaomi~1mimo-v2.6-flash` | | | | | | | | 11 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 85.71% | SE ±4.68 pp | $4.33 | 1,375.92source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.333799 \| USD \| per_test \| Lower is better \| \| `latency` \| 1375.915 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":351561251,"cache_write_tokens":7467280,"input_tokens":359036095,"output_tokens":3233493,"reasoning_tokens":2180818} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-opus-5-5` | | | | | | | | 11 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 85.71% | SE ±4.68 pp | $1.26 | 724.09source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.258688 \| USD \| per_test \| Lower is better \| \| `latency` \| 724.086 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":154122230,"cache_write_tokens":null,"input_tokens":178356232,"output_tokens":1468957,"reasoning_tokens":1218609} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/google~1gemini-3.6-flash` | | | | | | | | 11 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 85.71% | SE ±4.68 pp | $0.05 | 556.02source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053798 \| USD \| per_test \| Lower is better \| \| `latency` \| 556.018 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":163675136,"cache_write_tokens":null,"input_tokens":167105583,"output_tokens":1611740,"reasoning_tokens":1272951} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/deepseek~1deepseek-v4.1-flash` | | | | | | | | 11 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 85.71% | SE ±4.68 pp | $0.99 | 657.73source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.986918 \| USD \| per_test \| Lower is better \| \| `latency` \| 657.734 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":246391784,"cache_write_tokens":0,"input_tokens":255260583,"output_tokens":1699451,"reasoning_tokens":1044559} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/meta~1muse_spark_1_2` | | | | | | | | 21 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 83.93% | SE ±4.91 pp | $0.06 | 497.94source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.929 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.062032 \| USD \| per_test \| Lower is better \| \| `latency` \| 497.943 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.91 pp sourcePopulation 47 stderr 4.908 tokenTotals {"cache_read_tokens":83042966,"cache_write_tokens":3893541,"input_tokens":86979941,"output_tokens":692378,"reasoning_tokens":357952} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-5.6-luna` | | | | | | | | 22 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 82.14% | SE ±5.12 pp | $1.11 | 578.13source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.110328 \| USD \| per_test \| Lower is better \| \| `latency` \| 578.127 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.12 pp sourcePopulation 47 stderr 5.118 tokenTotals {"cache_read_tokens":288278765,"cache_write_tokens":0,"input_tokens":297692494,"output_tokens":1686914,"reasoning_tokens":1039576} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/meta~1muse_spark_1_3` | | | | | | | | 22 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 82.14% | SE ±5.12 pp | $1.68 | 1,007.05source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.680944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1007.05 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.12 pp sourcePopulation 47 stderr 5.118 tokenTotals {"cache_read_tokens":262257238,"cache_write_tokens":6482240,"input_tokens":268745630,"output_tokens":2546353,"reasoning_tokens":1828930} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-sonnet-5` | | | | | | | | 22 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 82.14% | SE ±5.12 pp | $1.84 | 500.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.840737 \| USD \| per_test \| Lower is better \| \| `latency` \| 500.83 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.12 pp sourcePopulation 47 stderr 5.118 tokenTotals {"cache_read_tokens":33744598,"cache_write_tokens":3331597,"input_tokens":37116904,"output_tokens":545692,"reasoning_tokens":309740} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-6-astra` | | | | | | | | 25 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 78.57% | SE ±5.48 pp | $0.29 | 443.98source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.571 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.285372 \| USD \| per_test \| Lower is better \| \| `latency` \| 443.983 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.48 pp sourcePopulation 47 stderr 5.483 tokenTotals {"cache_read_tokens":30657892,"cache_write_tokens":3114644,"input_tokens":33813954,"output_tokens":504558,"reasoning_tokens":289398} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/openai~1gpt-6.1-sol` | | | | | | | | 26 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 75% | SE ±5.79 pp | $2.06 | 1,501.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.060219 \| USD \| per_test \| Lower is better \| \| `latency` \| 1501.462 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.79 pp sourcePopulation 47 stderr 5.786 tokenTotals {"cache_read_tokens":862219106,"cache_write_tokens":0,"input_tokens":872573276,"output_tokens":2155708,"reasoning_tokens":752843} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/minimax~1MiniMax-M3` | | | | | | | | 27 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 73.21% | SE ±5.92 pp | $0.36 | 698.94source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.214 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.363507 \| USD \| per_test \| Lower is better \| \| `latency` \| 698.939 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.92 pp sourcePopulation 47 stderr 5.918 tokenTotals {"cache_read_tokens":183828171,"cache_write_tokens":null,"input_tokens":222465635,"output_tokens":1300133,"reasoning_tokens":1087005} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/google~1gemini-3.5-flash-lite` | | | | | | | | 28 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 64.29% | SE ±6.4 pp | $0.13 | 661.56source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.126262 \| USD \| per_test \| Lower is better \| \| `latency` \| 661.559 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.4 pp sourcePopulation 47 stderr 6.403 tokenTotals {"cache_read_tokens":145289216,"cache_write_tokens":null,"input_tokens":163464203,"output_tokens":706505,"reasoning_tokens":634831} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/inception~1mercury-2.5` | | | | | | | --- Cite as: CyberBench v1.1 / patch — Benchize, https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch (retrieved 2026-10-21)