# Terminal-Bench 2.0 / easy > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy. State-of-the-art set of difficult terminal-based tasks Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy Last updated: Wed, 21 Oct 2026 18:05:10 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Conquered State-of-the-art set of difficult terminal-based tasks Agents & tools · Models tested: 23 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-high.md)High | 100% | SE ±0 pp | $21.71 | 14,704.81source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 21.713924 \| USD \| per_test \| Lower is better \| \| `latency` \| 14704.807 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort high harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label anthropic/claude-opus-4-7 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/anthropic~1claude-opus-4-7` | | | | | | | | 1 | [Gemini 2.5 Pro](https://benchize.com/models/gemini-2-5-pro.md) | 100% | SE ±0 pp | $10.5 | 17,722.7source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.502075 \| USD \| per_test \| Lower is better \| \| `latency` \| 17722.698 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-2.5-pro Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-2.5-pro` | | | | | | | | 1 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 100% | SE ±0 pp | $3.75 | 10,147.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.753614 \| USD \| per_test \| Lower is better \| \| `latency` \| 10147.584 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-3-flash-preview` | | | | | | | | 1 | [GPT-5.3 Codex](https://benchize.com/models/gpt-5-3-codex-high.md)High | 100% | SE ±0 pp | $7.68 | 11,264.79source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.677245 \| USD \| per_test \| Lower is better \| \| `latency` \| 11264.79 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label openai/gpt-5.3-codex Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/openai~1gpt-5.3-codex` | | | | | | | | 1 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 100% | SE ±0 pp | $11.48 | 12,079.23source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.482876 \| USD \| per_test \| Lower is better \| \| `latency` \| 12079.231 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-3.1-pro-preview` | | | | | | | | 1 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 100% | SE ±0 pp | $40.53 | 9,079.06source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 40.526503 \| USD \| per_test \| Lower is better \| \| `latency` \| 9079.061 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-3.5-flash` | | | | | | | | 7 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 91.67% | SE ±8.33 pp | $11.8 | 10,511.91source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.799576 \| USD \| per_test \| Lower is better \| \| `latency` \| 10511.909 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 67 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/anthropic~1claude-opus-4-8` | | | | | | | | 8 | [GPT-5.5](https://benchize.com/models/gpt-5-5-high.md)High | 85% | SE ±15 pp | $84.07 | 51,135.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 84.073356 \| USD \| per_test \| Lower is better \| \| `latency` \| 51135.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15 pp sourcePopulation 67 stderr 15 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label openai/gpt-5.5 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/openai~1gpt-5.5` | | | | | | | | 9 | [Gemma 4 31B](https://benchize.com/models/gemma-4-31b-it-high.md)High | 75% | SE ±25 pp | — | 19,030.38source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `latency` \| 19030.377 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemma-4-31b-it` | | | | | | | | 9 | [MiniMax M2.5](https://benchize.com/models/minimax-m2-5.md) | 75% | SE ±25 pp | $1.89 | 16,787.91source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.894882 \| USD \| per_test \| Lower is better \| \| `latency` \| 16787.905 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 196608 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label minimax/MiniMax-M2.5 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/minimax~1MiniMax-M2.5` | | | | | | | | 9 | [Laguna M 1](https://benchize.com/models/laguna-m-1.md) | 75% | SE ±25 pp | — | 0source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label poolside/laguna-m.1 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/poolside~1laguna-m.1` | | | | | | | | 9 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-off.md)Non-thinking | 75% | SE ±25 pp | $11.55 | 13,694.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.546985 \| USD \| per_test \| Lower is better \| \| `latency` \| 13694.866 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning false Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/anthropic~1claude-sonnet-4-6` | | | | | | | | 9 | [MiniMax M2.7](https://benchize.com/models/minimax-m2-7.md) | 75% | SE ±25 pp | $1.65 | 18,668.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.646383 \| USD \| per_test \| Lower is better \| \| `latency` \| 18668.818 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 196608 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label minimax/MiniMax-M2.7 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/minimax~1MiniMax-M2.7` | | | | | | | | 9 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-off.md)Non-thinking | 75% | SE ±25 pp | — | 18,430.61source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `latency` \| 18430.614 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning false Reasoning effort unknown Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/deepseek~1deepseek-v4-pro` | | | | | | | | 9 | [Laguna Xs 2](https://benchize.com/models/laguna-xs-2.md) | 75% | SE ±25 pp | — | 0source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label poolside/laguna-xs.2 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/poolside~1laguna-xs.2` | | | | | | | | 9 | [Mistral Medium 3.5](https://benchize.com/models/mistral-medium-3-5-high.md)High | 75% | SE ±25 pp | $104.81 | 27,882.22source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 104.807458 \| USD \| per_test \| Lower is better \| \| `latency` \| 27882.215 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label mistralai/mistral-medium-3.5 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/mistralai~1mistral-medium-3.5` | | | | | | | | 17 | [Mistral Large 3 2512](https://benchize.com/models/mistral-large-2512.md) | 50% | SE ±28.87 pp | $0.68 | 26,261.72source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.678233 \| USD \| per_test \| Lower is better \| \| `latency` \| 26261.72 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.065 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label mistralai/mistral-large-2512 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/mistralai~1mistral-large-2512` | | | | | | | | 17 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md)High | 50% | SE ±28.87 pp | $0.46 | 14,860.31source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.46031 \| USD \| per_test \| Lower is better \| \| `latency` \| 14860.307 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-3.1-flash-lite-preview` | | | | | | | | 17 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 50% | SE ±28.87 pp | $4.81 | 22,682.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.810641 \| USD \| per_test \| Lower is better \| \| `latency` \| 22682.927 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/minimax~1MiniMax-M3` | | | | | | | | 17 | [Mistral Small 4](https://benchize.com/models/mistral-small-2603-high.md)High | 50% | SE ±28.87 pp | $2.35 | 9,075source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.353195 \| USD \| per_test \| Lower is better \| \| `latency` \| 9074.999 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label mistralai/mistral-small-2603 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/mistralai~1mistral-small-2603` | | | | | | | | 17 | [MiniMax M2.1](https://benchize.com/models/minimax-m2-1.md) | 50% | SE ±28.87 pp | $1.57 | 17,401.07source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.56576 \| USD \| per_test \| Lower is better \| \| `latency` \| 17401.069 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label minimax/MiniMax-M2.1 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/minimax~1MiniMax-M2.1` | | | | | | | | 17 | [Trinity Large Thinking](https://benchize.com/models/trinity-large-thinking.md) | 50% | SE ±28.87 pp | $7.24 | 22,588.73source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.243399 \| USD \| per_test \| Lower is better \| \| `latency` \| 22588.728 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Arcee-Ai reasoning Not reported Reasoning effort unknown Temperature 0.3 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label arcee-ai/trinity-large-thinking Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/arcee-ai~1trinity-large-thinking` | | | | | | | | 23 | [Command A](https://benchize.com/models/command-a.md) | 0% | SE ±0 pp | $54.22 | 16,924.99source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 54.220382 \| USD \| per_test \| Lower is better \| \| `latency` \| 16924.986 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 8000 provider Cohere reasoning Not reported Reasoning effort unknown Temperature 0.3 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 67 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label cohere/command-a-03-2025 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/cohere~1command-a-03-2025` | | | | | | | --- Cite as: Terminal-Bench 2.0 / easy — Benchize, https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy (retrieved 2026-10-21)