# Vals Multimodal Index / terminal bench 2 1 > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1. Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLM's can have on the economy. Canonical URL: https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1 Last updated: Wed, 21 Oct 2026 18:05:10 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLM's can have on the economy. Multimodal · Models tested: 20 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 85.77% | SE ±1.35 pp | $1.02 | 372.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.768 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.023319 \| USD \| per_test \| Lower is better \| \| `latency` \| 372.819 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.35 pp sourcePopulation 33 stderr 1.35 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/openai~1gpt-5.6-sol` | | | | | | | | 2 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-high.md)High | 84.64% | SE ±0.99 pp | $0.89 | 752.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.644 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.886936 \| USD \| per_test \| Lower is better \| \| `latency` \| 752.932 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort high harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-opus-5` | | | | | | | | 3 | [Claude Fable 5](https://benchize.com/models/claude-fable-5.md) | 80.52% | SE ±1.35 pp | $1.43 | 504.54source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.429025 \| USD \| per_test \| Lower is better \| \| `latency` \| 504.537 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Anthropic reasoning Not reported Reasoning effort unknown Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.35 pp sourcePopulation 33 stderr 1.35 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-fable-5` | | | | | | | | 4 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 79.03% | SE ±0.99 pp | $0.05 | 355.75source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.026 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053767 \| USD \| per_test \| Lower is better \| \| `latency` \| 355.751 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/openai~1gpt-5.6-luna` | | | | | | | | 5 | [GPT-5.5](https://benchize.com/models/gpt-5-5-high.md)High | 76.4% | SE ±0.65 pp | $0.74 | 427.41source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.404 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.741883 \| USD \| per_test \| Lower is better \| \| `latency` \| 427.412 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 33 stderr 0.649 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label openai/gpt-5.5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/openai~1gpt-5.5` | | | | | | | | 6 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 74.53% | SE ±2.09 pp | $0.8 | 635.04source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.532 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.796027 \| USD \| per_test \| Lower is better \| \| `latency` \| 635.043 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.09 pp sourcePopulation 33 stderr 2.085 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-sonnet-5` | | | | | | | | 7 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 74.16% | SE ±1.12 pp | $0.87 | 373.22source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.157 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.868179 \| USD \| per_test \| Lower is better \| \| `latency` \| 373.223 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.12 pp sourcePopulation 33 stderr 1.124 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3.5-flash` | | | | | | | | 8 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 73.78% | SE ±1.63 pp | $1.17 | 552.15source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.783 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.174404 \| USD \| per_test \| Lower is better \| \| `latency` \| 552.153 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 33 stderr 1.633 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3.6-flash` | | | | | | | | 9 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-xhigh.md)Extra high | 73.41% | SE ±2.09 pp | $0.28 | 260.9source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.408 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.284936 \| USD \| per_test \| Lower is better \| \| `latency` \| 260.9 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.09 pp sourcePopulation 33 stderr 2.085 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/openai~1gpt-5.6-terra` | | | | | | | | 10 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 71.91% | SE ±0.65 pp | $2.41 | 929.9source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.91 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.409772 \| USD \| per_test \| Lower is better \| \| `latency` \| 929.902 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 33 stderr 0.649 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-opus-4-8` | | | | | | | | 11 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 70.79% | SE ±1.12 pp | $0.58 | 410.25source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.787 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.582677 \| USD \| per_test \| Lower is better \| \| `latency` \| 410.249 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.12 pp sourcePopulation 33 stderr 1.124 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3.1-pro-preview` | | | | | | | | 12 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2.md) | 69.66% | SE ±0.65 pp | $0.5 | 879.27source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.663 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.499472 \| USD \| per_test \| Lower is better \| \| `latency` \| 879.273 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Meta reasoning Not reported Reasoning effort unknown Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 33 stderr 0.649 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/meta~1muse_spark_1_2` | | | | | | | | 13 | [Muse Spark 1.1](https://benchize.com/models/muse-spark-1-1-xhigh.md)Extra high | 69.29% | SE ±0.99 pp | $0.42 | 624.77source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.417396 \| USD \| per_test \| Lower is better \| \| `latency` \| 624.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/meta~1muse_spark_1_1` | | | | | | | | 14 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-high.md)High | 68.54% | SE ±1.3 pp | $1.98 | 816.59source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.539 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.977087 \| USD \| per_test \| Lower is better \| \| `latency` \| 816.587 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort high harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 33 stderr 1.297 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-7 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-opus-4-7` | | | | | | | | 15 | [MiMo-V2.5](https://benchize.com/models/mimo-v2-5.md) | 60.67% | SE ±5.21 pp | $0.02 | 775.24source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.674 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.021658 \| USD \| per_test \| Lower is better \| \| `latency` \| 775.239 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.21 pp sourcePopulation 33 stderr 5.207 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label xiaomi/mimo-v2.5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/xiaomi~1mimo-v2.5` | | | | | | | | 16 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-off.md)Non-thinking | 57.3% | SE ±0.65 pp | $0.57 | 704.23source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.303 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.572392 \| USD \| per_test \| Lower is better \| \| `latency` \| 704.225 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning false Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 33 stderr 0.649 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/anthropic~1claude-sonnet-4-6` | | | | | | | | 17 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 53.93% | SE ±1.3 pp | $0.15 | 376.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.933 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.154312 \| USD \| per_test \| Lower is better \| \| `latency` \| 376.158 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 33 stderr 1.297 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3-flash-preview` | | | | | | | | 18 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 53.56% | SE ±0.75 pp | $0.21 | 1,023.42source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.558 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.205973 \| USD \| per_test \| Lower is better \| \| `latency` \| 1023.416 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.75 pp sourcePopulation 33 stderr 0.749 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/minimax~1MiniMax-M3` | | | | | | | | 19 | [Mistral Medium 3.5](https://benchize.com/models/mistral-medium-3-5-high.md)High | 38.95% | SE ±2.7 pp | $1.19 | 989.17source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.951 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.190984 \| USD \| per_test \| Lower is better \| \| `latency` \| 989.169 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 33 stderr 2.701 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label mistralai/mistral-medium-3.5 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/mistralai~1mistral-medium-3.5` | | | | | | | | 20 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md)High | 34.08% | SE ±0.99 pp | $0.08 | 278.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.082 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.084894 \| USD \| per_test \| Lower is better \| \| `latency` \| 278.342 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3.1-flash-lite-preview` | | | | | | | --- Cite as: Vals Multimodal Index / terminal bench 2 1 — Benchize, https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1 (retrieved 2026-10-21)