# Devstral 2 2512 > Source: Benchize (https://benchize.com) — https://benchize.com/models/devstral-2512. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/devstral-2512. Mistral AI Canonical URL: https://benchize.com/models/devstral-2512 Last updated: Thu, 22 Oct 2026 07:10:12 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Mistral AI ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [SWE-bench / >4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3E4%20hours.md) | 0% | SE ±0 pp | 32 / 39 | $1.38 | 419.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.378311 \| USD \| per_test \| Lower is better \| \| `latency` \| 419.145 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 88 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/>4 hours/mistralai~1devstral-2512` | | | | | | | | | [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 51.7% | SE ±2.55 pp | 40 / 43 | $0.05 | 83.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.697 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053806 \| USD \| per_test \| Lower is better \| \| `latency` \| 83.438 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.55 pp sourcePopulation 143 stderr 2.553 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/mistralai~1devstral-2512` | | | | | | | | | [SVGBench Arena](https://benchize.com/benchmarks/svgbench%3Aarena.md) | 1,418 | — | 96 / 191 | — | — | [svgbench](https://svgbench.ai/api/leaderboard?shape=2) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1418 \| rating \| reported \| Higher is better \| ### Configuration Reasoning effort unknown sourceModelId model_mistralai_devstral_2512 ### Reported statistics coverageCount 27 draws 0 judged 98 losses 46 rank 136 ratingDeviation 28 wins 52 ### Source [Original source](https://svgbench.ai/api/leaderboard?shape=2) Source svgbench Original model label Devstral 2 2512 Observed 2026-10-22T04:30:04.538Z Evidence SHA-256 `5c0a9c869593820d57d2c05f6d416e1b44a740b4835e66c66e9ad8b8edc59e2c` Source locator `/rows/132` | | | | | | | | | [LiveCodeBench / easy](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aeasy.md) | 90.68% | SE ±1.62 pp | 40 / 43 | $0.00424 | 19.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.683 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.004242 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.735 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.62 pp sourcePopulation 143 stderr 1.62 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/easy/mistralai~1devstral-2512` | | | | | | | | | [SWE-bench / 15 min - 1 hour](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A15%20min%20-%201%20hour.md) | 56.32% | SE ±3.08 pp | 34 / 39 | $0.82 | 264.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.322 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.819155 \| USD \| per_test \| Lower is better \| \| `latency` \| 264.845 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.08 pp sourcePopulation 88 stderr 3.076 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/15 min - 1 hour/mistralai~1devstral-2512` | | | | | | | | | [LiveCodeBench / overall](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aoverall.md) | 51.84% | SE ±1.17 pp | 40 / 43 | $0.06 | 87.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.841 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.056631 \| USD \| per_test \| Lower is better \| \| `latency` \| 87.57 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 143 stderr 1.174 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/overall/mistralai~1devstral-2512` | | | | | | | | | [SWE-bench Verified](https://benchize.com/benchmarks/swe-bench%3Averified.md) | 53.8% | — | 37 / 63 | — | — | [SWE-bench](https://www.swebench.com/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 53.8 \| percent \| verified \| Higher is better \| ### Configuration agent mini-SWE-agent agentOrganization SWE-agent models ["devstral-2512"] Reasoning effort Not reported sourceSubmission 20251209_mini-v1.17.2_devstral-2512 subset Verified system ["System: Attempts - 1"] ### Reported statistics instanceCalls 75.086 instanceCost Not reported sourceWarning Not reported ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label Devstral (2512) Observed 2026-10-22T02:30:05.773Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/2/results/95` | | | | | | | | | [SWE-bench / overall](https://benchize.com/benchmarks/vals-ai%3Aswebench%3Aoverall.md) | 62.8% | SE ±2.16 pp | 33 / 39 | $0.75 | 243.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.747244 \| USD \| per_test \| Lower is better \| \| `latency` \| 243.516 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 88 stderr 2.164 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/overall/mistralai~1devstral-2512` | | | | | | | | | [SWE-bench / <15 min fix](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3C15%20min%20fix.md) | 80.41% | SE ±2.86 pp | 32 / 39 | $0.59 | 202.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.412 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.589267 \| USD \| per_test \| Lower is better \| \| `latency` \| 202.284 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.86 pp sourcePopulation 88 stderr 2.857 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/<15 min fix/mistralai~1devstral-2512` | | | | | | | | | [LiveCodeBench / hard](https://benchize.com/benchmarks/vals-ai%3Alcb%3Ahard.md) | 13.14% | SE ±1.81 pp | 40 / 43 | $0.11 | 159.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 13.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.111846 \| USD \| per_test \| Lower is better \| \| `latency` \| 159.536 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.81 pp sourcePopulation 143 stderr 1.806 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/hard/mistralai~1devstral-2512` | | | | | | | | | [SWE-bench / 1-4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A1-4%20hours.md) | 26.19% | SE ±6.87 pp | 34 / 39 | $0.99 | 288.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.19 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.985003 \| USD \| per_test \| Lower is better \| \| `latency` \| 288.881 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.2 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.87 pp sourcePopulation 88 stderr 6.867 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label mistralai/devstral-2512 Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/1-4 hours/mistralai~1devstral-2512` | | | | | | | | --- Cite as: Devstral 2 2512 — Benchize, https://benchize.com/models/devstral-2512 (retrieved 2026-10-22)