# MASK

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Amask. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Amask.

As AI systems become more powerful, autonomous, and agentic, users need confidence that these systems won't deceive them. A model that regularly produces statements contradicting its internal beliefs is not only unsuitable for many applications but potentially dangerous. Lying AI systems might conceal information they prefer users not to see or expose their owners to legal liability through misrepresentation. While the AI safety field aspires to create honest AI systems, recent studies show concerning indicators of deceptive behavior in leading language models. In partnership with theCenter for AI Safety(CAIS), we introduce Model Alignment between Statements and Knowledge (MASK), a benchmark specifically designed to measure honesty in language models when they are pressured, directly or indirectly, to lie. MASK isolates the concept of honesty by first eliciting a model's underlying beliefs and then testing whether it will contradict those beliefs when prompted or pressured to lie.

Canonical URL: https://benchize.com/benchmarks/scale%3Amask
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

As AI systems become more powerful, autonomous, and agentic, users need confidence that these systems won't deceive them. A model that regularly produces statements contradicting its internal beliefs is not only unsuitable for many applications but potentially dangerous. Lying AI systems might conceal information they prefer users not to see or expose their owners to legal liability through misrepresentation. While the AI safety field aspires to create honest AI systems, recent studies show concerning indicators of deceptive behavior in leading language models. In partnership with theCenter for AI Safety(CAIS), we introduce Model Alignment between Statements and Knowledge (MASK), a benchmark specifically designed to measure honesty in language models when they are pressured, directly or indirectly, to lie. MASK isolates the concept of honesty by first eliciting a model's underlying beliefs and then testing whether it will contradict those beliefs when prompted or pressured to lie.

Safety & robustness · Models tested: 25 · Score publishers: [scale](https://labs.scale.com/leaderboard/mask)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [Claude Opus 4.6](https://benchize.com/models/claude-opus-4-6-off.md)Non-thinking | 96.28source score | CI 95.87–96.69 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 96.28 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort non-thinking Model configuration claude-opus-4-6 (Non-Thinking) sourceModelVersion claude-opus-4-6 (Non-Thinking) thinkingEnabled false ### Reported statistics Uncertainty Confidence interval: 95.87–96.69 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 1 reportedConfidenceHalfWidth 0.41 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label claude-opus-4-6 (Non-Thinking) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/0` |  |  |  |  |
| 2 | [Claude Sonnet 4](https://benchize.com/models/claude-sonnet-4-thinking.md)Thinking | 95.33source score | CI 93.04–97.62 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 95.33 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 (Thinking) sourceModelVersion claude-4-sonnet-20250514-thinking thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 93.04–97.62 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 2.29 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude Sonnet 4 (Thinking) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/2` |  |  |  |  |
| 3 | [gpt-oss-120b](https://benchize.com/models/gpt-oss-120b.md) | 92source score | CI 91.14–92.86 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 92 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gpt-oss-120b sourceModelVersion gpt-oss-120b thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 91.14–92.86 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 6 reportedConfidenceHalfWidth 0.86 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label gpt-oss-120b Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/5` |  |  |  |  |
| 4 | [Claude Sonnet 4](https://benchize.com/models/claude-sonnet-4.md) | 89.27source score | CI 87.26–91.28 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 89.27 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion claude-sonnet-4-20250514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 87.26–91.28 contaminationWarning Not reported rank 9 reportedConfidenceHalfWidth 2.01 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude Sonnet 4 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/8` |  |  |  |  |
| 5 | [Claude Opus 4](https://benchize.com/models/claude-opus-4-thinking.md)Thinking | 87.87source score | CI 84.11–91.63 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 87.87 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Opus 4 (Thinking) sourceModelVersion claude-opus-4-20250514-thinking thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 84.11–91.63 contaminationWarning Not reported rank 10 reportedConfidenceHalfWidth 3.76 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude Opus 4 (Thinking) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/9` |  |  |  |  |
| 6 | [gpt-oss-20b](https://benchize.com/models/gpt-oss-20b.md) | 86.46source score | CI 84.39–88.53 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 86.46 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gpt-oss-20b sourceModelVersion gpt-oss-20b thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 84.39–88.53 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 14 reportedConfidenceHalfWidth 2.07 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label gpt-oss-20b Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/13` |  |  |  |  |
| 7 | [Claude Opus 4](https://benchize.com/models/claude-opus-4.md) | 80.28source score | CI 79.66–80.9 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 80.28 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Opus 4 sourceModelVersion claude-opus-4-20250514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 79.66–80.9 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 24 reportedConfidenceHalfWidth 0.62 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude Opus 4 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/23` |  |  |  |  |
| 8 | [Claude 3 Opus](https://benchize.com/models/claude-3-opus.md) | 79source score | CI 77.69–80.31 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 79 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude 3 Opus sourceModelVersion claude-3-opus-20240229 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 77.69–80.31 contaminationWarning Not reported rank 26 reportedConfidenceHalfWidth 1.31 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude 3 Opus Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/25` |  |  |  |  |
| 9 | [O1 Pro](https://benchize.com/models/o1-pro.md) | 61.6source score | CI 60.74–62.46 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 61.6 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o1-Pro sourceModelVersion o1-pro thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 60.74–62.46 contaminationWarning Not reported rank 33 reportedConfidenceHalfWidth 0.86 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label o1-Pro Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/32` |  |  |  |  |
| 10 | [GPT-4.1 Nano](https://benchize.com/models/gpt-4-1-nano.md) | 61.4source score | CI 59.61–63.19 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 61.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration GPT-4.1 nano sourceModelVersion gpt-4.1-nano-2025-04-14 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 59.61–63.19 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 35 reportedConfidenceHalfWidth 1.79 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label GPT-4.1 nano Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/34` |  |  |  |  |
| 11 | [Qwen3 235B A22B](https://benchize.com/models/qwen3-235b-a22b.md) | 56.4source score | CI 51.42–61.38 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 56.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Qwen3-235B-A22B sourceModelVersion Qwen3-235B-A22B thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 51.42–61.38 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 43 reportedConfidenceHalfWidth 4.98 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Qwen3-235B-A22B Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/42` |  |  |  |  |
| 12 | [Gemini 2.5 Pro Preview 06-05](https://benchize.com/models/gemini-2-5-pro-preview.md) | 55.67source score | CI 51.16–60.17 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 55.666 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-2.5-pro-preview-06-05 sourceModelVersion gemini-2.5-pro-preview-06-05 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 51.16–60.17 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 45 reportedConfidenceHalfWidth 4.508 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label gemini-2.5-pro-preview-06-05 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/44` |  |  |  |  |
| 13 | [R1 0528](https://benchize.com/models/deepseek-r1-0528.md) | 53source score | CI 48.8–57.2 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 53 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration DeepSeek-R1-0528 sourceModelVersion DeepSeek-R1-0528 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 48.8–57.2 contaminationWarning Not reported rank 48 reportedConfidenceHalfWidth 4.2 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label DeepSeek-R1-0528 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/47` |  |  |  |  |
| 14 | [Llama 3.3 70B Instruct](https://benchize.com/models/llama-3-3-70b-instruct.md) | 51.93source score | CI 46.95–56.91 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 51.93 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Llama 3.3 70B Instruct sourceModelVersion llama-3-3-70b-instruct thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 46.95–56.91 contaminationWarning Not reported rank 49 reportedConfidenceHalfWidth 4.98 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Llama 3.3 70B Instruct Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/48` |  |  |  |  |
| 15 | [GPT-4.1](https://benchize.com/models/gpt-4-1.md) | 51.13source score | CI 50.1–52.16 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 51.13 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration GPT-4.1 sourceModelVersion gpt-4.1-2025-04-14 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 50.1–52.16 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 50 reportedConfidenceHalfWidth 1.03 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label GPT-4.1 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/49` |  |  |  |  |
| 16 | [GPT-4.1 Mini](https://benchize.com/models/gpt-4-1-mini.md) | 50source score | CI 47.83–52.17 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 50 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration GPT-4.1 mini sourceModelVersion gpt-4.1-mini-2025-04-14 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 47.83–52.17 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 51 reportedConfidenceHalfWidth 2.17 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label GPT-4.1 mini Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/50` |  |  |  |  |
| 17 | [O3 Mini](https://benchize.com/models/o3-mini-low.md)Low | 49.73source score | CI 46.5–52.96 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 49.73 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort low Model configuration o3 mini (Low) sourceModelVersion o3-mini-2025-01-31 (low) thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 46.5–52.96 contaminationWarning Not reported rank 53 reportedConfidenceHalfWidth 3.23 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label o3 mini (Low) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/52` |  |  |  |  |
| 17 | [Llama 4 Maverick](https://benchize.com/models/llama-4-maverick.md) | 49.73source score | CI 48.13–51.33 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 49.73 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Llama 4 Maverick sourceModelVersion llama-4-17b-omni-instruct-fp8-128e-final-vllm thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 48.13–51.33 contaminationWarning Not reported rank 52 reportedConfidenceHalfWidth 1.6 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Llama 4 Maverick Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/51` |  |  |  |  |
| 19 | [Gemini 2.0 Flash](https://benchize.com/models/gemini-2-0-flash.md) | 49.07source score | CI 47.06–51.08 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 49.07 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemini 2.0 Flash sourceModelVersion gemini-2.0-flash thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 47.06–51.08 contaminationWarning Not reported rank 56 reportedConfidenceHalfWidth 2.01 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Gemini 2.0 Flash Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/55` |  |  |  |  |
| 20 | [O3 Mini](https://benchize.com/models/o3-mini-medium.md)Medium | 48.93source score | CI 47.68–50.18 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 48.93 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort medium Model configuration o3 mini (Medium) sourceModelVersion o3-mini-2025-01-31 (medium) thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 47.68–50.18 contaminationWarning Not reported rank 57 reportedConfidenceHalfWidth 1.25 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label o3 mini (Medium) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/56` |  |  |  |  |
| 21 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview.md) | 48.4source score | CI 47.54–49.26 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 48.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-3.1-flash-lite-preview sourceModelVersion gemini-3.1-flash-lite-preview thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 47.54–49.26 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 59 reportedConfidenceHalfWidth 0.86 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label gemini-3.1-flash-lite-preview Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/58` |  |  |  |  |
| 22 | [O3 Mini](https://benchize.com/models/o3-mini-high.md)High | 46.8source score | CI 44.22–49.38 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 46.8 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort high Model configuration o3 mini (High) sourceModelVersion o3-mini-2025-01-31 (high) thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 44.22–49.38 contaminationWarning Not reported rank 61 reportedConfidenceHalfWidth 2.58 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label o3 mini (High) Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/60` |  |  |  |  |
| 23 | [DeepSeek V3.1](https://benchize.com/models/deepseek-v3-1.md) | 46.27source score | CI 44.67–47.87 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 46.27 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration DeepSeek-V3.1 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 44.67–47.87 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 63 reportedConfidenceHalfWidth 1.6 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label DeepSeek-V3.1 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/62` |  |  |  |  |
| 24 | [Mistral Medium 3](https://benchize.com/models/mistral-medium-3.md) | 42.6source score | CI 39.34–45.86 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 42.6 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Mistral Medium 3 sourceModelVersion mistral-medium-latest thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 39.34–45.86 contaminationWarning Not reported rank 65 reportedConfidenceHalfWidth 3.26 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Mistral Medium 3 Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/64` |  |  |  |  |
| 25 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview.md) | 42.4source score | CI 41.1–43.7 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 42.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-3.1-pro-preview sourceModelVersion gemini-3.1-pro-preview thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 41.1–43.7 contaminationWarning Potential contamination warning: This model was evaluated after the public release of MASK, allowing model builder access to the prompts and solutions. rank 67 reportedConfidenceHalfWidth 1.3 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label gemini-3.1-pro-preview Observed 2026-10-21T16:35:38.046Z Evidence SHA-256 `506198f10b9789b267cc2a11dcdfedc27067703da430d78b374b2c5dffb8ec7d` Source locator `/rows/66` |  |  |  |  |

---
Cite as: MASK — Benchize, https://benchize.com/benchmarks/scale%3Amask (retrieved 2026-10-21)