# Adversarial Robustness

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Aadversarial_robustness. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Aadversarial_robustness.

The Scale Adversarial Robustness Evaluation runs on 1,000 human-written prompts that are designed to trigger harmful responses from large language models. These prompts cover broad topics from illegal activities and hate speech all the way to niche topics such as harm to children and self-harm. The focus is on harm that is universally recognized as problematic, rather than issues that might be deemed harmful by one organization but not by another. Among these prompts, almost all of them are clearly adversarial - deliberately designed to elicit problematic responses from the model, as opposed to harmful responses that an innocent user may randomly stumble upon.Defining harm caused by LLMs necessarily touches on a product’s intended purpose and role, and is likely to differ between companies and platforms. In creating this evaluation standard we do not claim to have the definitive answer for product intent. Neither do we favor any particular company’s view on safety. Instead we independently created the harm definitions, and seek to establish reasonable benchmarks for comparison across current frontier models. We encourage suggestions and contributions from the community to refine and add details to these definitions for future versions of the leaderboard.This leaderboard aims to measure the community's progress in steering the models to avoid producing harmful output. The ranking is based solely on the “high harm” category, with the reference numbers for "general harm" (either high harm or low harm) included in the blog post. Please see the “Definitions of Harm” for additional detail. The leaderboard ranks the models based on the number of violations in their responses to 1000 adversarial prompts, from the lowest to the highest number of violations. Models with fewer violations are considered more robust against adversarial actors according to our measurements. Prompts were selected with a principle of universal harm, i.e. those that seek to elicit responses for which model providers, under most circumstances, would agree a model should be guarded against.Please see the topic distribution for the prompts below:

Canonical URL: https://benchize.com/benchmarks/scale%3Aadversarial_robustness
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

The Scale Adversarial Robustness Evaluation runs on 1,000 human-written prompts that are designed to trigger harmful responses from large language models. These prompts cover broad topics from illegal activities and hate speech all the way to niche topics such as harm to children and self-harm. The focus is on harm that is universally recognized as problematic, rather than issues that might be deemed harmful by one organization but not by another. Among these prompts, almost all of them are clearly adversarial - deliberately designed to elicit problematic responses from the model, as opposed to harmful responses that an innocent user may randomly stumble upon.Defining harm caused by LLMs necessarily touches on a product’s intended purpose and role, and is likely to differ between companies and platforms. In creating this evaluation standard we do not claim to have the definitive answer for product intent. Neither do we favor any particular company’s view on safety. Instead we independently created the harm definitions, and seek to establish reasonable benchmarks for comparison across current frontier models. We encourage suggestions and contributions from the community to refine and add details to these definitions for future versions of the leaderboard.This leaderboard aims to measure the community's progress in steering the models to avoid producing harmful output. The ranking is based solely on the “high harm” category, with the reference numbers for "general harm" (either high harm or low harm) included in the blog post. Please see the “Definitions of Harm” for additional detail. The leaderboard ranks the models based on the number of violations in their responses to 1000 adversarial prompts, from the lowest to the highest number of violations. Models with fewer violations are considered more robust against adversarial actors according to our measurements. Prompts were selected with a principle of universal harm, i.e. those that seek to elicit responses for which model providers, under most circumstances, would agree a model should be guarded against.Please see the topic distribution for the prompts below:

Safety & robustness · Models tested: 4 · Score publishers: [scale](https://labs.scale.com/leaderboard/adversarial_robustness)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [Mistral Large](https://benchize.com/models/mistral-large.md) | 37source score | CI 23–51 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 37 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Mistral Large sourceModelVersion mistral-large-2402 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 23–51 contaminationWarning Not reported rank 7 reportedConfidenceHalfWidth 14 ### Source [Original source](https://labs.scale.com/leaderboard/adversarial_robustness) Source scale Original model label Mistral Large Observed 2026-10-21T16:35:39.820Z Evidence SHA-256 `9ad617f4026dbd8729d7a28d03eca7818e4f0f6e7e03675760b4a0d2f1ab8f47` Source locator `/rows/6` |  |  |  |  |
| 2 | [GPT-4 Turbo Preview](https://benchize.com/models/gpt-4-turbo-preview.md) | 20source score | CI 9–31 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 20 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration GPT-4 Turbo Preview sourceModelVersion gpt-4-0125-preview thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 9–31 contaminationWarning Not reported rank 6 reportedConfidenceHalfWidth 11 ### Source [Original source](https://labs.scale.com/leaderboard/adversarial_robustness) Source scale Original model label GPT-4 Turbo Preview Observed 2026-10-21T16:35:39.820Z Evidence SHA-256 `9ad617f4026dbd8729d7a28d03eca7818e4f0f6e7e03675760b4a0d2f1ab8f47` Source locator `/rows/5` |  |  |  |  |
| 3 | [Gemini 1.5 Flash](https://benchize.com/models/gemini-1-5-flash.md) | 14source score | CI 5–23 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 14 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemini 1.5 Flash sourceModelVersion gemini-1.5-flash-preview-0514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 5–23 contaminationWarning Not reported rank 4 reportedConfidenceHalfWidth 9 ### Source [Original source](https://labs.scale.com/leaderboard/adversarial_robustness) Source scale Original model label Gemini 1.5 Flash Observed 2026-10-21T16:35:39.820Z Evidence SHA-256 `9ad617f4026dbd8729d7a28d03eca7818e4f0f6e7e03675760b4a0d2f1ab8f47` Source locator `/rows/3` |  |  |  |  |
| 4 | [Claude 3 Opus](https://benchize.com/models/claude-3-opus.md) | 13source score | CI 4–22 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 13 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude 3 Opus sourceModelVersion claude-3-opus-20240229 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 4–22 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 9 ### Source [Original source](https://labs.scale.com/leaderboard/adversarial_robustness) Source scale Original model label Claude 3 Opus Observed 2026-10-21T16:35:39.820Z Evidence SHA-256 `9ad617f4026dbd8729d7a28d03eca7818e4f0f6e7e03675760b4a0d2f1ab8f47` Source locator `/rows/2` |  |  |  |  |

---
Cite as: Adversarial Robustness — Benchize, https://benchize.com/benchmarks/scale%3Aadversarial_robustness (retrieved 2026-10-21)