# Arabic

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Aarabic. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Aarabic.

The Scale AI Multilingual Prompts Dataset, composed of 1,000 prompts per language, is tailored to enhance models’ interaction capabilities across multiple languages. This dataset specifically aims to refine chatbots' proficiency in engaging with Arabic users, reflecting complexity of Arabic communication. This dataset introduces single-turn prompts across a diversified range of scenarios, aiming to evaluate and improve models’ responses in both general and culturally nuanced conversations.

Canonical URL: https://benchize.com/benchmarks/scale%3Aarabic
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

The Scale AI Multilingual Prompts Dataset, composed of 1,000 prompts per language, is tailored to enhance models’ interaction capabilities across multiple languages. This dataset specifically aims to refine chatbots' proficiency in engaging with Arabic users, reflecting complexity of Arabic communication. This dataset introduces single-turn prompts across a diversified range of scenarios, aiming to evaluate and improve models’ responses in both general and culturally nuanced conversations.

Language & multilingual · Models tested: 4 · Score publishers: [scale](https://labs.scale.com/leaderboard/arabic)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [O3 Mini](https://benchize.com/models/o3-mini.md) | 1,093source score | CI 1,063–1,123 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1093 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o3-mini sourceModelVersion o3-mini-2025-01-31 (high) thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 1,063–1,123 contaminationWarning Not reported rank 6 reportedConfidenceHalfWidth 30 ### Source [Original source](https://labs.scale.com/leaderboard/arabic) Source scale Original model label o3-mini Observed 2026-10-21T16:35:42.378Z Evidence SHA-256 `7345ef9704ce4cb50fca32d9b2234bd7051deae3c0c303e2d3c6dc464dcb403e` Source locator `/rows/5` |  |  |  |  |
| 2 | [Gemini 1.5 Flash](https://benchize.com/models/gemini-1-5-flash.md) | 967source score | CI 928–1,006 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 967 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemini 1.5 Flash sourceModelVersion gemini-1.5-flash-preview-0514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 928–1,006 contaminationWarning Not reported rank 14 reportedConfidenceHalfWidth 39 ### Source [Original source](https://labs.scale.com/leaderboard/arabic) Source scale Original model label Gemini 1.5 Flash Observed 2026-10-21T16:35:42.378Z Evidence SHA-256 `7345ef9704ce4cb50fca32d9b2234bd7051deae3c0c303e2d3c6dc464dcb403e` Source locator `/rows/13` |  |  |  |  |
| 3 | [Llama 3.3 70B Instruct](https://benchize.com/models/llama-3-3-70b-instruct.md) | 808source score | CI 772–844 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 808 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Llama 3.3 70B Instruct sourceModelVersion llama-3-3-70b-instruct thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 772–844 contaminationWarning Not reported rank 17 reportedConfidenceHalfWidth 36 ### Source [Original source](https://labs.scale.com/leaderboard/arabic) Source scale Original model label Llama 3.3 70B Instruct Observed 2026-10-21T16:35:42.378Z Evidence SHA-256 `7345ef9704ce4cb50fca32d9b2234bd7051deae3c0c303e2d3c6dc464dcb403e` Source locator `/rows/16` |  |  |  |  |
| 4 | [Gemma 2 27B](https://benchize.com/models/gemma-2-27b-it.md) | 661source score | CI 632–690 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 661 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemma 2 27B sourceModelVersion gemma-2-27b thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 632–690 contaminationWarning Not reported rank 19 reportedConfidenceHalfWidth 29 ### Source [Original source](https://labs.scale.com/leaderboard/arabic) Source scale Original model label Gemma 2 27B Observed 2026-10-21T16:35:42.378Z Evidence SHA-256 `7345ef9704ce4cb50fca32d9b2234bd7051deae3c0c303e2d3c6dc464dcb403e` Source locator `/rows/18` |  |  |  |  |

---
Cite as: Arabic — Benchize, https://benchize.com/benchmarks/scale%3Aarabic (retrieved 2026-10-21)