# Japanese

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Ajapanese. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Ajapanese.

The Scale AI Multilingual Prompts Dataset, composed of 1,000 prompts per language, is tailored to enhance models’ interaction capabilities across multiple languages. This dataset specifically aims to refine chatbots' proficiency in engaging with Japanese users, reflecting complexity of Japanese communication. This dataset introduces single-turn prompts across a diversified range of scenarios, aiming to evaluate and improve models’ responses in both general and culturally nuanced conversations.

Canonical URL: https://benchize.com/benchmarks/scale%3Ajapanese
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

The Scale AI Multilingual Prompts Dataset, composed of 1,000 prompts per language, is tailored to enhance models’ interaction capabilities across multiple languages. This dataset specifically aims to refine chatbots' proficiency in engaging with Japanese users, reflecting complexity of Japanese communication. This dataset introduces single-turn prompts across a diversified range of scenarios, aiming to evaluate and improve models’ responses in both general and culturally nuanced conversations.

Language & multilingual · Models tested: 4 · Score publishers: [scale](https://labs.scale.com/leaderboard/japanese)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [O3 Mini](https://benchize.com/models/o3-mini.md) | 1,104source score | CI 1,073–1,135 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 1104 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o3-mini sourceModelVersion o3-mini-2025-01-31 (high) thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 1,073–1,135 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 31 ### Source [Original source](https://labs.scale.com/leaderboard/japanese) Source scale Original model label o3-mini Observed 2026-10-21T16:35:43.483Z Evidence SHA-256 `4e06382afea8e1dcd069bf285565181b1fcc5d2c9a59b10f055209d04c438654` Source locator `/rows/2` |  |  |  |  |
| 2 | [Gemini 1.5 Flash](https://benchize.com/models/gemini-1-5-flash.md) | 966source score | CI 910–1,022 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 966 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemini 1.5 Flash sourceModelVersion gemini-1.5-flash-preview-0514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 910–1,022 contaminationWarning Not reported rank 12 reportedConfidenceHalfWidth 56 ### Source [Original source](https://labs.scale.com/leaderboard/japanese) Source scale Original model label Gemini 1.5 Flash Observed 2026-10-21T16:35:43.483Z Evidence SHA-256 `4e06382afea8e1dcd069bf285565181b1fcc5d2c9a59b10f055209d04c438654` Source locator `/rows/11` |  |  |  |  |
| 3 | [Gemma 2 27B](https://benchize.com/models/gemma-2-27b-it.md) | 892source score | CI 850–934 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 892 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Gemma 2 27B sourceModelVersion gemma-2-27b thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 850–934 contaminationWarning Not reported rank 15 reportedConfidenceHalfWidth 42 ### Source [Original source](https://labs.scale.com/leaderboard/japanese) Source scale Original model label Gemma 2 27B Observed 2026-10-21T16:35:43.483Z Evidence SHA-256 `4e06382afea8e1dcd069bf285565181b1fcc5d2c9a59b10f055209d04c438654` Source locator `/rows/14` |  |  |  |  |
| 4 | [Llama 3.3 70B Instruct](https://benchize.com/models/llama-3-3-70b-instruct.md) | 778source score | CI 743–813 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 778 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Llama 3.3 70B Instruct sourceModelVersion llama-3-3-70b-instruct thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 743–813 contaminationWarning Not reported rank 16 reportedConfidenceHalfWidth 35 ### Source [Original source](https://labs.scale.com/leaderboard/japanese) Source scale Original model label Llama 3.3 70B Instruct Observed 2026-10-21T16:35:43.483Z Evidence SHA-256 `4e06382afea8e1dcd069bf285565181b1fcc5d2c9a59b10f055209d04c438654` Source locator `/rows/15` |  |  |  |  |

---
Cite as: Japanese — Benchize, https://benchize.com/benchmarks/scale%3Ajapanese (retrieved 2026-10-21)