# AudioMultiChallenge - Text Output

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Aaudiomc-text. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Aaudiomc-text.

Audio MultiChallenge is a multi-turn benchmark that evaluates conversational intelligence in spoken dialogue systems and audio-language models, including both speech-to-speech and audio-input models. Unlike ASR- or TTS-based evaluations, it measures whether models can follow instructions, integrate prior context, remain self-consistent, and handle natural speech corrections across extended dialogues. The dataset is built entirely from human speech containing disfluencies, interruptions, non-monotonic phrasing, and ambient noise, conditions under which current models frequently fail. Each conversation concludes with a set of atomic rubrics that precisely define the requirements for the model’s final response.

Canonical URL: https://benchize.com/benchmarks/scale%3Aaudiomc-text
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Audio MultiChallenge is a multi-turn benchmark that evaluates conversational intelligence in spoken dialogue systems and audio-language models, including both speech-to-speech and audio-input models. Unlike ASR- or TTS-based evaluations, it measures whether models can follow instructions, integrate prior context, remain self-consistent, and handle natural speech corrections across extended dialogues. The dataset is built entirely from human speech containing disfluencies, interruptions, non-monotonic phrasing, and ambient noise, conditions under which current models frequently fail. Each conversation concludes with a set of atomic rubrics that precisely define the requirements for the model’s final response.

Multimodal · Models tested: 8 · Score publishers: [scale](https://labs.scale.com/leaderboard/audiomc-text)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 60.4source score | CI 55.91–64.89 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 60.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort high Model configuration gemini-3.8-flash (high) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 55.91–64.89 contaminationWarning Not reported rank 1 reportedConfidenceHalfWidth 4.49 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemini-3.8-flash (high) Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/0` |  |  |  |  |
| 2 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 58.41source score | CI 53.88–62.94 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 58.41 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort high Model configuration gemini-4-argon (high) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 53.88–62.94 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 4.53 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemini-4-argon (high) Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/2` |  |  |  |  |
| 3 | [Gemini 2.5 Pro](https://benchize.com/models/gemini-2-5-pro-thinking.md)Thinking | 46.9source score | CI 42.32–51.48 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 46.9 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-2.5-pro (Thinking) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 42.32–51.48 contaminationWarning Not reported rank 7 reportedConfidenceHalfWidth 4.58 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemini-2.5-pro (Thinking) Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/6` |  |  |  |  |
| 4 | [Gemini 2.5 Flash](https://benchize.com/models/gemini-2-5-flash-thinking.md)Thinking | 40.04source score | CI 35.54–44.54 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 40.04 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-2.5-flash (Thinking) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 35.54–44.54 contaminationWarning Not reported rank 8 reportedConfidenceHalfWidth 4.5 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemini-2.5-flash (Thinking) Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/7` |  |  |  |  |
| 5 | [Voxtral Small 24B 2507](https://benchize.com/models/voxtral-small-24b-2507.md) | 26.33source score | CI 22.28–30.38 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 26.33 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Voxtral-Small-24B-2507 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 22.28–30.38 contaminationWarning Not reported rank 10 reportedConfidenceHalfWidth 4.05 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label Voxtral-Small-24B-2507 Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/9` |  |  |  |  |
| 6 | [Gemini 2.5 Flash](https://benchize.com/models/gemini-2-5-flash.md) | 26.11source score | CI 22.07–30.15 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 26.11 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-2.5-flash sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 22.07–30.15 contaminationWarning Not reported rank 11 reportedConfidenceHalfWidth 4.04 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemini-2.5-flash Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/10` |  |  |  |  |
| 7 | [Phi 4 Multimodal Instruct](https://benchize.com/models/phi-4-multimodal-instruct.md) | 15.49source score | CI 12.16–18.82 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 15.49 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Phi-4-multimodal-instruct sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 12.16–18.82 contaminationWarning Not reported rank 17 reportedConfidenceHalfWidth 3.33 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label Phi-4-multimodal-instruct Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/16` |  |  |  |  |
| 7 | [Gemma 3n e4b It](https://benchize.com/models/gemma-3n-e4b-it.md) | 15.49source score | CI 12.16–18.82 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 15.49 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemma-3n-E4B-it sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 12.16–18.82 contaminationWarning Not reported rank 18 reportedConfidenceHalfWidth 3.33 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc-text) Source scale Original model label gemma-3n-E4B-it Observed 2026-10-21T16:35:34.434Z Evidence SHA-256 `cba4a93a229e658c6a1f1b2e426b552c7328c1ba23a1f6675af6c651d6ed951f` Source locator `/rows/17` |  |  |  |  |

---
Cite as: AudioMultiChallenge - Text Output — Benchize, https://benchize.com/benchmarks/scale%3Aaudiomc-text (retrieved 2026-10-21)