# AudioMultiChallenge

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/scale%3Aaudiomc. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/scale%3Aaudiomc.

Audio MultiChallenge is a multi-turn benchmark that evaluates conversational intelligence in spoken dialogue systems and audio-language models, including both speech-to-speech and audio-input models. Unlike ASR- or TTS-based evaluations, it measures whether models can follow instructions, integrate prior context, remain self-consistent, and handle natural speech corrections across extended dialogues. The dataset is built entirely from human speech containing disfluencies, interruptions, non-monotonic phrasing, and ambient noise, conditions under which current models frequently fail. Each conversation concludes with a set of atomic rubrics that precisely define the requirements for the model’s final response.

Canonical URL: https://benchize.com/benchmarks/scale%3Aaudiomc
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Audio MultiChallenge is a multi-turn benchmark that evaluates conversational intelligence in spoken dialogue systems and audio-language models, including both speech-to-speech and audio-input models. Unlike ASR- or TTS-based evaluations, it measures whether models can follow instructions, integrate prior context, remain self-consistent, and handle natural speech corrections across extended dialogues. The dataset is built entirely from human speech containing disfluencies, interruptions, non-monotonic phrasing, and ambient noise, conditions under which current models frequently fail. Each conversation concludes with a set of atomic rubrics that precisely define the requirements for the model’s final response.

Multimodal · Models tested: 6 · Score publishers: [scale](https://labs.scale.com/leaderboard/audiomc)

## Model results

| # | Model | Score | ± |  |
| --- | --- | --- | --- | --- |
| 1 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 60.4source score | CI 55.91–64.89 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 60.4 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort high Model configuration gemini-3.8-flash (high) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 55.91–64.89 contaminationWarning Not reported rank 1 reportedConfidenceHalfWidth 4.49 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label gemini-3.8-flash (high) Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/0` |  |  |  |  |
| 2 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 58.41source score | CI 53.88–62.94 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 58.41 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort high Model configuration gemini-4-argon (high) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 53.88–62.94 contaminationWarning Not reported rank 3 reportedConfidenceHalfWidth 4.53 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label gemini-4-argon (high) Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/2` |  |  |  |  |
| 3 | [Voxtral Small 24B 2507](https://benchize.com/models/voxtral-small-24b-2507.md) | 26.33source score | CI 22.28–30.38 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 26.33 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Voxtral-Small-24B-2507* sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 22.28–30.38 contaminationWarning Not reported rank 16 reportedConfidenceHalfWidth 4.05 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label Voxtral-Small-24B-2507* Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/15` |  |  |  |  |
| 4 | [Gemini 2.5 Flash](https://benchize.com/models/gemini-2-5-flash.md) | 26.11source score | CI 22.07–30.15 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 26.11 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemini-2.5-flash* sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 22.07–30.15 contaminationWarning Not reported rank 17 reportedConfidenceHalfWidth 4.04 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label gemini-2.5-flash* Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/16` |  |  |  |  |
| 5 | [Phi 4 Multimodal Instruct](https://benchize.com/models/phi-4-multimodal-instruct.md) | 15.49source score | CI 12.16–18.82 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 15.49 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Phi-4-multimodal-instruct* sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 12.16–18.82 contaminationWarning Not reported rank 28 reportedConfidenceHalfWidth 3.33 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label Phi-4-multimodal-instruct* Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/27` |  |  |  |  |
| 5 | [Gemma 3n e4b It](https://benchize.com/models/gemma-3n-e4b-it.md) | 15.49source score | CI 12.16–18.82 |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 15.49 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration gemma-3n-E4B-it* sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 12.16–18.82 contaminationWarning Not reported rank 27 reportedConfidenceHalfWidth 3.33 ### Source [Original source](https://labs.scale.com/leaderboard/audiomc) Source scale Original model label gemma-3n-E4B-it* Observed 2026-10-21T16:35:33.690Z Evidence SHA-256 `1d659c306faff19bd29de2470af9e472df8aeaac8d655db5258a1e8679c090aa` Source locator `/rows/26` |  |  |  |  |

---
Cite as: AudioMultiChallenge — Benchize, https://benchize.com/benchmarks/scale%3Aaudiomc (retrieved 2026-10-21)