# HMMT Feb 2025

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2025. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2025.

HMMT Feb 2025: Solving final-answer mathematics competition problems. This competition contains 30 problems. MathArena reports normalized performance across repeated attempts and a 95% confidence interval.

Canonical URL: https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2025
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Conquered

HMMT Feb 2025: Solving final-answer mathematics competition problems. This competition contains 30 problems. MathArena reports normalized performance across repeated attempts and a 95% confidence interval.

Mathematics · Models tested: 34 · Score publishers: [matharena](https://matharena.ai/?comp=hmmt--hmmt_feb_2025)

## Model results

| # | Model | Score | ± | Cost |  |
| --- | --- | --- | --- | --- | --- |
| 1 | [GPT-5.2](https://benchize.com/models/gpt-5-2-xhigh.md)Extra high | 100% | 95% CI 100–100% | 0.24usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.24 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort xhigh sourceModelName GPT-5.2 (xhigh) sourceModelVersion openai/gpt-52 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 100–100% confidenceInterval95 0.00 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5.2 (xhigh) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/0` |  |  |  |  |  |
| 2 | [Step 3.5 Flash](https://benchize.com/models/step-3-5-flash.md) | 98.33% | 95% CI 96.04–100.62% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 98.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.014 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Step 3.5 Flash sourceModelVersion stepfun/3.5-flash ### Reported statistics Uncertainty 95% confidence interval: 96.04–100.62% confidenceInterval95 2.29 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Step 3.5 Flash Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/1` |  |  |  |  |  |
| 2 | [GPT-5.2](https://benchize.com/models/gpt-5-2-high.md)High | 98.33% | 95% CI 96.04–100.62% | 0.16usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 98.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.16 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT-5.2 (high) sourceModelVersion openai/gpt-52-high thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 96.04–100.62% confidenceInterval95 2.29 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5.2 (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/2` |  |  |  |  |  |
| 4 | [Gemini 3 Flash](https://benchize.com/models/gemini-3-flash.md) | 97.5% | 95% CI 94.71–100.29% | 0.06usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 97.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.062 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Gemini 3 Flash sourceModelVersion gemini/gemini-3-flash ### Reported statistics Uncertainty 95% confidence interval: 94.71–100.29% confidenceInterval95 2.79 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Gemini 3 Flash Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/4` |  |  |  |  |  |
| 4 | [GLM-5](https://benchize.com/models/glm-5.md) | 97.5% | 95% CI 94.71–100.29% | 0.09usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 97.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.09 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName GLM 5 sourceModelVersion glm/glm-5 ### Reported statistics Uncertainty 95% confidence interval: 94.71–100.29% confidenceInterval95 2.79 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GLM 5 Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/5` |  |  |  |  |  |
| 6 | [GLM-4.6](https://benchize.com/models/glm-4-6.md) | 93.33% | 95% CI 88.87–97.79% | 0.05usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.047 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName GLM 4.6 sourceModelVersion glm/glm-46 ### Reported statistics Uncertainty 95% confidence interval: 88.87–97.79% confidenceInterval95 4.46 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GLM 4.6 Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/8` |  |  |  |  |  |
| 6 | [GPT-5.1](https://benchize.com/models/gpt-5-1-high.md)High | 93.33% | 95% CI 88.87–97.79% | 0.22usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.22 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT-5.1 (high) sourceModelVersion openai/gpt-51 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 88.87–97.79% confidenceInterval95 4.46 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5.1 (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/11` |  |  |  |  |  |
| 8 | [gpt-oss-120b](https://benchize.com/models/gpt-oss-120b-high.md)High | 90% | 95% CI 84.63–95.37% | 0.02usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.016 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT OSS 120B (high) sourceModelVersion openai/oss-120b thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 84.63–95.37% confidenceInterval95 5.37 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT OSS 120B (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/16` |  |  |  |  |  |
| 9 | [GPT-5 Mini](https://benchize.com/models/gpt-5-mini-high.md)High | 89.17% | 95% CI 83.61–94.73% | 0.03usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.17 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.034 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT-5-mini (high) sourceModelVersion openai/gpt-5-mini thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 83.61–94.73% confidenceInterval95 5.56 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5-mini (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/17` |  |  |  |  |  |
| 10 | [GPT-5](https://benchize.com/models/gpt-5-high.md)High | 88.33% | 95% CI 82.59–94.07% | 0.17usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.17 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT-5 (high) sourceModelVersion openai/gpt-5 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 82.59–94.07% confidenceInterval95 5.74 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5 (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/18` |  |  |  |  |  |
| 11 | [Gemini 2.5 Pro](https://benchize.com/models/gemini-2-5-pro.md) | 82.5% | 95% CI 75.7–89.3% | 0.13usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.13 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Gemini 2.5 Pro sourceModelVersion gemini/gemini-pro-2.5 ### Reported statistics Uncertainty 95% confidence interval: 75.7–89.3% confidenceInterval95 6.80 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Gemini 2.5 Pro Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/22` |  |  |  |  |  |
| 12 | [GLM-4.5](https://benchize.com/models/glm-4-5.md) | 77.5% | 95% CI 70.03–84.97% | 0.06usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.056 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName GLM 4.5 sourceModelVersion glm/glm-45 ### Reported statistics Uncertainty 95% confidence interval: 70.03–84.97% confidenceInterval95 7.47 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GLM 4.5 Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/24` |  |  |  |  |  |
| 12 | [O3](https://benchize.com/models/o3-high.md)High | 77.5% | 95% CI 70.03–84.97% | 0.12usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.12 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName o3 (high) sourceModelVersion openai/o3 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 70.03–84.97% confidenceInterval95 7.47 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o3 (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/25` |  |  |  |  |  |
| 14 | [R1 0528](https://benchize.com/models/deepseek-r1-0528.md) | 76.67% | 95% CI 69.1–84.24% | 0.06usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.056 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName DeepSeek-R1-0528 sourceModelVersion deepseek/deepseek_r1_0528 ### Reported statistics Uncertainty 95% confidence interval: 69.1–84.24% confidenceInterval95 7.57 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label DeepSeek-R1-0528 Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/28` |  |  |  |  |  |
| 14 | [gpt-oss-20b](https://benchize.com/models/gpt-oss-20b-high.md)High | 76.67% | 95% CI 69.1–84.24% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.011 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT OSS 20B (high) sourceModelVersion openai/oss-20b thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 69.1–84.24% confidenceInterval95 7.57 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT OSS 20B (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/27` |  |  |  |  |  |
| 16 | [GPT-5 Nano](https://benchize.com/models/gpt-5-nano-high.md)High | 74.17% | 95% CI 66.34–82% | 0.02usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.17 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.015 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName GPT-5-nano (high) sourceModelVersion openai/gpt-5-nano thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 66.34–82% confidenceInterval95 7.83 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GPT-5-nano (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/30` |  |  |  |  |  |
| 16 | [Grok 3 Mini](https://benchize.com/models/grok-3-mini-high.md)High | 74.17% | 95% CI 66.34–82% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.17 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.011 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName Grok 3 Mini (high) sourceModelVersion xai/grok-3-mini-high thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 66.34–82% confidenceInterval95 7.83 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Grok 3 Mini (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/29` |  |  |  |  |  |
| 18 | [GLM-4.5 Air](https://benchize.com/models/glm-4-5-air.md) | 69.17% | 95% CI 60.91–77.43% | 0.03usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.17 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.031 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName GLM 4.5 Air sourceModelVersion glm/glm-45-air ### Reported statistics Uncertainty 95% confidence interval: 60.91–77.43% confidenceInterval95 8.26 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label GLM 4.5 Air Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/31` |  |  |  |  |  |
| 19 | [O3 Mini](https://benchize.com/models/o3-mini-high.md)High | 67.5% | 95% CI 59.12–75.88% | 0.08usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.078 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort high sourceModelName o3-mini (high) sourceModelVersion openai/o3-mini thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 59.12–75.88% confidenceInterval95 8.38 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o3-mini (high) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/33` |  |  |  |  |  |
| 19 | [O4 Mini](https://benchize.com/models/o4-mini-medium.md)Medium | 67.5% | 95% CI 59.12–75.88% | 0.03usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.032 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort medium sourceModelName o4-mini (medium) sourceModelVersion openai/o4-mini--medium thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 59.12–75.88% confidenceInterval95 8.38 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o4-mini (medium) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/32` |  |  |  |  |  |
| 21 | [Gemini 2.5 Flash](https://benchize.com/models/gemini-2-5-flash-thinking.md)Thinking | 64.17% | 95% CI 55.59–72.75% | 0.1usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.17 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.10 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Gemini 2.5 Flash (Thinking) sourceModelVersion gemini/gemini-flash-2.5 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 55.59–72.75% confidenceInterval95 8.58 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Gemini 2.5 Flash (Thinking) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/36` |  |  |  |  |  |
| 22 | [Qwen3 235B A22B](https://benchize.com/models/qwen3-235b-a22b.md) | 62.5% | 95% CI 53.84–71.16% | 0.009usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.009 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Qwen3-235B-A22B sourceModelVersion qwen/qwen3_235b_a22b ### Reported statistics Uncertainty 95% confidence interval: 53.84–71.16% confidenceInterval95 8.66 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Qwen3-235B-A22B Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/37` |  |  |  |  |  |
| 23 | [Claude Opus 4](https://benchize.com/models/claude-opus-4.md) | 60% | 95% CI 51.23–68.77% | 1.23usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 1.23 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Claude-Opus-4.0 (Think) sourceModelVersion anthropic/claude-opus-4 ### Reported statistics Uncertainty 95% confidence interval: 51.23–68.77% confidenceInterval95 8.77 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Claude-Opus-4.0 (Think) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/38` |  |  |  |  |  |
| 24 | [O3 Mini](https://benchize.com/models/o3-mini-medium.md)Medium | 53.33% | 95% CI 44.4–62.26% | 0.03usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.034 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort medium sourceModelName o3-mini (medium) sourceModelVersion openai/o3-mini--medium thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 44.4–62.26% confidenceInterval95 8.93 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o3-mini (medium) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/39` |  |  |  |  |  |
| 25 | [Grok 3 Mini](https://benchize.com/models/grok-3-mini-low.md)Low | 50.83% | 95% CI 41.89–59.77% | 0.003usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.83 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.003 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort low sourceModelName Grok 3 Mini (low) sourceModelVersion xai/grok-3-mini-low thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 41.89–59.77% confidenceInterval95 8.94 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Grok 3 Mini (low) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/40` |  |  |  |  |  |
| 25 | [Qwen3 30B A3B](https://benchize.com/models/qwen3-30b-a3b.md) | 50.83% | 95% CI 41.89–59.77% | 0.006usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.83 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.006 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Qwen3-30B-A3B sourceModelVersion qwen/qwen3_30b_a3b ### Reported statistics Uncertainty 95% confidence interval: 41.89–59.77% confidenceInterval95 8.94 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Qwen3-30B-A3B Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/41` |  |  |  |  |  |
| 27 | [O1](https://benchize.com/models/o1-medium.md)Medium | 48.33% | 95% CI 39.39–57.27% | 0.89usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.89 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort medium sourceModelName o1 (medium) sourceModelVersion openai/o1 thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 39.39–57.27% confidenceInterval95 8.94 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o1 (medium) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/42` |  |  |  |  |  |
| 28 | [O4 Mini](https://benchize.com/models/o4-mini-low.md)Low | 47.5% | 95% CI 38.57–56.43% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.012 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort low sourceModelName o4-mini (low) sourceModelVersion openai/o4-mini--low thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 38.57–56.43% confidenceInterval95 8.93 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o4-mini (low) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/43` |  |  |  |  |  |
| 29 | [DeepSeek R1](https://benchize.com/models/deepseek-r1.md) | 41.67% | 95% CI 32.85–50.49% | 0.03usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.028 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName DeepSeek-R1 sourceModelVersion deepseek/deepseek_r1 ### Reported statistics Uncertainty 95% confidence interval: 32.85–50.49% confidenceInterval95 8.82 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label DeepSeek-R1 Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/45` |  |  |  |  |  |
| 30 | [O3 Mini](https://benchize.com/models/o3-mini-low.md)Low | 28.33% | 95% CI 20.27–36.39% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.012 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false Reasoning effort low sourceModelName o3-mini (low) sourceModelVersion openai/o3-mini--low thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 20.27–36.39% confidenceInterval95 8.06 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label o3-mini (low) Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/52` |  |  |  |  |  |
| 31 | [Gemini 2.0 Flash](https://benchize.com/models/gemini-2-0-flash.md) | 13.33% | 95% CI 7.25–19.41% | 0.001usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 13.33 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.001 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName gemini-2.0-flash sourceModelVersion gemini/gemini-flash-2.0 ### Reported statistics Uncertainty 95% confidence interval: 7.25–19.41% confidenceInterval95 6.08 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label gemini-2.0-flash Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/54` |  |  |  |  |  |
| 32 | [Gemini 2.0 Pro](https://benchize.com/models/gemini-2-0-pro.md) | 7.5% | 95% CI 2.79–12.21% | 0.01usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.011 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName gemini-2.0-pro sourceModelVersion gemini/gemini-pro-2.0 ### Reported statistics Uncertainty 95% confidence interval: 2.79–12.21% confidenceInterval95 4.71 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label gemini-2.0-pro Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/57` |  |  |  |  |  |
| 33 | [GPT-4o](https://benchize.com/models/gpt-4o.md) | 5.83% | 95% CI 1.64–10.02% | 0.008usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.83 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.008 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName gpt-4o sourceModelVersion openai/gpt-4o ### Reported statistics Uncertainty 95% confidence interval: 1.64–10.02% confidenceInterval95 4.19 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label gpt-4o Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/58` |  |  |  |  |  |
| 34 | [Claude 3.5 Sonnet](https://benchize.com/models/claude-3-5-sonnet.md) | 1.67% | 95% CI -0.62–3.96% | 0.008usd per problem |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.008 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2025 predicted false sourceModelName Claude-3.5-Sonnet sourceModelVersion anthropic/claude-35 ### Reported statistics Uncertainty 95% confidence interval: -0.62–3.96% confidenceInterval95 2.29 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2025) Source matharena Original model label Claude-3.5-Sonnet Source version hmmt--hmmt_feb_2025 Observed 2026-10-21T16:52:35.608Z Evidence SHA-256 `8554225c26eb7a3c2649943ee7cf6b404ec1f7441b20fbdcfccbf35b9a864864` Source locator `/table/rows/59` |  |  |  |  |  |

---
Cite as: HMMT Feb 2025 — Benchize, https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2025 (retrieved 2026-10-21)