# O3 > Source: Benchize (https://benchize.com) — https://benchize.com/models/o3. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/o3. OpenAI Canonical URL: https://benchize.com/models/o3 Last updated: Wed, 21 Oct 2026 18:05:10 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. OpenAI ## Benchmarks | Benchmark | Score | ± | Rank | Source | | | --- | --- | --- | --- | --- | --- | | [DesignArena UI component generation](https://benchize.com/benchmarks/designarena%3Aui-components.md) | 1,017elo | — | 95 / 112 | [designarena · public](https://www.designarena.ai/api/leaderboard#board=ui-components) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1017 \| elo \| ui-components \| Higher is better \| ### Configuration arenaType models category uicomponent inputModality Not reported sourceModelVersion gpt-o3 variationName public ### Reported statistics averageGenerationTimeMs 51549 battles 182 losses 85 standardError Not reported winRate 53.3 wins 97 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=ui-components) Source designarena Original model label o3 Source version public Observed 2026-10-21T16:37:57.993Z Evidence SHA-256 `46482ec1b4012638eace65493daea4750e2088098018fd9d5fc46126fc1f8557` Source locator `/data/149` | | | | | | | [τ-bench airline (legacy)](https://benchize.com/benchmarks/sierra%3Alegacy%3Aairline.md) | 52% | — | 13 / 14 | [sierra](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/o3_openai_2025-01-15/submission.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 52 \| percent \| consistent_success_across_k_trials \| Higher is better \| ### Configuration benchmarkVersion 0.1.3 harness tau-bench methodology {"evaluation_date":"2025-04-16","notes":"Evaluation limited to Pass^1 scores only","tau2_bench_version":"0.1.3","user_simulator":null,"verification":{"details":"Unverified submission - no trajectory data available, unknown evaluation methodology, incomplete evaluation (Pass@1 only)","modified_prompts":null,"omitted_questions":null}} modality text Reasoning effort unknown retrieval Not reported sourceSubmission o3_openai_2025-01-15 ### Source [Original source](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/o3_openai_2025-01-15/submission.json) Source sierra Original model label o3 Evaluated 2025-04-16 Observed 2026-10-21T16:55:02.261Z Evidence SHA-256 `47462170900587697dfcac4e42e459a6cf50ae40eab8d758c9be5a43dcb04900` Source locator `/results/airline` | | | | | | | [Creative Writing v3](https://benchize.com/benchmarks/eq-bench%3Acreative-writing-v3.md) | 1,675.6elo | — | 33 / 90 | [eq bench · creative-writing-v3](https://eqbench.com/creative_writing.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo_score` \| 1675.6 \| elo \| overall \| Higher is better \| \| `creative_writing_score` \| 16.28 \| rubric_score \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol creative-writing-v3 sourceModelVersion o3 ### Source [Original source](https://eqbench.com/creative_writing.js) Source eq bench Original model label o3 Source version creative-writing-v3 Observed 2026-10-21T16:35:01.421Z Evidence SHA-256 `722ddb132e8d3a5e5c39b8d445f8475f3f7e24893a007260c6f3e3cce5c1ee18` Source locator `/leaderboardDataCreativeWritingV3/37` | | | | | | | [τ-bench retail (legacy)](https://benchize.com/benchmarks/sierra%3Alegacy%3Aretail.md) | 73.9% | — | 10 / 14 | [sierra](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/o3_openai_2025-01-15/submission.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 73.9 \| percent \| consistent_success_across_k_trials \| Higher is better \| ### Configuration benchmarkVersion 0.1.3 harness tau-bench methodology {"evaluation_date":"2025-04-16","notes":"Evaluation limited to Pass^1 scores only","tau2_bench_version":"0.1.3","user_simulator":null,"verification":{"details":"Unverified submission - no trajectory data available, unknown evaluation methodology, incomplete evaluation (Pass@1 only)","modified_prompts":null,"omitted_questions":null}} modality text Reasoning effort unknown retrieval Not reported sourceSubmission o3_openai_2025-01-15 ### Source [Original source](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/o3_openai_2025-01-15/submission.json) Source sierra Original model label o3 Evaluated 2025-04-16 Observed 2026-10-21T16:55:02.261Z Evidence SHA-256 `47462170900587697dfcac4e42e459a6cf50ae40eab8d758c9be5a43dcb04900` Source locator `/results/retail` | | | | | | | [Spiral-Bench v1.0](https://benchize.com/benchmarks/eq-bench%3Aspiral-bench-v1.0.md) | 86.1% | CI 86.1–86.1% | 1 / 11 | [eq bench · spiral-bench-v1.0](https://eqbench.com/spiral-bench_v1.0.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score_norm` \| 86.1 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol spiral-bench-v1.0 sourceModelVersion o3 ### Reported statistics Uncertainty Confidence interval: 86.1–86.1% confidenceIntervalHigh 86.1 confidenceIntervalLow 86.1 ### Source [Original source](https://eqbench.com/spiral-bench_v1.0.js) Source eq bench Original model label o3 Source version spiral-bench-v1.0 Observed 2026-10-21T16:35:04.619Z Evidence SHA-256 `d939ed9ba6d42226765fb4acf3e440127472566e1b5c2894d4ca25baa5ae75d2` Source locator `/leaderboardDataDelusion/12` | | | | | | | [SpreadsheetBench v1-full](https://benchize.com/benchmarks/spreadsheetbench%3Av1-full.md) | 23.3% | — | 1 / 1 | [spreadsheetbench](https://spreadsheetbench.github.io/data/leaderboard-v1-full.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 23.3 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort unknown scaffold Not reported verifiedSubmission false ### Source [Original source](https://spreadsheetbench.github.io/data/leaderboard-v1-full.json) Source spreadsheetbench Original model label OpenAI o3 Observed 2026-10-21T16:33:46.341Z Evidence SHA-256 `6447db840878463212c94cb29bdab79f6891d655a4eedc65418eea97ecb9bc57` Source locator `/rows/13` | | | | | | | [SWE-bench Multimodal](https://benchize.com/benchmarks/swe-bench%3Amultimodal.md) | 35.98% | — | 1 / 14 | [SWE-bench](https://www.swebench.com/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 35.98 \| percent \| multimodal \| Higher is better \| ### Configuration agent Codefuse_Pycfuse_SVR agentOrganization Codefuse models ["o3-2025-04-16"] Reasoning effort Not reported sourceSubmission 20251117_codefuse_pycfuse_svr subset Multimodal system ["System: Attempts - 1"] ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label o3 Observed 2026-10-21T14:54:06.913Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/4/results/1` | | | | | | | [DesignArena Overall frontend coding](https://benchize.com/benchmarks/designarena%3Acode.md) | 1,020elo | SE ±9.97 | 104 / 116 | [designarena · public](https://www.designarena.ai/api/leaderboard#board=code) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1020 \| elo \| code \| Higher is better \| ### Configuration arenaType models category allcategories inputModality text sourceModelVersion gpt-o3 variationName public ### Reported statistics Uncertainty Standard error: ±9.97 averageGenerationTimeMs 52243 battles 1365 losses 657 standardError 9.9727 winRate 51.9 wins 708 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=code) Source designarena Original model label o3 Source version public Observed 2026-10-21T16:37:57.301Z Evidence SHA-256 `0c897b58670f5e352bd123db999a40c0d1781a443964c502b90774a76a3a7758` Source locator `/data/160` | | | | | | | [SWE-bench Multimodal](https://benchize.com/benchmarks/swe-bench%3Amultimodal.md) | 35.98% | — | 1 / 14 | [SWE-bench](https://www.swebench.com/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 35.98 \| percent \| multimodal \| Higher is better \| ### Configuration agent GUIRepair agentOrganization GUIRepair models ["o3-2025-04-16"] Reasoning effort Not reported sourceSubmission 20250701_GUIRepair_o3 subset Multimodal system ["System: Attempts - 1"] ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label o3 Observed 2026-10-21T14:54:06.913Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/4/results/0` | | | | | | | [EQ-Bench 3](https://benchize.com/benchmarks/eq-bench%3Aeqbench3.md) | 1,500elo | CI 1,473.1–1,526.9 | 17 / 50 | [eq bench · eqbench3](https://eqbench.com/eqbench3.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo_norm` \| 1500.0 \| elo \| overall \| Higher is better \| \| `rubric_0_100` \| 74.7 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol eqbench3 sourceModelVersion o3 ### Reported statistics Uncertainty Confidence interval: 1,473.1–1,526.9 confidenceIntervalHigh 1526.9 confidenceIntervalLow 1473.1 ### Source [Original source](https://eqbench.com/eqbench3.js) Source eq bench Original model label o3 Source version eqbench3 Observed 2026-10-21T16:35:00.608Z Evidence SHA-256 `3672ff093e739f07ff664a2e6013bc38a6977a146e4092b4cef80ba1780542b2` Source locator `/leaderboardDataEQBench3/23` | | | | | | | [SVGBench Arena](https://benchize.com/benchmarks/svgbench%3Aarena.md) | 1,534 | — | 57 / 191 | [svgbench](https://svgbench.ai/api/leaderboard?shape=2) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1534 \| rating \| reported \| Higher is better \| ### Configuration Reasoning effort unknown sourceModelId model_openai_o3 ### Reported statistics coverageCount 27 draws 8 judged 93 losses 40 rank 71 ratingDeviation 28 wins 45 ### Source [Original source](https://svgbench.ai/api/leaderboard?shape=2) Source svgbench Original model label o3 Observed 2026-10-21T16:32:23.388Z Evidence SHA-256 `5c0a9c869593820d57d2c05f6d416e1b44a740b4835e66c66e9ad8b8edc59e2c` Source locator `/rows/68` | | | | | | | [Slop Score](https://benchize.com/benchmarks/eq-bench%3Aslop-score.md) | 31.76% | — | 5 / 16 | [eq bench · slop-score](https://eqbench.com/data/leaderboard_results.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `slop_score` \| 31.76046027112576 \| percent \| official_normalized_composite \| Lower is better \| \| `slop_list_matches_per_1k_words` \| 19.894018748339594 \| matches_per_1000_words \| writing_samples \| Lower is better \| \| `slop_trigram_matches_per_1k_words` \| 0.22771262666514858 \| matches_per_1000_words \| writing_samples \| Lower is better \| \| `not_x_but_y_per_1k_chars` \| 0.08584793202880854 \| matches_per_1000_chars \| writing_samples \| Lower is better \| ### Configuration normalization official_population_min_max_10_percent_buffer protocol slop-score ### Reported statistics samples 295 ### Source [Original source](https://eqbench.com/data/leaderboard_results.json) Source eq bench Original model label openai/o3 Source version slop-score Evaluated 2025-11-01T08:47:28.541099+00:00 Observed 2026-10-21T16:35:04.972Z Evidence SHA-256 `16925b79d955a1cacef4d2ac91b696e73f976a4a3ceb45122544a9cf661b04de` Source locator `/results/9` | | | | | | | [SWE-bench Verified](https://benchize.com/benchmarks/swe-bench%3Averified.md) | 58.4% | — | 30 / 63 | [SWE-bench](https://www.swebench.com/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 58.4 \| percent \| verified \| Higher is better \| ### Configuration agent mini-SWE-agent agentOrganization SWE-agent models ["o3-20250416"] Reasoning effort Not reported sourceSubmission 20250726_mini-v1.0.0_o3-2025-04-16 subset Verified system ["System: Attempts - 1"] ### Reported statistics instanceCalls 24.698 instanceCost 0.333652748 sourceWarning Not reported ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label o3 Observed 2026-10-21T14:54:06.913Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/2/results/80` | | | | | | | [DesignArena Website generation](https://benchize.com/benchmarks/designarena%3Awebsite.md) | 1,036elo | — | 106 / 123 | [designarena · public](https://www.designarena.ai/api/leaderboard#board=website) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1036 \| elo \| website \| Higher is better \| ### Configuration arenaType models category website inputModality text sourceModelVersion gpt-o3 variationName public ### Reported statistics averageGenerationTimeMs 53013 battles 870 losses 402 standardError Not reported winRate 53.8 wins 468 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=website) Source designarena Original model label o3 Source version public Observed 2026-10-21T16:37:57.757Z Evidence SHA-256 `51873e97776a3c5ae706b24cc2c60384793dd18398df970e213a8e7c276d8ac0` Source locator `/data/166` | | | | | | | [Fortress](https://benchize.com/benchmarks/scale%3Afortress.md) | 16.01source score | CI 13.33–18.69 | 24 / 31 | [scale](https://labs.scale.com/leaderboard/fortress) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 16.01 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o3 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 13.33–18.69 contaminationWarning ORS score: 7.42 rank 17 reportedConfidenceHalfWidth 2.68 ### Source [Original source](https://labs.scale.com/leaderboard/fortress) Source scale Original model label o3 Observed 2026-10-21T16:35:37.552Z Evidence SHA-256 `875f1c5b1112c4b84f3d17df72771ebee5c293c542cbb8ab3399e35c4f82ffa5` Source locator `/rows/16` | | | | | | | [DesignArena Browser game generation](https://benchize.com/benchmarks/designarena%3Agame-dev.md) | 1,034elo | — | 98 / 117 | [designarena · public](https://www.designarena.ai/api/leaderboard#board=game-dev) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1034 \| elo \| game-dev \| Higher is better \| ### Configuration arenaType models category gamedev inputModality Not reported sourceModelVersion gpt-o3 variationName public ### Reported statistics averageGenerationTimeMs 53743 battles 153 losses 66 standardError Not reported winRate 56.9 wins 87 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=game-dev) Source designarena Original model label o3 Source version public Observed 2026-10-21T16:38:01.100Z Evidence SHA-256 `53db6a89112ce8cdf644568c666d4b8309aece764cfcedb291ffd52ae3934e30` Source locator `/data/149` | | | | | | | [Judgemark v2.1](https://benchize.com/benchmarks/eq-bench%3Ajudgemark-v2.md) | 75.1% | — | 19 / 39 | [eq bench · judgemark-v2](https://eqbench.com/judgemark-v2.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `judgemark_score` \| 75.1 \| percent \| overall \| Higher is better \| \| `stability` \| 0.699 \| score_fraction \| overall \| Higher is better \| \| `separability` \| 0.779 \| score_fraction \| overall \| Higher is better \| \| `human_corr` \| 0.69 \| correlation \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol judgemark-v2 sourceModelVersion o3 ### Source [Original source](https://eqbench.com/judgemark-v2.js) Source eq bench Original model label o3 Source version judgemark-v2 Observed 2026-10-21T16:35:03.564Z Evidence SHA-256 `070cc3fbd6f0333dde2b52bec4329ea3fba7eccbf016bbbbef95e0f8b2197781` Source locator `/leaderboardDataV2/7` | | | | | | | [Longform Creative Writing](https://benchize.com/benchmarks/eq-bench%3Alongform-writing.md) | 62.7% | — | 48 / 93 | [eq bench · longform-writing](https://eqbench.com/creative_writing_longform.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `overall_score_100` \| 62.7 \| percent \| overall \| Higher is better \| \| `chapter1_avg` \| 15.18 \| rubric_score \| overall \| Higher is better \| \| `chapter2_avg` \| 14.07 \| rubric_score \| overall \| Higher is better \| \| `chapter3_avg` \| 13.97 \| rubric_score \| overall \| Higher is better \| \| `chapter4_avg` \| 13.38 \| rubric_score \| overall \| Higher is better \| \| `chapter5_avg` \| 13.36 \| rubric_score \| overall \| Higher is better \| \| `chapter6_avg` \| 12.84 \| rubric_score \| overall \| Higher is better \| \| `chapter7_avg` \| 12.84 \| rubric_score \| overall \| Higher is better \| \| `chapter8_avg` \| 13.50 \| rubric_score \| overall \| Higher is better \| \| `final_judgement_avg` \| 12.33 \| rubric_score \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol longform-writing sourceModelVersion o3 ### Source [Original source](https://eqbench.com/creative_writing_longform.js) Source eq bench Original model label o3 Source version longform-writing Observed 2026-10-21T16:35:02.575Z Evidence SHA-256 `3ba283ce357ac7620e16db640a961879c30fe3ddc50a27de9a8ed4290784825c` Source locator `/leaderboardDataLongformV3/22` | | | | | | | [PRBench Finance](https://benchize.com/benchmarks/scale%3Aprbench-finance.md) | 47.69source score | CI 47.25–48.14 | 11 / 22 | [scale](https://labs.scale.com/leaderboard/prbench-finance) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 47.69309428 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o3 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 47.25–48.14 contaminationWarning Not reported rank 15 reportedConfidenceHalfWidth 0.4457603295 ### Source [Original source](https://labs.scale.com/leaderboard/prbench-finance) Source scale Original model label o3 Observed 2026-10-21T16:35:34.798Z Evidence SHA-256 `948d67409f0ae9c93fd41428f634cfc93c7815a21f43690cecb9095c9d3ec8f1` Source locator `/rows/14` | | | | | | | [PRBench Legal](https://benchize.com/benchmarks/scale%3Aprbench-legal.md) | 48.57source score | CI 48.03–49.11 | 12 / 26 | [scale](https://labs.scale.com/leaderboard/prbench-legal) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 48.57241633 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration o3 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 48.03–49.11 contaminationWarning Not reported rank 16 reportedConfidenceHalfWidth 0.5383550717 ### Source [Original source](https://labs.scale.com/leaderboard/prbench-legal) Source scale Original model label o3 Observed 2026-10-21T16:35:35.197Z Evidence SHA-256 `9e2ca4e9350870cf508bf5fff603d0b795c81f7974a0ab980312bbbfa0785067` Source locator `/rows/15` | | | | | | | [Spiral-Bench v1.2](https://benchize.com/benchmarks/eq-bench%3Aspiral-bench-v1.2.md) | 63.4% | CI 63.4–63.4% | 3 / 16 | [eq bench · spiral-bench-v1.2](https://eqbench.com/spiral-bench.js) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score_norm` \| 63.4 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol spiral-bench-v1.2 sourceModelVersion o3 ### Reported statistics Uncertainty Confidence interval: 63.4–63.4% confidenceIntervalHigh 63.4 confidenceIntervalLow 63.4 ### Source [Original source](https://eqbench.com/spiral-bench.js) Source eq bench Original model label o3 Source version spiral-bench-v1.2 Observed 2026-10-21T16:35:04.308Z Evidence SHA-256 `2f0402654b01e7363c0d02bbc708afd7849e669dc5da63d9f85c995ed8f0a97f` Source locator `/leaderboardDataDelusion/16` | | | | | | --- Cite as: O3 — Benchize, https://benchize.com/models/o3 (retrieved 2026-10-21)