Writing & creativity benchmarks

Writing video scripts against a fixed reference style, with separate rubric scores, model-judge preferences and evaluation settings.

Author workflows covering creative writing, editing, translation, instruction following and factual reliability in Novelcrafter’s published evaluation suite.

Short-form creative writing evaluated by LLM judges using pairwise Elo comparisons and writing-quality rubrics. Reported scores are preserved before optional interactive…

Models write short stories to the same constrained creative briefs. Independent evaluators make pairwise comparisons of matched stories in both orders. The current pooled-panel…

Planning, revising and writing an eight-part story from a short prompt. LLM judges assess chapter quality and the completed narrative; the published overall score and…

Writing long-form video scripts for twelve test prompts, assessed by blind model judges on hook, retention, storytelling, voice, pacing and clarity; reported cost is relative to a…

Human pairwise preferences over SVG illustrations generated from the same prompts, reported on the publisher’s rating scale.

LLM-judged creative writing quality on the legacy v2 rubric. Original published scores are retained without the page’s optional vocabulary-complexity or repetitive-language slider…

Repetitive writing patterns measured from stock words, trigrams and contrast constructions. The official composite weights normalized frequencies by 60%, 15% and 25%; lower scores…

Humour analysis evaluated by EQ-Bench, measuring how well language models identify and explain comedic material. The published score and confidence interval are kept independently…

Human pairwise preferences on creative writing prompts in Arena’s text evaluation. The original organizer publishes model ratings and uncertainty; these are preference ratings…

Human pairwise preferences on industry entertainment and sports and media prompts in Arena’s text evaluation. The original organizer publishes model ratings and uncertainty; these…

Human pairwise preferences on industry writing and literature and language prompts in Arena’s text evaluation. The original organizer publishes model ratings and uncertainty;…

Human pairwise preferences on creative writing prompts in Arena’s text style control evaluation. The original organizer publishes model ratings and uncertainty, adjusting for…

Human pairwise preferences on industry entertainment and sports and media prompts in Arena’s text style control evaluation. The original organizer publishes model ratings and…

Human pairwise preferences on industry writing and literature and language prompts in Arena’s text style control evaluation. The original organizer publishes model ratings and…