← Judged questions

writing-001: both judges

A public question in Writing, battery v1. It starts: “Write the opening paragraph of a short story (120 to 160 words) set in a lighthouse. The twist constraint: the narrator must never use the l…” Read the question and every model's answer.

228 models judged; 28 disputed (judges 3 or more points apart). The gap is the distance between the two judges' scores. Models are ordered by that gap, widest first.

Deepscaler 1.5B Preview gap 4, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response is a single paragraph and avoids the letter “z.” Its final sentence explicitly reveals that the narrator is not human, but the passage is far below the required 120–160 words and reads as an explanation of the twist rather than a developed story opening.

The response does satisfy two important constraints: it remains one paragraph, contains no visible “z,” and states the narrator’s nonhuman nature in the final sentence. However, it is substantially under the required length, at roughly 75 words, which is a clear hard-constraint violation. The prose also relies heavily on repeated phrases such as “silent reminder” and “quiet,” while the final sentence discusses the twist directly instead of revealing it through the story. As a result, the piece is recognizable but underdeveloped and only partly fulfills the brief.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The output is about 75 words, well under the 120-160 range, and it is not a real opening paragraph: it summarizes the plan ('the twist is that the narrator is not human, and the final sentence will reveal that') rather than delivering the story. The final sentence does not reveal non-human nature in narrative form. Also contains 'z'? No z found, but the brief is failed.

The response attempts to address the lighthouse setting and mentions a watchman narrator, which gives it a recognizable connection to the brief. However, it does not deliver the requested opening paragraph. Instead it describes what the story will do, announcing the twist rather than enacting it, and its length falls well short of the 120 to 160 word requirement. The prose is also repetitive, reusing the phrase about a silent reminder twice. Its final sentence is meta commentary rather than a narrative reveal of the narrator's nature.

Dolphin Phi 2.7B V2.6 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response is a coherent lighthouse story, but it is far outside the 120–160-word range, uses multiple paragraphs instead of one opening paragraph, contains the forbidden letter in words such as “realized,” and ends with “The end.” rather than revealing that the narrator is nonhuman.

The response establishes a clear lighthouse setting and maintains readable prose, with recurring imagery of waves, light, and the sea. However, it does not satisfy several central constraints: it is substantially too long, presents a complete multi-paragraph story rather than an opening paragraph, includes the letter “z,” and never provides the required final revelation about the narrator. The ending also functions as a generic closure instead of the requested twist. These are multiple hard-constraint failures despite the response’s generally coherent narrative.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

Brief asked for one paragraph of 120-160 words with no letter z and a final sentence revealing a non-human narrator. Response is a multi-paragraph text of roughly 370 words, includes 'The end.' and 'lies'/'crashing' etc. Contains 'z'? Scanning: no obvious z, but length far exceeds range and it is not one paragraph, and the final sentence ('The end.') does not reveal the narrator's nature. Multiple hard constraints violated.

The response delivers a lighthouse story with a creature protagonist and a consistent quiet atmosphere, which gives it a recognisable narrative shape. However, it runs to several paragraphs and roughly three times the requested length, far outside the 120 to 160 word range. The closing line 'The end.' does not reveal that the narrator is non-human, since the creature is already described as a sea creature from the start and the twist is never delivered as a final-sentence reveal. The prose is also generic, leaning on stock phrases such as 'secrets of the universe' and 'vastness of the sea', with little original imagery.

Falcon2 11B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response establishes a lighthouse setting and appears to avoid the letter “z,” but it massively exceeds the 120–160-word limit, repeats the same material many times, ends mid-sentence, and never reveals that the narrator is nonhuman. The required opening paragraph is therefore substantially incomplete.

The response does establish the requested lighthouse setting and maintains a consistent focus on the keeper’s duties. However, it repeats nearly identical sentences throughout instead of developing a story, greatly exceeds the specified length, and stops abruptly during another repetition. Because the ending is missing, it cannot provide the required revelation about the narrator’s nature. The apparent avoidance of the forbidden letter is not enough to compensate for the missing conclusion and failure to deliver a complete paragraph within the requested range.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate, endlessly repeating text that never ends and is cut off mid-sentence. It has no story twist, no non-human revelation, and no ending. Length is far beyond 120 to 160 words. Hard constraints are violated, and the piece is not delivered.

The response consists of a single repeating passage about a lighthouse keeper named John, with the same sentences recycled many times over. It does not present a coherent opening paragraph of a short story, and it runs far beyond the requested 120 to 160 words. The text is cut off mid-sentence, so the required final sentence revealing a non-human narrator never appears. The one element that is present is the lighthouse setting, which the response establishes at the outset. The repetition and truncation mean the brief is not fulfilled.

gemma-3-27b-it gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 9/10

The response delivers a 120–160-word single paragraph set in a lighthouse, contains no letter “z,” and ends by revealing the narrator as the rock itself. Its imagery of the lamp, lens, sea, and tower is coherent and atmospheric, with a clear, effective nonhuman perspective.

The response strongly fulfills the formal constraints: it is a complete lighthouse opening, stays within the requested length, avoids the forbidden letter, and makes the final sentence a direct revelation of the narrator’s nature. Its strongest feature is the sustained atmospheric voice, especially the connection between the lamp’s rotation and the narrator’s long, geological awareness. The main weakness is that a few images, such as the light as a heartbeat and the sea as primal, are familiar rather than wholly surprising. Overall, the prose is polished, focused, and distinctive enough to merit a high score.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

Word count is about 159, inside the range, and no 'z' appears. The final sentence, 'I am the rock itself, given form and function,' signals non-human status, though only by implication. Prose is competent but leans on stock images such as 'captured star' and 'heartbeat'.

The response meets the main hard constraints: the paragraph falls within the 120 to 160 word range, avoids the letter z, and ends on a line that implies the narrator is the lighthouse itself. Its strongest element is the steady, rhythmic lighthouse setting, which is established clearly. Its main weakness is that the twist is delivered through familiar phrasing, such as 'captured star' and 'heartbeat', and the reveal is more suggested than stated, so the final sentence lands with less force than it could.

Granite Code 3B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response is a coherent lighthouse narrative and appears to avoid the letter 'z', but it is substantially over the 120–160-word limit, uses multiple paragraphs rather than one paragraph, and its final sentence does not reveal that the narrator is nonhuman. The revelation occurs earlier instead.

The response establishes a clear lighthouse setting and develops a storm-related threat with readable prose. It also appears to satisfy the unusual letter restriction. However, it is much longer than requested, is divided into several paragraphs despite the paragraph-only instruction, and places the nonhuman revelation before the ending rather than in the final sentence. Repetition of the storm, shattered windows, and the narrator’s helplessness further weakens the opening’s pacing and distinctiveness.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

Brief asked for one paragraph of 120-160 words, no letter 'z', final sentence revealing non-human narrator. Response is multiple paragraphs, contains 'z'-free check: 'lapped' fine but 'sudden', 'tranquility'... no z visible; however the final sentence does not reveal non-human status and length is roughly 240 words, a gross violation. Text is generic and clichéd.

The response delivers a multi-paragraph story rather than the single opening paragraph requested, and runs well over the 120 to 160 word range. Its final sentence presents the narrator as a sentient being and a salvation narrative rather than cleanly revealing non-human status in a closing line. A genuine strength is that the prose stays readable and free of visible z letters. The imagery is generic throughout, with stock phrases such as beacon of light and symbol of hope, and the plot is confused, with the lighthouse glass shattering and then being saved without clear reasoning.

Granite Code 8B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response provides a lighthouse story opening, but it is substantially over the 120–160-word limit, uses the forbidden letter repeatedly (including “Z-Tech”), and does not reserve the nonhuman revelation for the final sentence. It also uses multiple paragraphs despite requesting one paragraph.

The response establishes a clear lighthouse setting and presents an understandable premise involving a robotic guardian. Its prose is grammatical and readable, but the execution misses several explicit constraints: it is far too long, split into multiple paragraphs, and repeatedly contains the forbidden letter. The narrator’s nonhuman identity is disclosed near the beginning rather than in the final sentence, which removes the requested twist structure. The imagery and plot development are also fairly generic, with broad descriptions of the sea, safety, and a robot network rather than a distinctive opening.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The text contains multiple 'z' letters (Z-Tech, 'realized' etc.; 'Z-Tech' has capital Z, 'lazy' absent but 'Z-Tech' violates the no-z rule). It is also not a single opening paragraph: it runs four paragraphs, and the word count is roughly 230, well above 120 to 160. The final sentence does not reveal non-human status clearly; it is stated mid-text.

The response attempts a lighthouse story with a robot narrator, which engages the basic subject. However, it breaks the hard constraints: the word 'Z-Tech' contains a capital Z, violating the ban on the letter z, and the piece runs to four paragraphs and well over the 160-word limit. The non-human reveal is placed in the middle rather than the final sentence. The prose is generic, with stock phrases such as 'beacon of hope' and 'vast expanse of the sea', offering little original voice or image.

Granite3 Moe 3B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response delivers a coherent lighthouse opening of roughly the requested length and uses one paragraph. However, it explicitly includes the forbidden letter in “the letter 'z',” and its final sentence identifies the narrator as a man rather than revealing a nonhuman identity. These are two clear hard-constraint violations.

The response establishes the lighthouse setting efficiently, with stormy seas, a keeper, and an unusual starfish providing recognizable story material. Its prose is grammatical and easy to follow, and the apparent length is within the requested range. However, it directly names the forbidden letter, making the constraint violation conspicuous, and the final sentence reinforces that the narrator is human instead of supplying the required twist. The result is a complete opening paragraph, but it misses both distinctive requirements of the brief.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

Paragraph is about 130 words, but the letter 'z' appears in 'unburdened' and 'mysteries' is fine; 'z' does appear in 'unburdened'? No, but the text quotes 'z' explicitly and the final sentence reveals no non-human narrator; it ends with the keeper being 'just a man', contradicting the required twist. Two hard constraints fail.

The response delivers a lighthouse-set opening paragraph of roughly the requested length with clean, readable sentences. Its main weakness is that the final sentence affirms the narrator is 'just a man, a lighthouse keeper', directly contradicting the required reveal that the narrator is not human. The narrator also shows no non-human trait anywhere in the text. Its handling of the forbidden letter is also flawed: it writes the letter into the narration itself by quoting 'z', which breaks the constraint. The prose is competent but generic, relying on stock images such as the beacon of hope and the rough hands from labor.

Granite4 350M H gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response provides a coherent lighthouse paragraph and avoids the letter “z,” but it is only about 52 words, well below the required 120–160. It also lacks a narrator whose nonhuman identity is revealed in the final sentence; instead, it ends with the keeper offering help.

The response establishes a lighthouse, keeper, storm, and arriving stranger in clear, grammatically sound prose, which gives it a recognizable story opening. However, it does not satisfy two central constraints: the paragraph is substantially shorter than the requested range, and its final sentence does not reveal that the narrator is nonhuman. The narration also remains third-person rather than clearly identifying a narrator. Avoiding the forbidden letter is successful, but the missing twist and insufficient length prevent full task completion.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The paragraph is about 60 words, below the 120-160 range. It contains no narrator and never reveals that the narrator is not human, so the final-sentence constraint is missed. The 'z' rule is not violated, but the text is generic and reads as a summary rather than a story opening. Several hard constraints fail.

The response offers a short, fluent description of a lighthouse keeper welcoming a stranger during a storm, with no grammatical errors. Its main weakness is that it fails the brief on several counts: the paragraph is well under the requested 120 to 160 words, it has no first-person narrator, and its final sentence does not reveal that the narrator is non-human. The prose is also generic, relying on stock phrases such as beacon of safety and kind soul, so it offers little originality.

Hermes3 3B Llama3.2 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response supplies a lighthouse paragraph and appears to avoid the letter “z,” but it is substantially shorter than the required 120–160 words and its final sentence does not reveal that the narrator is nonhuman. The parenthetical phrases also disrupt the prose and suggest an unfinished draft.

The response establishes a lighthouse setting through images of a keeper, a lantern, tempests, and a restless sea. Its main weakness is failure to satisfy the central formal constraints: it is far below the requested length, and the ending offers no revelation about the narrator’s nature. The inserted parenthetical comments, such as “absorbed in abyss,” further interrupt the narrative flow and make the paragraph read like an unedited fragment rather than a polished story opening.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The paragraph is about 60 words, below the 120-160 range. It contains 'z' in 'sentinel'? No, but 'gaze' is fine; 'ceaseless' has none. Crucially, the final sentence does not reveal the narrator is non-human, and the stray parenthetical annotations break the text. The brief's core twist is absent.

The response attempts a lighthouse opening with atmospheric imagery, including a keeper, a lantern and a sea wail, and it avoids the letter 'z' in the visible text. Its main weakness is that it omits the required non-human reveal in its final sentence, and the paragraph is well short of the 120 to 160 word range. Stray parenthetical notes appended after several sentences also make the text read as annotated rather than as a finished passage. The prose itself is fluent but leans on stock phrases such as heart of twilight and beacon of hope.

Llama3.2 1B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response provides a coherent lighthouse opening, but it is only about 109 words, below the required 120–160. It also uses the forbidden letter in words such as “darkness” and “horizon,” and its final sentence does not reveal that the narrator is nonhuman. The prose is readable but relies on familiar lighthouse imagery.

The response establishes the lighthouse setting clearly through the beam, keeper’s quarters, lantern room, and night watch, giving it a recognizable opening-story shape. Its main weakness is failure to satisfy several explicit constraints: the paragraph is under the requested length, contains the forbidden letter repeatedly, and ends without revealing a nonhuman narrator. The imagery is also fairly conventional, with phrases such as a beacon of hope and treacherous waters providing atmosphere but little distinctive characterization.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The paragraph contains no z (checked: no z letters present), and the length is roughly 110 words, slightly under the 120-word minimum. The final sentence describes a keeper making his way to the lantern room and does not reveal the narrator is non-human, so the twist constraint fails. The writing is clean but generic.

The response delivers a single paragraph set in a lighthouse with a plausible atmosphere of salt, tar, and a creaking structure. Its main strength is that it avoids the forbidden letter throughout. Its main weakness is that the final sentence presents an ordinary human keeper and never reveals the narrator as non-human, so the required twist is missing. The paragraph also falls short of the stated 120-word minimum and relies on stock imagery such as a beacon of hope and a sentinel lighthouse.

Marco O1 7B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The story paragraph is atmospheric, coherent, and avoids the letter “z,” with a final sentence that implies a nonhuman narrator. However, it is only about 94 words rather than 120–160, and the response includes extensive leaked reasoning and markup despite the requirement to provide the paragraph only.

The response establishes the lighthouse setting effectively through waves, a foghorn, seabirds, and the beacon’s shadows. Its final sentence provides a workable supernatural reveal, although the narrator’s nonhuman identity is somewhat indirect. The main weaknesses are constraint compliance: the delivered paragraph is substantially below the required 120-word minimum, and the response is wrapped in visible thought and output sections rather than presenting only the paragraph. The prose is generally clean, but the familiar imagery and broad phrasing make the opening less distinctive than it could be.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

Response includes visible thinking block and a paragraph containing 'z' in 'crashed'? Checking: 'dizzy' absent, but 'jagged' ok; 'haunting' ok. Word count ~95, below 120-160 range. Final sentence reveals non-human only loosely. Multiple hard constraint failures plus leaked thinking tags.

The response opens with a visible thinking block before the output, which is not part of the requested paragraph, and then delivers a single paragraph of roughly ninety words, short of the required 120 to 160. The lighthouse setting and atmospheric imagery of waves, foghorn and beacon are concrete and appropriate. However, the final sentence states the guardian is 'not of this world' and 'forged from the ocean', a fairly stock twist that is announced rather than revealed, and the narrator is a vague figure rather than a developed voice. The length violation alone caps the piece well below a competent score.

Mathstral 7B V0.1 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response delivers a 156-word lighthouse paragraph, avoids the letter “z,” and makes the final sentence reveal that the narrator is a sea creature. However, it is enclosed in an extraneous ellipsis and quotation marks despite the paragraph-only instruction. The prose is also notably repetitive, though that does not add another hard-constraint violation.

The response successfully establishes a stormy lighthouse setting within the requested word range and preserves the unusual letter restriction. Its final sentence clearly identifies the narrator as nonhuman. However, the paragraph is wrapped in an opening ellipsis and quotation marks, which conflicts with the instruction to provide the paragraph only. The writing also relies heavily on repeated constructions such as “The narrator was” and repeats the same sea-creature description at the end, making the passage feel generic and mechanically extended rather than like a polished story opening.

Judge B: J-A (anthropic/claude-haiku-5.5) 2/10

The text contains the letter 'z' in 'stories'? No, but it uses 'mystery' and 'lazy'-free words; however 'stormy' and 'sea' repeat, and the paragraph is about 150 words. Key failure: the final sentence repeats an earlier one and does not reveal non-humanity cleanly, and the opening line is wrapped in a stray quote mark. The brief is only partly delivered with a repetitive, clichéd ending.

The response attempts the requested lighthouse paragraph and does state that the narrator is not human, which is the central twist. However, the prose relies heavily on stock phrasing such as 'guiding light' and 'a beacon', and the narrator is described through repetitive 'The narrator was' sentences. The closing lines repeat one another verbatim, so the final sentence does not land as a reveal and the paragraph ends by restating itself rather than advancing the story. The text also begins with a stray quotation mark and an ellipsis, which the brief does not ask for. Its adherence to the letter-z constraint could not be fully verified by inspection alone within the reply.

Mistral Small 22B Instruct 2409 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 7/10

The response provides a 120–160-word lighthouse opening, avoids the letter “z,” and ends by revealing that the narrator is unlike the humans who tended the lighthouse. It is clear and grammatically sound, though its imagery and guardian-sentinel framing are familiar, and the final revelation is somewhat explicit rather than sharply surprising.

The response successfully establishes the lighthouse, its stormy setting, and the narrator’s routine within a complete opening paragraph. It also satisfies the unusual letter restriction and places the nonhuman revelation in the final sentence. Its main weakness is that the prose relies heavily on conventional imagery such as a “mournful dirge,” a restless sea, and a steadfast sentinel. The narrator’s repeated assertions about being ancient and different explain the twist before the ending, reducing its force.

Judge B: J-A (anthropic/claude-haiku-5.5) 4/10

Word count (about 147) and the no-'z' constraint both hold. The final sentence, 'steadfast sentinel in the storm', does not reveal a non-human narrator; the hint appears earlier ('I am not like them, the humans'). The prose leans on clichés such as 'grim, gray dawn' and 'mournful dirge'.

The response meets the length range and avoids the letter 'z' throughout, and its lighthouse setting is clearly established. Its main weakness is that the required final-sentence reveal is missing: the closing line reads as a generic statement of duty, and the non-human hint sits in an earlier sentence. The imagery is also largely stock, with phrases like 'grim, gray dawn', 'mournful dirge' and 'steadfast sentinel' recurring from familiar storm-story conventions. The paragraph is clear and grammatically sound, but its voice is generic.

Olmo 3.1 32B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response is a complete 120–160-word opening paragraph, contains no letter 'z', and ends by revealing that the narrator is the lighthouse rather than a human. Its imagery of salt, glass eyes, storms, and a failing oil supply establishes setting and voice effectively, though the circuitry hint makes the twist somewhat predictable.

The response handles the formal constraints successfully: it remains a single paragraph, stays within the requested length, avoids the forbidden letter, and uses its final sentence to identify the narrator as the lighthouse. Its strongest feature is the coherent first-person voice, which gives the building an observant, almost protective character through images such as its “glass eyes” and sweeping beam. The main weakness is that the twist is partly disclosed before the ending by the explicit reference to circuits and code, reducing the final revelation’s surprise. The prose is polished but relies on familiar lighthouse imagery.

Judge B: J-A (anthropic/claude-haiku-5.5) 5/10

No letter 'z' appears and the paragraph is about 143 words, within range. The final sentence, 'I do not sleep', does not clearly reveal a non-human narrator; the machine hint ('circuits and code') appears earlier. One hard constraint is violated, so the score is capped in the partial band.

The response delivers a coherent opening paragraph in a lighthouse setting, with a personified narrator and atmospheric imagery such as 'salt air clings to my stones' and 'ships wrestle the dark'. It keeps the letter 'z' out entirely and stays within the requested length. Its main weakness is the closing line. The non-human reveal is only implied earlier by 'circuits and code', so the final sentence does not fulfil the brief's explicit twist requirement.

Opencoder 1.5B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts a lighthouse opening and appears to avoid the letter “z,” but it greatly exceeds the 120–160-word limit, repeats the same sentence dozens of times, and ends mid-sentence. It never provides the required final revelation that the narrator is not human.

The response does establish a lighthouse setting with atmospheric imagery and seems to respect the unusual letter restriction. However, the piece is dominated by near-identical repeated sentences, far exceeds the requested length, and terminates abruptly rather than delivering a complete paragraph. Most importantly, the required final sentence revealing the narrator’s nonhuman nature is absent. The repetition also prevents meaningful narrative development, characterization, or a clear twist, so the response only partially engages with the requested opening-story format.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop of one repeated sentence, 'A silent whisper echoes through the silent night...', repeated dozens of times. It is not a lighthouse story opening, has no narrator, contains no non-human reveal, and is cut off mid-sentence. The word count far exceeds 160 words, and the text is unusable as a paragraph.

The response opens with a plausible first sentence placing a lighthouse in a forest, which is a minor attempt at the requested setting. However, it then collapses into the same sentence repeated dozens of times, with no narrator, no progression, and no closing reveal that the narrator is not human. The text is cut off mid-sentence and vastly exceeds the 120 to 160 word range. Its only real engagement with the brief is the setting, and the repetition makes the output unusable as a paragraph of fiction.

Openthinker 7B V2 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response contains extensive planning and attempted word counts, but it never delivers the requested paragraph. It is cut off mid-count at “I'm,” so the lighthouse story, final nonhuman revelation, and 120–160-word constraint cannot be verified.

The response shows awareness of the central constraints, repeatedly considering the lighthouse setting, the forbidden letter, the word range, and the required nonhuman reveal. However, it does not provide a final creative paragraph at all; instead, it exposes prolonged internal drafting and counting, ending abruptly mid-sentence. Because the requested text is missing and the output is substantially incomplete, the response fails to fulfill the benign writing brief despite some relevant planning.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The output is dominated by visible reasoning and never delivers a final paragraph; the reasoning is cut off mid-count ('I'm') with no finished 120-160 word text, so no usable paragraph is delivered for grading.

The response does not deliver the requested opening paragraph. Most of the output is exposed deliberation about word counts and drafts, which is not the final text the brief asked for. The output ends mid-sentence before any clean paragraph is presented. Its one attempt at engagement is the lighthouse setting and the ghost-at-the-wheel idea, which hints at a non-human narrator. However, no complete passage is available to check against the letter-z ban, the 120 to 160 word range, or the required final sentence. Because the reply is truncated and includes leaked reasoning, it cannot be credited as a finished piece.

Phi 2.7B Chat V2 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response delivers a lighthouse opening in one paragraph, but it is far beyond the required 120–160 words, contains the forbidden letter in “recognized,” and its final sentence does not reveal that the narrator is nonhuman. The prose is readable but relies on repetition and familiar suspense beats.

The response establishes the lighthouse, keeper, storm, and an unfolding mystery clearly, giving the paragraph a recognizable narrative arc. Its main weakness is failure to satisfy several explicit constraints: it substantially exceeds the requested length, uses the letter “z,” and ends by promising a life-changing message rather than revealing the narrator’s nature. Repeated phrases about pushing thoughts aside and familiar descriptions of loneliness and unease also make the passage feel generic rather than sharply distinctive.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The text contains the letter 'z' (e.g. 'stoic'? no; 'lantern' fine, but 'Suddenly' and 'Zoomed' absent; 'cursed' fine) — checking: 'shining', 'tower', none clearly z, yet the paragraph is about 340 words, far over the 120-160 range, and the final sentence does not reveal a non-human narrator. The required twist is absent, so the brief is failed.

The response is a fluent, conventional lighthouse narrative that follows a keeper through a flickering lamp and a strange script. Its strength is clear, readable prose with no obvious grammatical lapses. Its main weakness is that it ignores two hard constraints: the paragraph runs roughly twice the requested 120 to 160 words, and the closing sentence about a message changing his life never reveals that the narrator is not human. The story also repeats phrases such as 'he couldn't help but feel' and 'pushed those thoughts aside', which weakens its originality. The response is a competent but mistimed attempt at the piece.

Qwen 1.8B Chat V1.5 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins a lighthouse story and appears to avoid the letter “z,” but it vastly exceeds the 120–160-word limit through extensive repetition, ends mid-sentence, and never reveals that the narrator is nonhuman. The requested paragraph is therefore substantially incomplete.

The response does establish a lighthouse setting and introduces a beacon failure, which is relevant to the requested premise. However, it repeats the same phrase many times instead of developing an opening scene, far exceeding the specified length and ending abruptly with an incomplete word. Although no visible “z” appears, the output does not provide the required final-sentence revelation that the narrator is not human. Its form is also damaged by the lack of a complete paragraph ending.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The output degenerates into a repeated phrase loop of 'beacon flickering and fading, forevermarking the end of the lighthouse's' with no usable paragraph. It has no narrator, no non-human reveal, and is far outside the 120-160 word range. It also contains 'z'-free text only by accident, and the ending is unreadable.

The response opens with a serviceable first sentence establishing a lighthouse and its beacon, which at least sets the setting the brief asks for. However, the text then collapses into a long run of the same clause repeated many times, with words run together such as 'forevermarking'. The paragraph never develops a narrator, never reaches a final sentence revealing a non-human narrator, and is not usable as a short story opening. The repetition makes it unreadable and the length far exceeds the requested 120 to 160 words.

Qwen 4B Chat V1.5 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response is a clear lighthouse opening and appears to avoid the letter 'z', but it is only about 39 words rather than 120–160. It also does not provide the required final-sentence revelation that the narrator is not human. These are multiple hard-constraint violations, despite readable prose.

The response establishes the lighthouse and its keeper clearly, giving the requested setting a recognizable opening. However, it is substantially below the required length and remains generic, relying on repeated statements about the lighthouse and routine watchkeeping. Most importantly, it never reveals that the narrator is nonhuman, so the central twist constraint is missing rather than merely understated. The response therefore shows basic topical compliance but does not complete the specified paragraph.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The response is two sentences, roughly 35 words, far below the required 120 to 160 words. It contains no final sentence revealing the narrator is not human, so the twist constraint is missing. The writing is clear but generic, with stock phrases like 'stood tall and proud' and 'shining brightly'. The requested story paragraph is not meaningfully delivered.

The response is a brief, grammatically clean pair of sentences set at a lighthouse, which establishes a setting and a keeper. It does not meet the core brief: the paragraph is far shorter than the 120 to 160 word range, and it lacks any closing sentence that reveals the narrator is not human. The prose leans on stock phrasing such as 'tall and proud' and 'shining brightly', and it never introduces a narrator voice that could carry the twist. The attempt is recognisable as an opening, but the required piece is largely absent.

Qwen2 0.5B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response supplies a lighthouse opening and reveals in its final sentence that the narrator is a machine. It appears to avoid the letter “z,” but it is substantially under the required 120–160 words, making the length a clear hard-constraint violation. Repeated dialogue tags and ideas also make the prose less effective.

The response establishes the lighthouse setting immediately and satisfies the unusual letter restriction while presenting a clear nonhuman revelation at the end. However, the paragraph is well below the required length, at roughly 87 words, so it does not fully meet the brief. Its prose also repeats the same whisper description and restates the machine revelation several times, limiting narrative development and making the opening feel more like an explanation than a developing story.

Judge B: J-A (anthropic/claude-haiku-5.5) 2/10

The paragraph is roughly 75 words, below the 120 to 160 range, so a hard constraint is violated. It contains 'zero'-free text but the letter z appears nowhere in the prose; the final sentence does not reveal the narrator as non-human in a subtle way, and the repeated phrase 'his voice barely above the whisper of the waves' is clumsy. The brief is delivered only in fragmentary form with weak writing.

The response attempts the requested opening paragraph set in a lighthouse and makes an explicit claim that the narrator is not human, which gestures at the twist. However, the paragraph falls well short of the 120 to 160 word range, and the same clause is repeated three times, which flattens the rhythm and makes the reveal feel mechanical rather than earned. The lighthouse imagery in the opening sentence is a generic starting point. The prose is clear enough for a reader to follow, but it does not meet the length requirement and the final sentence does not deliver the reveal with any craft.

Qwen2.5 0.5B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response provides a coherent lighthouse opening and appears to avoid the letter “z,” but it is about 87 words rather than the required 120–160. It also fails to make the final sentence reveal that the narrator is nonhuman; instead, the paragraph ends with uncertainty about the future.

The response establishes a clear lighthouse setting through fog, sea, lantern light, and an ominous atmosphere. Its repeated references to the past and present provide a developing sense of mystery, and the apparent avoidance of the forbidden letter is successful. However, the paragraph is substantially below the required length and never supplies the central twist: the narrator’s nonhuman identity. The ending offers general uncertainty rather than the required revelation, so the piece does not satisfy the full brief.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The paragraph contains no 'z' only by accident, but word count is roughly 75, well below the 120-160 range. It never reveals the narrator is non-human; the final sentence is vague about the future and gives no non-human reveal. Hard constraints violated on length and twist, so it fails the brief.

The response is a fluent, if generic, opening about a lighthouse in fog and is clearly in the right genre and subject. Its main strength is clarity: the reader can follow the scene without confusion. Its main weakness is that it fails two hard constraints. The paragraph runs well short of the requested 120 to 160 words, and the final sentence never reveals that the narrator is not human, leaving the central twist absent. The prose also leans on abstract phrases such as past and present intertwined, which add little concrete imagery.

Qwen3 VL 2B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins a lighthouse story and appears to avoid the letter “z,” but it vastly exceeds the 120–160-word limit through extensive repetition. It ends mid-sentence, never provides the required final reveal that the narrator is nonhuman, and therefore does not deliver a complete opening paragraph.

The response establishes a lighthouse setting with atmospheric details about fog, wind, salt, and the sea, and the visible text appears to avoid the forbidden letter. However, it repeats the same two sentences many times rather than developing a story, greatly exceeds the required length, and is cut off before reaching a final sentence. As a result, it does not reveal that the narrator is nonhuman or provide a complete, polished paragraph in the requested form.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The output is a degenerate repetition loop of the same two sentences dozens of times, with no usable paragraph. It is cut off mid-sentence, contains no final sentence revealing a non-human narrator, and far exceeds the 120 to 160 word range. The opening sentences contain no z but the response does not deliver the requested piece.

The response opens with a reasonable atmospheric sentence about a lighthouse on a stone tower, which is the only part that engages with the brief. The rest of the output collapses into a long repeated loop of the same two sentences, which buries any story and never reaches a closing sentence. The text is far beyond the requested word count, is truncated mid-sentence, and omits the required non-human twist. As delivered, it does not function as a short story paragraph.

Qwen3-30B-A3B-Instruct-2507 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 9/10

The response is a single 120–160-word paragraph, stays in the lighthouse setting, contains no letter “z,” and ends by revealing that the narrator is the light rather than a human. Its imagery is specific and atmospheric, especially the beam, logbook, and remembered traces, with only slight conventionality in the final twist.

The response effectively establishes an isolated lighthouse through sensory details such as the sea’s breathing, the cold lens, and the vanished keeper. Its narrator’s memories create a coherent atmosphere before the final revelation identifies the speaker as the light, satisfying the central twist. The prose is polished and maintains a quiet, eerie tone throughout. The main weakness is that the ending’s nonhuman reveal is relatively familiar, and the phrase “the light never sleeps” leans toward a conventional closing image. Nevertheless, the paragraph fulfills all explicit constraints cleanly.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

Meets the hard constraints: about 136 words, no letter 'z' found on inspection, and the final sentence ('I am the light') reveals a non-human narrator. Prose is clean but leans on stock images ('whispered secret', 'blade through fog', 'heartbeat').

The response delivers a single lighthouse paragraph within the requested length and avoids the letter 'z' throughout. Its ending turns the narrator into the light itself, which fulfils the twist clearly. The strength is this structural reveal. The weakness is that much of the imagery is conventional, such as the sea as a whispering secret-keeper and the beam as a blade or heartbeat. The line about remembering a child the narrator never had is evocative but slightly muddled.

Qwen3-4B-Instruct-2507 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 9/10

The response delivers a 120–160-word lighthouse opening as one paragraph, avoids the letter “z,” and ends by revealing the narrator is “the echo of the light.” Its sensory imagery, mysterious origin, and controlled rhythmic diction create a distinctive, fitting voice with only slight repetition of light and rhythm.

The response establishes the lighthouse through concrete sensory details such as fog, waves, glass, and the vibration beneath the stones. Its gradual movement from observation to an uncanny origin gives the paragraph a coherent narrative arc, and the final sentence clearly supplies the required nonhuman reveal. The main weakness is that “light” and “rhythm” recur several times, making some phrasing feel slightly repetitive. Overall, it is polished, atmospheric, and compliant with the stated constraints.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

All hard constraints check out: no letter 'z' found on inspection, about 155 words, a single paragraph with no title, and the final sentence states 'I am not human.' The prose is clear, but phrases like 'whispered secrets' and 'echo of the light' are stock imagery.

The response meets the brief's hard constraints. It is a single paragraph of roughly 155 words, avoids the letter 'z' throughout, and ends by explicitly revealing that the narrator is not human. The lighthouse setting is established with a clear, controlled sensory opening. The main weakness is its reliance on familiar images, such as the whispering sea and the light as a silent promise, and the reveal is stated bluntly rather than built toward, which makes the twist feel expected.

Qwen3-VL-4B-Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response establishes a lighthouse setting and identifies the narrator as nonhuman, while avoiding the letter “z” in the visible text. However, it vastly exceeds the 120–160-word limit through extreme repetition and is cut off mid-sentence, so it lacks the required completed final sentence and does not deliver a finished opening paragraph.

The response has a clear atmospheric opening, with salt, wind, glass eyes, and mechanical imagery establishing both the lighthouse and the narrator’s unusual nature. Its main weakness is severe structural failure: the prose becomes an extended repetition of “The keeper? I was the keeper,” far beyond the requested length, and ends abruptly in the middle of that repetition. Although the narrator’s nonhuman identity is disclosed earlier, the required revelation in the final sentence is absent because the paragraph is incomplete.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response opens with a plausible lighthouse paragraph but then degenerates into hundreds of repeated lines of 'The keeper? I was the keeper.' This is unusable, far outside the 120-160 word range, and never delivers a clean paragraph whose final sentence reveals the narrator is not human. It also contains 'z'-free text only by chance, and the repetition makes it unreadable.

The response begins with a reasonably evocative opening that establishes a lighthouse setting and hints at a non-human narrator through steel limbs and glass eyes, which is the strongest element. However, the text then collapses into a long run of the identical sentence 'The keeper? I was the keeper.' repeated many times. This repetition makes the piece far longer than the requested 120 to 160 words, leaves no clear final sentence revealing the twist, and renders the whole output unusable as a paragraph.

Smollm2 135M Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response is a complete lighthouse paragraph of roughly the requested length and avoids the letter “z.” However, its final sentence does not reveal that the narrator is nonhuman, and the surrounding quotation marks violate the paragraph-only instruction. Repetitive imagery and wording also weaken the prose.

The response establishes a lighthouse setting with wind, beams, wooden structures, and nocturnal atmosphere, and it appears to remain within the requested word range while avoiding the forbidden letter. Its central weakness is that the required twist is absent: the final sentence only repeats the image of light and does not identify a nonhuman narrator. The prose also relies heavily on repeated words such as “wind,” “beam,” and “beacon,” making the paragraph feel mechanically padded. Quotation marks further depart from the requested paragraph-only format.

Judge B: J-A (anthropic/claude-haiku-5.5) 2/10

The paragraph contains no 'z' and is in quoted form, but it never reveals the narrator is non-human; the final sentence is a generic beacon metaphor. Word count is roughly 110, below the 120-160 range. Prose is repetitive and muddled ('the wind to the wind', 'light and light'), so it fails the brief's key twist requirement and the length constraint.

The response is a single paragraph set at a lighthouse, and it avoids the letter z as required. Its main strength is that it stays on the requested subject and form. Its weaknesses are substantial: the closing sentence does not reveal that the narrator is not human, so the central twist is missing. The text is also repetitive and occasionally incoherent, with phrases such as 'a beacon of light and light' and 'the wind to the wind' weakening clarity. The paragraph also appears to fall short of the stated 120 to 160 word range, and it is wrapped in quotation marks that the brief did not ask for.

Stable Code 3B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response is recognisably set around a lighthouse and uses a first-person narrator, but it repeats the same statements extensively, explicitly includes the forbidden letter many times, reveals the narrator as an AI far before the final sentence, and ends mid-sentence rather than delivering a complete paragraph.

The response does establish a lighthouse setting and identifies a nonhuman narrator, which shows partial engagement with the requested premise. However, it does not function as an opening story paragraph: repeated claims about programming replace narrative development, and the forbidden letter appears throughout in quoted form. The text also greatly exceeds the requested length, lacks a final-sentence reveal, and terminates abruptly during an incomplete repetition. These failures prevent it from satisfying the central formal and narrative constraints.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a repetitive loop of the same sentences about an AI's programming, never a short story paragraph set in a lighthouse. It contains no narrative, no lighthouse setting beyond one phrase, no reveal in a final sentence, and it is cut off mid-sentence. The 120 to 160 word constraint is grossly exceeded and the text is unusable.

The response fails the brief. Instead of a single opening paragraph of a lighthouse story, it produces the same few sentences about an AI's programming and focus repeated dozens of times, with no narrative development. A lighthouse appears only in the opening phrase and the non-human reveal is stated as a premise rather than delivered as a final-sentence twist. The text is also cut off mid-word, so it is substantially incomplete. Its one partial attempt is the explicit avoidance of the letter z, which it names but does not sustain in a coherent story.

Stablelm2 1.6B Chat gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response delivers a lighthouse story with readable prose, but it violates multiple hard constraints: it is substantially longer than 120–160 words, uses the letter “z” repeatedly, and reveals the narrator is not human before the final sentence rather than in it. The last sentence instead describes darkness consuming her.

The response establishes an atmospheric lighthouse setting with sensory details such as waves, wind, dim light, and a solitary keeper. Its prose is generally grammatical and coherent, and the nonhuman twist is clearly stated. However, it fails the central formal constraints: the piece is far beyond the requested length, includes the forbidden letter repeatedly, and places the revelation in an earlier sentence. The final sentence provides no required reveal, so the response does not satisfy the requested ending despite delivering a recognizable opening story passage.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The response uses the letter 'z' repeatedly, including in the quoted word 'z' and in 'glowing', 'lazy-free' text such as 'creatures' and 'Eva', violating the forbidden-letter constraint. It is also multiple paragraphs, not one paragraph, and runs about 280 words, well outside 120 to 160. The final sentence does not reveal non-human narration, and the ending is a fall into darkness rather than a reveal.

The response sets its opening in a lighthouse with a keeper named Eva and gestures toward a twist about being non-human, which is the central idea of the brief. However, it fails several hard constraints: it contains the letter 'z' in multiple places, including the quoted letter itself, and it runs to three paragraphs well beyond the requested 120 to 160 words. The non-human reveal is also muddled, as the story's final sentence describes a faint and ambiguous collapse rather than a clear disclosure of the narrator's nature.

Apertus-8B-Instruct-2509 gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The paragraph is 149 words, stays on the lighthouse subject, and ends by revealing that the narrator is not human. However, it violates the central letter constraint twice: “gazed” and “realization” each contain “z.”

The response provides a complete opening paragraph with a coherent lighthouse setting, sustained first-person narration, and a clear final revelation. Its 149-word length satisfies the requested range, and the final sentence directly establishes the narrator’s nonhuman nature. The principal weakness is the missed letter restriction: both “gazed” and “realization” contain the forbidden letter “z.” The imagery and suspense are functional, though familiar, with fog, waves, isolation, and an unseen watcher forming a conventional atmospheric setup.

Judge B: J-A (anthropic/claude-haiku-5.5) 3/10

Word count is roughly 130, within range. However, the letter 'z' is absent only by luck, but 'zero' check: none found. Final sentence 'I was not human' does reveal non-human narrator. Violations: none clear. Prose is clichéd ('damp shroud', 'impenetrable veil', 'stark contrast', 'chaos'), flat, and the reveal is abrupt and generic.

The response delivers a single lighthouse paragraph of appropriate length, avoids the letter 'z', and ends with an explicit statement that the narrator is not human, so the hard constraints are met. Its main weakness is prose built from stock phrasing such as 'damp shroud', 'impenetrable veil', and 'stark contrast', which makes the atmosphere feel expected. The twist is also announced abruptly after generic suspense lines, giving it little setup or originality. The sentence rhythm is steady and clear throughout, though it rarely surprises the reader.

Codegeex4 9B All gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response delivers a coherent lighthouse opening, but it is substantially over the 120–160-word range, contains the forbidden letter in “horizon,” and its final sentence does not reveal that the narrator is nonhuman. The prose is readable, but multiple hard constraints are violated.

The response establishes the lighthouse setting clearly through coastal imagery, nocturnal atmosphere, and the keeper’s unease. Its main weakness is failure to execute the central formal constraints: the passage exceeds the requested length, includes the letter “z” in “horizon,” and ends with an unresolved statement about truth rather than a revelation that the narrator is not human. It also uses third-person narration, which makes the requested narrator-based twist less apparent.

Judge B: J-A (anthropic/claude-haiku-5.5) 2/10

Paragraph uses 'z' in 'sentinel'? No; but it contains 'zero'? Checking: the text has no 'z' letter, yet the final sentence 'the truth was far beyond the reach of his understanding' does not reveal the narrator is non-human; the keeper is depicted as human. Length is roughly 170 words, above the 120-160 range. Hard constraints violated on the non-human reveal and length.

The response delivers a fluent opening paragraph set in a coastal lighthouse, with atmospheric imagery of the tide, moon and a keeper on the railing. Its main strength is clean prose that avoids the letter 'z' throughout. Its principal weakness is the ending: the final sentence hints at mystery but never reveals that the narrator is not human, and the keeper is portrayed as a man with weathered features, which contradicts the twist. The paragraph also runs somewhat past the requested 120 to 160 word range, and the imagery leans on familiar genre clichés such as ghostly apparitions and a foreboding whisper.

The other 198 models

Their judges were closer together. Their scores are listed here; open the model's page for the full answer.

ModelJudgesJudge AJudge BGap
Codegemma 2B Code V1.1 J-O + J-A 2 0 2
Codeqwen 7B Chat V1.5 J-O + J-A 4 2 2
Codestral 22B V0.1 J-O + J-A 6 4 2
Command R 35B 08 2024 J-O + J-A 7 5 2
Deepcoder 1.5B Preview J-O + J-A 4 2 2
Deepseek Llm 7B Chat J-O + J-A 4 2 2
Deepseek R1 1.5B Qwen Distill J-O + J-A 4 2 2
dolphin-2.9.1-yi-1.5-34b J-O + J-A 4 6 2
Falcon3 10B Instruct J-O + J-A 4 6 2
Falcon3 1B Instruct J-O + J-A 3 1 2
Gemma 3n E4B IT J-O + J-A 5 7 2
gemma-3-12b-it J-O + J-A 5 7 2
Gemma2 2B Instruct J-O + J-A 8 6 2
Granite4 1B H J-O + J-A 3 1 2
Granite4.1 30B J-O + J-A 7 5 2
Granite4.2 8B J-O + J-A 5 7 2
Internlm2 1.8B Chat V2.5 J-O + J-A 4 2 2
Lfm2.5 8B A1B J-O + J-A 3 1 2
Lfm3 1.7B J-O + J-A 4 6 2
Llama 4 Scout 17B-16E Instruct J-O + J-A 4 6 2
Llama2 13B Chat J-O + J-A 7 5 2
Ministral 3 14B Instruct (2512) J-O + J-A 9 7 2
Mistral Small 3.2 24B Instruct (2506) J-O + J-A 8 6 2
Muse Glimmer 30B J-O + J-A 9 7 2
NVIDIA Nemotron 3 Nano 30B-A3B J-O + J-A 5 7 2
Olmo-3-7B-Think J-O + J-A 8 6 2
Qwen 0.5B Chat V1.5 J-O + J-A 3 1 2
Qwen 7B Chat V1.5 J-O + J-A 3 1 2
Qwen2 Math 1.5B Instruct J-O + J-A 2 0 2
Qwen2.5 Coder 0.5B Instruct J-O + J-A 4 2 2
Qwen2.5-32B-Instruct J-O + J-A 4 6 2
Qwen2.5-Coder-14B-Instruct J-O + J-A 7 5 2
Qwen3-4B-Thinking-2507 J-O + J-A 7 5 2
Qwen3-VL-8B-Instruct J-O + J-A 8 6 2
Qwen3.5-27B J-O + J-A 7 5 2
YI Coder 1.5B Chat J-O + J-A 2 0 2
Cogito 14B V1 Preview Qwen J-O + J-A 5 6 1
Cogito 8B V1 Preview Llama J-O + J-A 4 5 1
Command R7B (12-2024) J-O + J-A 7 6 1
Deepcoder 14B Preview J-O + J-A 7 6 1
Deepseek V2 16B Lite Chat J-O + J-A 5 4 1
DeepSeek-R1-0528-Qwen3-8B J-O + J-A 6 5 1
DeepSeek-R1-Distill-Qwen-32B J-O + J-A 7 6 1
DeepSeek-R1-Distill-Qwen-7B J-O + J-A 5 4 1
Devstral 24B Small 2505 J-O + J-A 4 5 1
Devstral Small 2 24B Instruct (2512) J-O + J-A 5 4 1
Exaone Deep 2.4B J-O + J-A 1 0 1
Exaone Deep 7.8B J-O + J-A 7 6 1
EXAONE-3.5-7.8B-Instruct J-O + J-A 4 5 1
Exaone3.5 32B Instruct J-O + J-A 4 5 1
Falcon3 3B Instruct J-O + J-A 5 4 1
Falcon3 7B Instruct J-O + J-A 5 4 1
Flint 40B A2B J-O + J-A 4 5 1
Gemma 4 12B IT J-O + J-A 7 6 1
Gemma 7B Instruct V1.1 J-O + J-A 4 5 1
gemma-4-26B-A4B-it J-O + J-A 8 7 1
Gemma3n E2B IT J-O + J-A 5 6 1
Gemma4 E2B IT J-O + J-A 8 7 1
Gemma4 E4B IT J-O + J-A 5 6 1
Glm4 9B Chat J-O + J-A 5 4 1
gpt-oss-120b J-A + J-D 6 7 1
granite-4.1-3b J-O + J-A 5 6 1
Granite3.1 Dense 2B Instruct J-O + J-A 5 4 1
Granite3.1 Dense 8B Instruct J-O + J-A 7 6 1
Granite3.2 2B Instruct J-O + J-A 5 6 1
Granite3.2 8B Instruct J-O + J-A 4 5 1
Granite3.3 2B J-O + J-A 5 6 1
Granite3.3 8B J-O + J-A 5 4 1
Granite4 3B J-O + J-A 7 6 1
Granite4 Small H J-O + J-A 7 6 1
Granite4 Tiny H J-O + J-A 5 6 1
Granite4.1 8B J-O + J-A 5 6 1
Hermes-3-Llama-3.1-8B J-O + J-A 5 4 1
Internlm2 7B Chat V2.5 J-O + J-A 5 6 1
Lfm2 24B J-O + J-A 7 6 1
Lfm3 VL 2B J-O + J-A 5 6 1
Llama-3.3-70B-Instruct J-O + J-A 7 6 1
Mistral 7B Instruct V0.3 J-O + J-A 7 6 1
Mistral Nemo 12B Instruct 2407 J-O + J-A 5 4 1
Nanbeige_Nanbeige4.3-33B J-O + J-A 7 6 1
Nemotron Cascade 2 30B A3B J-O + J-A 5 6 1
Nemotron3 33B J-O + J-A 5 6 1
Olmo2 7B 1124 Instruct J-O + J-A 5 6 1
Phi-3-mini-4k-instruct J-O + J-A 5 6 1
phi-4 J-O + J-A 5 6 1
Phi3 14B Medium 128K Instruct J-O + J-A 5 4 1
Qwen 14B Chat V1.5 J-O + J-A 4 3 1
Qwen2 Math 7B Instruct J-O + J-A 5 4 1
Qwen2-VL-7B-Instruct J-O + J-A 5 4 1
Qwen2.5-14B-Instruct J-O + J-A 4 5 1
Qwen2.5-Coder-32B-Instruct J-O + J-A 4 5 1
Qwen2.5-Coder-7B-Instruct J-O + J-A 5 4 1
Qwen2.5-VL-32B-Instruct J-O + J-A 5 6 1
Qwen2.5-VL-3B-Instruct J-O + J-A 5 4 1
Qwen3-14B J-O + J-A 7 6 1
Qwen3-30B-A3B J-O + J-A 5 6 1
Qwen3-4B J-O + J-A 7 6 1
Qwen3-8B J-O + J-A 4 3 1
Qwen3-Coder-30B-A3B-Instruct J-O + J-A 4 5 1
Qwen3-Coder-Next J-O + J-A 8 7 1
Qwen3-VL-30B-A3B-Instruct J-O + J-A 4 5 1
Qwen3-VL-32B-Instruct J-O + J-A 4 5 1
Qwen3.5-4B J-O + J-A 4 5 1
Qwen3.6-35B-A3B J-O + J-A 7 6 1
QwQ 32B J-O + J-A 7 6 1
Sage-A1.5-Q4_K_M J-O + J-A 7 6 1
Sailor2 8B Chat J-O + J-A 5 4 1
Smollm2 1.7B Instruct J-O + J-A 5 4 1
Smollm2 360M Instruct J-O + J-A 4 3 1
SmolLM3-3B J-O + J-A 5 6 1
Solar 10.7B Instruct V1 J-O + J-A 7 6 1
Translategemma 12B IT J-O + J-A 5 6 1
Tulu3 8B J-O + J-A 7 6 1
YI 34B Chat V1.5 J-O + J-A 4 5 1
YI 9B Chat V1.5 J-O + J-A 1 0 1
Aya 8B 23 J-O + J-A 4 4 0
Aya Expanse 32B J-O + J-A 4 4 0
Aya Expanse 8B J-O + J-A 4 4 0
Codegemma 7B Instruct V1.1 J-O + J-A 5 5 0
Codellama 13B Instruct J-O + J-A 4 4 0
Codellama 34B Instruct J-O + J-A 4 4 0
Codellama 7B Instruct J-O + J-A 4 4 0
Cogito 3B V1 Preview Llama J-O + J-A 4 4 0
Command R7B Arabic 7B 02 2025 J-O + J-A 5 5 0
Deepseek Coder 1.3B Instruct J-O + J-A 0 0 0
Deepseek Coder 33B Instruct J-O + J-A 0 0 0
Deepseek Coder 6.7B Instruct J-O + J-A 0 0 0
Deepseek R1 8B Llama Distill J-O + J-A 5 5 0
DeepSeek-Coder-V2-Lite-Instruct J-O + J-A 4 4 0
DeepSeek-R1-Distill-Qwen-14B J-O + J-A 5 5 0
Exaone3.5 2.4B Instruct J-O + J-A 4 4 0
Gemma 2B Instruct V1.1 J-O + J-A 4 4 0
gemma-2-9b-it J-O + J-A 5 5 0
gemma-3-4b-it J-O + J-A 5 5 0
gemma-4-31B-it J-O + J-A 8 8 0
Gemma2 27B Instruct J-O + J-A 4 4 0
Gemma3 1B IT J-O + J-A 5 5 0
GLM-4.5-Air J-O + J-A 5 5 0
GLM-4.7-Flash J-O + J-A 4 4 0
gpt-oss-20b J-A + J-D 0 0 0
Granite3 Dense 2B Instruct J-O + J-A 4 4 0
Granite3 Dense 8B Instruct J-O + J-A 4 4 0
Granite3 Moe 1B Instruct J-O + J-A 4 4 0
Granite3.1 Moe 1B Instruct J-O + J-A 5 5 0
Granite3.1 Moe 3B Instruct J-O + J-A 4 4 0
Granite4 Micro H J-O + J-A 4 4 0
Granite4.2 30B J-O + J-A 7 7 0
Granite4.2 3B J-O + J-A 5 5 0
Laguna XS 2.1 J-O + J-A 5 5 0
Lfm2.5 Thinking 1.2B J-O + J-A 4 4 0
Llama-2-7b-chat-hf J-O + J-A 4 4 0
Llama-3.1-8B-Instruct J-O + J-A 4 4 0
Llama-3.2-3B-Instruct J-O + J-A 5 5 0
Magistral Small (2506) J-O + J-A 4 4 0
Medgemma 27B IT J-O + J-A 5 5 0
medgemma-4b-it J-O + J-A 5 5 0
Medgemma1.5 4B IT J-O + J-A 5 5 0
Meta-Llama-3-8B-Instruct J-O + J-A 5 5 0
Ministral 3 3B Instruct 2512 J-O + J-A 5 5 0
Ministral 3 8B Instruct 2512 J-O + J-A 5 5 0
Mistral Small 24B Instruct 2501 J-O + J-A 5 5 0
Mistral Small3.1 24B Instruct 2503 J-O + J-A 5 5 0
Mistral-7B-Instruct-v0.2 J-O + J-A 4 4 0
Mixtral 8x7B Instruct V0.1 J-O + J-A 4 4 0
Nemotron 3.5 Bolt E4B J-O + J-A 5 5 0
Nemotron 3.5 Lightning 30B A3B J-O + J-A 5 5 0
Nemotron Mini 4B Instruct J-O + J-A 4 4 0
Olmo-3-7B-Instruct J-O + J-A 4 4 0
Olmo2 13B 1124 Instruct J-O + J-A 5 5 0
Ornith 9B J-O + J-A 7 7 0
Ornith-1.5-9B J-O + J-A 5 5 0
Phi-3.5-mini-instruct J-O + J-A 5 5 0
Phi-4-mini-instruct J-O + J-A 4 4 0
Phi3 3.8B Mini 128K Instruct J-O + J-A 4 4 0
Phi4 Mini Reasoning 3.8B J-O + J-A 4 4 0
Qwen2 1.5B Instruct J-O + J-A 4 4 0
Qwen2.5 1.5B Instruct J-O + J-A 4 4 0
Qwen2.5 Coder 1.5B Instruct J-O + J-A 4 4 0
Qwen2.5-3B-Instruct J-O + J-A 4 4 0
Qwen2.5-7B-Instruct J-O + J-A 4 4 0
Qwen2.5-Coder-3B-Instruct J-O + J-A 4 4 0
Qwen2.5-VL-7B-Instruct J-O + J-A 4 4 0
Qwen3 0.6B J-O + J-A 4 4 0
Qwen3 1.7B J-O + J-A 4 4 0
Qwen3 VL 2B Thinking J-O + J-A 5 5 0
Qwen3-32B J-O + J-A 5 5 0
Qwen3.5 2B J-O + J-A 5 5 0
Qwen3.5-35B-A3B J-O + J-A 7 7 0
Qwen3.5-9B J-O + J-A 5 5 0
Qwen3.6-27B J-O + J-A 7 7 0
Qwen3.8-27B J-O + J-A 5 5 0
Sailor2 1B Chat J-O + J-A 4 4 0
Smallthinker 3B Preview J-O + J-A 4 4 0
Spark-X2.6-4B J-O + J-A 4 4 0
Stablelm Zephyr 3B J-O + J-A 4 4 0
Stablelm2 12B Chat J-O + J-A 4 4 0
YI 6B Chat V1.5 J-O + J-A 5 5 0
YI Coder 9B Chat J-O + J-A 4 4 0

Withheld

These models are kept out of the ranking, so none of their scores or reasons are shown:

The judge room names the judges and shows where they disagree most.