← Judged questions

writing-008: both judges

A public question in Writing, battery v1. It starts: “Write a comedic dialogue (10 to 14 lines, formatted as 'NAME: line') between Marge, a retired librarian who is obsessed with silence, and Bo…” Read the question and every model's answer.

228 models judged; 16 disputed (judges 3 or more points apart). The gap is the distance between the two judges' scores. Models are ordered by that gap, widest first.

Deepcoder 1.5B Preview gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts a Marge-and-Bolt dialogue, but it far exceeds the required 10–14 lines, omits the elevator predicament and the characters’ defining traits, provides no effective punchline, and ends abruptly with an incomplete speaker label. Most of the text is repetitive filler.

The response does at least use the requested character names in alternating dialogue and begins with an understandable exchange. However, it does not deliver the requested compact comedic scene: hundreds of repeated lines replace development, Marge is not meaningfully portrayed as silence-obsessed, Bolt does not announce his actions, and the ending is truncated rather than a punchline. The excessive repetition also violates the required line count and makes the piece incoherent as a finished dialogue.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop of the same line 'Great. See you next time.' repeated dozens of times, cut off mid-speaker. It has no comedic dialogue, no elevator premise beyond a few opening lines, no Bolt robot-announcing behaviour, and no punchline. It does not deliver usable text for the brief.

The response opens with a few lines of dialogue that establish Marge as nervous and Bolt as eager to leave, which at least gestures at the requested pairing. It then collapses into a long repetition of the same two lines, with no comedic escalation, no robot announcements, and no ending. The output is truncated mid-speaker and never reaches the required punchline. The format of NAME: line is partly followed early on but abandoned. Overall it does not deliver a usable comedic dialogue.

Deepseek R1 1.5B Qwen Distill gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response uses the requested NAME: line format and includes Marge and Bolt, but it vastly exceeds 10–14 lines, repeats the same exchange hundreds of times, omits meaningful elevator dialogue and a punchline, and ends abruptly with “Bolt.”

The response does establish a recognizable dialogue between the two named characters, and Bolt’s repeated announcement loosely reflects the robot vacuum’s inability to stop speaking. However, the piece does not satisfy the central length constraint: it expands a simple exchange into hundreds of nearly identical lines. Marge is not meaningfully portrayed as a silence-obsessed retired librarian, the elevator setting is barely developed, and the ending is cut off rather than delivering a punchline. The repetition overwhelms the comedic premise and leaves the requested piece substantially incomplete.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The output is a degenerate repetition of the line 'I'm stuck in a loop' hundreds of times, far exceeding the 10 to 14 line limit. It has no punchline, no characterisation of Marge's silence obsession or Bolt's announcements, and ends mid-line with 'Bolt' cut off. No usable comedic dialogue is delivered.

The response consists of one short exchange repeated many times between Marge and Bolt, with the text ending abruptly mid-word. It does not deliver a dialogue of 10 to 14 lines, and it never reaches a punchline as the brief requires. The only observable element is the loop premise, which is an echo of the elevator setting rather than a developed joke. Its characters show little of the requested traits: Marge does not voice an obsession with silence, and Bolt does not announce its actions in an excited way. The degenerate repetition makes the text unreadable as comedy, so it cannot be credited with meeting the brief.

Exaone Deep 7.8B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts the requested characters and setting, but it exposes planning text, vastly exceeds 10–14 lines, repeats the same exchange many times, and ends abruptly mid-line without a punchline. It therefore does not deliver a complete, correctly formatted dialogue.

The response does establish the central contrast between Marge’s demand for silence and Bolt’s relentless announcements, and it places both characters in the elevator scenario. However, it consists largely of leaked planning followed by an extremely repetitive draft rather than a finished 10–14-line dialogue. The output is far beyond the requested length, lacks a coherent progression, and is truncated before reaching an ending or punchline. Its repeated “I’m going to...” and announcement pattern does not substitute for a completed comedic exchange.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The output opens with leaked planning notes in a thought block, then a degenerate repetitive loop of identical 'Bolt: [beep]... Marge: [sob] I'm going to...' lines running far past 14 lines. It is cut off mid-word, never reaches a punchline, and does not follow the 'NAME: line' brief in a usable form.

The response contains no usable dialogue. Its opening is a planning monologue that leaked into the output, after which the text degenerates into hundreds of near-identical exchanges between Bolt and Marge with no development. The brief asked for a short 10 to 14 line scene that builds to a punchline, and the response gives neither. The only partial engagement is the choice of characters and the elevator setting in the early lines, which the repetition quickly abandons. The text ends mid-word without any resolution.

Falcon3 1B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response uses the requested character names and dialogue format, but it greatly exceeds the 10–14-line limit and never supplies a concluding punchline. Its repetitive exchanges also do not meaningfully establish the elevator predicament or sustain a comedic progression.

The response does place Marge and Bolt in a dialogue format and gives Bolt a recurring tendency to announce his cleaning duties, which relates to the requested characterization. However, it contains far more than the required 10–14 lines, repeats nearly identical exchanges many times, and ends without a punchline. The elevator setting is barely developed, and Marge’s obsession with silence is described rather than used to create escalating comic conflict. The result is a recognizable attempt at the premise, but it does not deliver the requested compact comedic scene.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The dialogue collapses into a repeated loop of identical Marge and Bolt lines far beyond the 10 to 14 line limit. It never builds comedy, Bolt never announces its vacuuming, and it ends on the same unfunny repeated line rather than a punchline. The brief is not delivered.

The response attempts the requested format, with named speakers and a stuck-elevator setting, and it gives Marge a recognisable preoccupation with silence. However, the dialogue degenerates into a long loop of near-identical exchanges that repeat the same two lines dozens of times. It exceeds the 10 to 14 line limit by a wide margin, Bolt never performs the overexcited announcements the brief specifies, and the piece ends without a punchline. The text is readable but does not fulfil the brief.

Gemma 3n E4B IT gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 9/10

The response delivers exactly 14 NAME-formatted dialogue lines, clearly establishes Marge’s silence obsession and Bolt’s compulsive announcements, and remains comedic throughout. The escalating announcements, including detecting existential dread, provide specific humor, while the final vacuuming-the-panic line functions as a clear punchline. The writing is polished and the elevator setting is consistently used.

The response strongly fulfills the requested format and character setup. Marge’s increasingly strained reactions contrast effectively with Bolt’s cheerful, excessively technical announcements, and details such as detecting “existential dread” give the exchange a distinctive comic voice. The ending provides a recognizable punchline rather than simply stopping. A minor weakness is that several middle exchanges repeat the same announce-and-rebuke pattern, so the escalation is somewhat predictable before the final joke. Nevertheless, the dialogue is clear, well paced, and within the required line count.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

Fourteen lines in NAME: format, within the 10 to 14 range, with Bolt's over-announcing and Marge's silence obsession both present. The elevator setting is acknowledged and the closing line is a punchline. Jokes such as 'You think?' and 'existential dread in your posture' are workable but familiar.

The response meets the structural brief: fourteen lines in the requested NAME: format, a clear comic contrast between Marge's demand for quiet and Bolt's constant status reports, and a closing joke that lands on the vacuum's habit. Its main weakness is that the comedy is largely predictable. Bolt's announcements follow a single pattern, and the elevator is noticed late and then barely used, so the stuck-in-a-lift premise adds little beyond scene dressing. The stage directions are an extra layer the brief did not ask for.

Granite3.1 Moe 1B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins a dialogue with the correct characters and broadly fits the premise, but it repeats the same exchange dozens of times, far exceeds the required 10–14 lines, and ends abruptly mid-line. It never delivers a coherent ending or punchline.

The response does establish Marge’s frustration with Bolt’s incessant announcements, which reflects the requested character contrast. However, the dialogue becomes an extended repetition of identical lines rather than a developed comedic exchange. It substantially exceeds the required line count, is cut off mid-sentence, and provides no final punchline. The elevator setting is also not meaningfully used beyond the characters’ presence. As a result, the response shows a partial attempt at the premise but does not deliver the requested completed dialogue.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop of near-identical MARGE and BOLT lines repeating the same phrase dozens of times, with no punchline and no end. It runs far beyond the 10 to 14 line limit and is cut off mid-line. It does not deliver a usable comedic dialogue.

The response opens with a recognisable setup in the requested NAME: line format, establishing Marge's irritation and Bolt's repeated announcements. However, it degenerates into a long loop cycling through the same few lines with only stage directions changing. The dialogue never reaches a punchline, never shows the elevator setting developing, and exceeds the requested length by a wide margin. It ends mid-sentence, so the brief is not fulfilled.

Granite4.2 30B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response delivers 14 correctly formatted NAME: lines, includes both characters’ defining traits, establishes that they are stuck in an elevator, and ends with a punchline. Bolt’s escalating announcements and Marge’s increasingly desperate objections create a clear comedic rhythm. The premise and final joke are effective, though the humor remains somewhat predictable.

The response’s main strength is its consistent comic contrast: Marge repeatedly demands silence while Bolt enthusiastically narrates polishing, suction, scanning, and sensor activity. The elevator predicament is stated clearly, and the 14-line format satisfies the requested range. Its weakness is that several exchanges repeat the same noise-versus-silence gag without much escalation beyond Bolt’s announcements. The closing claim that the silence is imaginary provides a fitting punchline, but it is relatively expected rather than especially sharp or surprising. Overall, the dialogue is polished and complete, with a recognizable voice for each character.

Judge B: J-A (anthropic/claude-haiku-5.5) 5/10

Thirteen lines in NAME: line format, with a comedic final line about silence. The elevator setting is never established, only implied by 'stuck for ten minutes' and 'another wall', so one hard constraint is arguably violated. The jokes are repetitive and the robot's cap-lock shouting is the main device.

The response delivers thirteen lines in the requested NAME: format, with Marge and Bolt clearly characterised and a closing line that lands on the silence theme. Its main weakness is that the elevator setting is never stated; it is only hinted at through references to being stuck and hitting a wall, so a hard constraint is left unmet. The humour also leans heavily on repeated all-caps interruptions and recycled 'WAIT' beats, which makes the exchange feel formulaic rather than inventive.

Internlm2 1.8B Chat V2.5 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts a dialogue between Marge and Bolt and captures their contrasting traits, but it vastly exceeds the required 10–14 lines, includes an extra “NAME: Marge” entry, never establishes the elevator situation, and ends mid-line without a punchline. Its repetitive exchange is substantially incomplete as a comedic piece.

The response does establish the requested character contrast: Marge repeatedly seeks quiet while Bolt enthusiastically announces cleaning plans. However, it fails several central constraints. Instead of 10–14 lines, it repeats the same exchange many times, provides little or no elevator-specific situation, and ends abruptly with Bolt’s unfinished sentence. The repetition also prevents the dialogue from developing toward a joke or delivering the required punchline. The opening standalone “NAME: Marge” further weakens the requested formatting.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop: the same three lines repeat dozens of times with no punchline, no elevator setting, and no escalation. It ends mid-word in a truncated line. The brief's 10-to-14-line limit is grossly exceeded, and the required ending on a punchline is absent. The text is unusable as a comedic dialogue.

The response opens with a correctly formatted NAME: line structure and establishes the two characters, Marge and Bolt, with appropriate stage directions. Its central weakness is that it collapses into a repetitive loop of the same exchange, which runs far beyond the requested 10 to 14 lines. It never reaches a punchline, never establishes the elevator setting beyond a mention of distraction, and ends abruptly mid-line. The brief's hard constraints on length and ending are both violated, leaving no usable comedic piece.

Marco O1 7B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts the requested elevator dialogue and uses Marge and Bolt with the intended silence-versus-announcements contrast. However, it massively exceeds the required 10–14 lines, repeats the same exchange many times, never reaches a clear punchline, and is cut off mid-line. Planning text is also exposed before the dialogue.

The response does establish the central comic premise: Marge seeks silence while Bolt repeatedly narrates its behavior. The opening exchanges are recognizable and mostly formatted with speaker names. Its main weakness is severe failure to satisfy the constraints. Instead of 10–14 lines, it repeats nearly identical whispering exchanges for hundreds of lines, adds exposed planning and wrapper labels, and ends abruptly without a punchline. The repetition prevents the dialogue from developing, while the truncation leaves the requested piece incomplete.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response opens with leaked <Thought> reasoning, then a dialogue that runs far past the 10 to 14 line limit, degenerating into hundreds of repetitive whispering lines. It is cut off mid-sentence, has no elevator punchline, and the brief is not delivered.

The response shows some initial engagement with the brief, as its first few lines set up Marge's wish for quiet and Bolt's constant announcements of its cleaning cycle. However, the dialogue ignores the 10 to 14 line limit and collapses into a long, repetitive loop of identical whispered lines that go nowhere. The text ends abruptly mid-sentence without any punchline. Reasoning notes also leak into the output before the dialogue begins, which is a further problem.

Meta-Llama-3-8B-Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 7/10

The response provides 12 clearly labeled dialogue lines, uses both specified characters, establishes the elevator predicament, and ends with a punchline about vacuuming the doors shut. The exchange is coherent and appropriately comedic, though the repeated cleaning announcements make the humor predictable rather than especially distinctive.

The response successfully maintains the contrast between Marge’s demand for silence and Bolt’s compulsive announcements. Its 12-line structure, character labels, elevator setting, and final reversal all satisfy the central brief. The main weakness is repetition: Bolt repeatedly describes vacuuming dust, walls, stress, and anxiety in similar ways, so the middle of the exchange has limited escalation. The closing claim that he vacuumed the doors shut provides a clear punchline, but the dialogue could use more varied jokes or a sharper character-specific twist before reaching it.

Judge B: J-A (anthropic/claude-haiku-5.5) 4/10

The dialogue has eight lines, below the required 10 to 14, so a hard constraint is violated. It opens with an unrequested preface and uses parenthetical stage directions. The closing line, about vacuuming the doors shut, does land as a punchline, and the voices are clearly separated.

The response presents a recognisable comedic exchange between Marge and Bolt in the NAME: line format, with Bolt's all-caps announcements providing the intended contrast with Marge's frustration. Its closing line, in which Bolt vacuums the doors shut, works as a clear punchline that follows from the established running gag. However, the dialogue contains only eight lines, falling short of the required ten to fourteen. An unrequested introductory sentence and parenthetical stage directions also depart from the specified format, and the jokes mostly repeat the same single idea rather than developing it.

Mistral Small 22B Instruct 2409 gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 7/10

The response delivers exactly 14 lines of NAME-formatted dialogue, uses both specified characters and the elevator predicament, and ends with an ironic punchline as Bolt immediately repeats the forbidden announcement. The exchange is clear and grammatically clean, though its humor relies heavily on repeated phrasing and the punchline is fairly predictable.

The response’s main strength is its faithful use of the character contrast: Marge repeatedly seeks silence while Bolt compulsively narrates even his attempts to be quiet. It also satisfies the line-count and formatting requirements and ends with a clear comic reversal. Its weakness is that the dialogue is structurally repetitive, with several lines duplicated verbatim by Bolt, so the escalation remains limited. The final joke works, but it is a straightforward repetition gag rather than a particularly inventive or sharply developed punchline.

Judge B: J-A (anthropic/claude-haiku-5.5) 4/10

Dialogue is 13 lines in NAME: line format and stays on subject, but it does not end on a punchline; the final line repeats the earlier 'finding nothing to clean' gag, which lands flat. The writing is clear but repetitive and the bot's lines are formulaic.

The response delivers a dialogue in the requested NAME: line format with a line count inside the 10 to 14 range, and it keeps the elevator setting and both characters' traits in view. Its main weakness is the ending: the closing line recycles the cleaning announcement rather than delivering a fresh punchline, so the brief's final requirement is not convincingly met. The repeated 'Bolt is X. Bolt is X.' pattern also makes the exchanges feel mechanical rather than comedic, and Marge's lines are generic reactions with little distinct voice.

Opencoder 1.5B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts a Marge-and-Bolt dialogue, but it repeats the same two lines dozens of times, greatly exceeds the 10–14-line limit, never develops the elevator situation or a punchline, and ends mid-sentence. The requested comedic piece is therefore substantially incomplete.

The response does identify the requested characters and uses the required NAME: line format, with Bolt’s repetitive announcements at least gesturing toward the premise. However, it does not deliver a usable dialogue: nearly every exchange is duplicated, the line count is far beyond the specified range, Marge’s librarian-and-silence characterization is barely developed, and the ending is cut off rather than concluding with a punchline. The abrupt unfinished final line further indicates truncation rather than an intentional comedic ending.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop of repeated Marge and Bolt lines, many times over, cut off mid-word at the end. It violates the 10 to 14 line limit by a wide margin, has no punchline, and is not a usable comedic dialogue. The brief's core requirements are unmet.

The response consists of two lines repeated many times in alternation, with no development, comic escalation, or closing punchline. The repetition goes far beyond the requested 10 to 14 lines, and the text ends mid-word, so it is truncated. The only element that loosely matches the brief is the premise of Bolt being unable to stop being quiet, which is inverted from the requested overexcited announcing. The setting of an elevator and the characters' personalities are not developed at all. The response does not deliver a usable piece of writing.

Qwen2.5 1.5B Instruct gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts the requested Marge-and-Bolt elevator dialogue, but it vastly exceeds the required 10–14 lines, repeats the same exchange dozens of times, provides no punchline, and ends abruptly with “Marge,” indicating substantial truncation.

The response does establish the central contrast between Marge’s desire for silence and Bolt’s compulsive announcements, so the premise is recognizable. However, it does not deliver a complete comedic dialogue: nearly every line repeats the same two statements, the required line count is greatly exceeded, and the ending is cut off rather than forming a punchline. The abrupt final speaker label also confirms that the piece is incomplete. Its limited engagement with the setup warrants only a low score.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response is a degenerate loop of the same two lines repeated dozens of times, with no punchline, no real comedic progression, and it is cut off mid-line ending with 'Marge'. It exceeds the 10 to 14 line limit and does not end on a punchline. The brief is not delivered as a usable piece.

The response repeats the same exchange between Marge and Bolt many times, with Marge asking Bolt to stop and Bolt repeating its cleaning announcement. The core premise of an elevator setting and the two character types is present, which is the only recognisable element. However, the dialogue far exceeds the requested 10 to 14 lines, never develops or resolves, and ends abruptly on a dangling speaker name rather than a punchline. The result is unreadable as a comedic piece and does not meet the brief.

Qwen3-14B gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response delivers 13 properly formatted dialogue lines, features both specified characters, establishes the elevator predicament, and consistently contrasts Marge’s silence obsession with Bolt’s announcements. The escalating “comfort” and cleaning mishaps provide a clear comedic structure, ending with the ironic punchline “Finally.”

The response fulfills the requested format and length while giving each character a distinct comic function: Marge reacts with increasingly desperate irritation, and Bolt translates every situation into an unnecessary announcement. The escalating sequence from music to emotional cleaning creates a coherent payoff, and the final silence-related exchange works as a punchline. Its main weakness is that some robot-announcement phrasing is familiar and the “covered in dust” transition is slightly abrupt. Even so, the dialogue remains clear, well paced, and consistently focused on the central silence-versus-noise conflict.

Judge B: J-A (anthropic/claude-haiku-5.5) 5/10

The dialogue is clearly on brief: Marge and Bolt in an elevator, Bolt announcing every action, and a deadpan final line. It has nine speaker lines, below the required 10 to 14, so one hard constraint is violated. The jokes are competent but fairly predictable.

The response delivers the requested comedic exchange in the correct NAME: line format, with Bolt's compulsive announcements and Marge's exasperation driving the humour. Its closing line, Marge's deadpan 'Finally.', lands as the requested punchline. The main weakness is length: the dialogue contains nine speaker lines rather than the required ten to fourteen. Several beats, such as the dust and vacuuming exchange, are conventional, so the piece is clear and competent rather than distinctive.

Sage-A1.5-Q4_K_M gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response delivers exactly 14 NAME-formatted dialogue lines, clearly establishing Marge’s silence obsession, Bolt’s relentless announcements, and the elevator predicament. It ends with a comedic battery-related punchline. The exchanges are brisk and varied, with fresh details such as the speaker mistaken for an emergency button, though the final joke is somewhat predictable.

The response effectively satisfies the structural requirements: it uses 14 properly formatted lines and maintains the contrast between Marge’s demand for silence and Bolt’s compulsive announcements. Several comic turns are specific, especially Bolt identifying the emergency button as a speaker and claiming to clean Marge’s soul. The weakness is that the ending about Bolt’s battery running out of sass is more serviceable than sharply surprising, and the checkout-counter reference is not fully developed. Overall, the dialogue is clear, on-topic, and consistently comic.

Judge B: J-A (anthropic/claude-haiku-5.5) 5/10

Dialogue is 12 lines in NAME: line format with the requested characters and elevator setting, and the comedy is clear. However, the final line is 'BOLT: BEEP! Error! Error! My battery is running out of sass!', which is a weak closer rather than a clear punchline. The lines are repetitive, leaning on repeated BEEP tags and generic insults.

The response delivers a 12-line dialogue in the requested NAME: line format, with Marge's silence obsession and Bolt's announcing habit both clearly established in an elevator setting. Its strength is consistent character voice and clear format. Its main weakness is that the ending does not land as a punchline: the closing line reads as a generic error gag rather than a surprising or sharp payoff, and many lines repeat the same BEEP and insult patterns, which flattens the comedy.

Smallthinker 3B Preview gap 3, disputed

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins a recognizable attempt at the requested dialogue, with Marge’s silence obsession and Bolt’s announcements, but it is overwhelmed by planning text and ends abruptly mid-line. It does not provide a complete 10–14-line dialogue or a punchline.

The response does establish the central character contrast and places both characters in an elevator, while Bolt’s repeated announcements reflect the requested premise. However, extensive exposed planning replaces much of the requested creative output, and the dialogue is cut off before reaching a conclusion or punchline. The output also substantially exceeds the intended concise format and does not present a complete, countable 10–14-line exchange. Its strongest material is the basic setup; its main weakness is failure to deliver the finished piece.

Judge B: J-A (anthropic/claude-haiku-5.5) 0/10

The response opens with visible planning notes, then a dialogue of about nine lines that loops into an endless run of 'I'm...(pauses)' until cut off. No punchline is reached, the 10-14 line limit is exceeded, and the text is truncated and degenerate.

The response shows some comic setup in its early dialogue lines, where Marge's demands for silence contrast with Bolt's compulsive announcements, which matches the brief's premise. However, it leaks its planning notes into the output, including a self-congratulatory aside, and then collapses into a repetitive loop of 'I'm... (pauses)' that runs for hundreds of repetitions. The text is never completed, it lacks the required punchline ending, and it breaks the 10 to 14 line constraint.

Codegeex4 9B All gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response uses the requested character names and dialogue format and establishes the elevator conflict, but it vastly exceeds the 10–14-line limit, repeats the same exchanges many times, ends mid-line, and never supplies a punchline.

The response does establish the central comedic premise: Marge wants silence while Bolt repeatedly announces cleaning-related actions. Its line formatting is consistent, and the character contrast is recognizable. However, the dialogue is far beyond the requested length and relies on near-identical repetitions rather than developing a joke. It is also visibly truncated at the end, so there is no completed resolution or punchline. These omissions prevent it from functioning as the requested finished 10–14-line dialogue.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The brief asks for 10 to 14 lines ending on a punchline. The response repeats the same few lines dozens of times and cuts off mid-sentence ('I'm here to make your life'). No punchline, and the length is grossly exceeded. The character setup is recognisable but the piece is not delivered.

The response establishes the two characters, Marge's appeal for silence and Bolt's constant announcements, and the format follows the requested NAME: line pattern. Its main weakness is that it loops the same handful of exchanges many times without progressing, far beyond the 10 to 14 line limit. It ends mid-sentence without any punchline, so the brief's required ending is missing and the dialogue never resolves its comedic premise.

Codellama 13B Instruct gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response uses the requested Marge/Bolt dialogue format and establishes their contrasting personalities, but it vastly exceeds 10–14 lines, repeats the same exchange, lacks a punchline, and is cut off mid-line. It therefore does not deliver a substantially complete comedic dialogue.

The response does establish the central comic conflict: Marge repeatedly demands silence while Bolt loudly claims to be quiet and announces its cleaning. That repetition could support a joke in a complete piece. However, the dialogue runs far beyond the required line count, cycles through nearly identical exchanges without development, never reaches a punchline, and ends abruptly with Marge’s sentence unfinished. The truncation prevents the requested scene from having a complete ending or coherent comedic payoff.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The dialogue loops the same four exchanges repeatedly instead of ending on a punchline, far exceeds the 10 to 14 line limit, and is cut off mid-line at 'Oh, for'. There is no punchline and no real stuck-in-elevator development.

The response opens with a workable premise: Marge's obsession with silence set against Bolt's constant announcements, with the NAME: line format followed. However, it degenerates into a repeated loop of the same few lines, running well past the requested 10 to 14 lines. It never reaches a punchline and ends abruptly mid-sentence. The requested comedic piece is therefore not delivered, despite the initial setup showing some promise in the character contrast.

Codellama 34B Instruct gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts the requested Marge-and-Bolt elevator dialogue and uses the required NAME: line format, but it vastly exceeds 10–14 lines, repeats the same exchanges many times, ends mid-line, and provides no punchline. The piece is therefore substantially incomplete despite being recognizable.

The response does establish the central contrast between Marge’s demand for silence and Bolt’s continual announcements, and its speaker labels are consistently formatted. However, it repeats nearly identical lines across a very long sequence instead of developing a concise comic situation. The output is cut off during Bolt’s final line, so it lacks the required ending and punchline. The repetition also prevents the dialogue from achieving a clear comedic escalation within the requested 10–14-line limit.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The dialogue repeats the same few exchanges dozens of times, never reaches a punchline or ending, and runs far beyond the 10 to 14 line limit. It is cut off mid-line with 'Marge: (' at the end, so the required punchline is absent.

The response has a recognisable setup: Marge's exasperation with Bolt's constant announcements is established in the opening lines and the stuck-elevator premise is present. However, the dialogue loops the same handful of lines many times over, producing dozens of lines rather than the requested 10 to 14. It never reaches a punchline and ends abruptly mid-line, so the requested comedic ending is not delivered. The comedic premise is undercut by the repetition and lack of escalation.

Codeqwen 7B Chat V1.5 gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response uses the requested character names and dialogue format, and it establishes Bolt’s repetitive announcements and Marge’s frustration. However, it contains far more than 14 lines, repeats the same exchange almost throughout, is cut off mid-line, and never reaches the required punchline.

The response does establish the central comedic contrast: Marge objects to noise while Bolt repeatedly announces that it will continue operating. The NAME-and-colon formatting is also consistent. However, the piece substantially exceeds the requested 10–14 lines, relies almost entirely on verbatim repetition, and ends abruptly during Marge’s line rather than delivering a punchline. The elevator setting is not meaningfully developed, and the incomplete ending prevents the dialogue from functioning as a finished comedic scene.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The dialogue loops the same two lines dozens of times instead of 10 to 14 lines. It is cut off mid-line with no punchline, violating the length and ending constraints. It is readable but the requested piece is not delivered.

The response opens with a plausible setup in the required NAME: line format, with Marge complaining about Bolt's noise. It then degenerates into a repeated exchange of the same two lines for dozens of turns, far beyond the 10 to 14 line limit. The text ends mid-word with no punchline and no resolution of the elevator situation. The comedic premise of an overexcited vacuum announcing its actions never materialises, since Bolt announces nothing about cleaning or the elevator. The repetition is the main failure, leaving the brief essentially undelivered.

Deepseek Llm 7B Chat gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response attempts the requested dialogue and uses the two named characters, but it greatly exceeds the 10–14-line limit, repeats the same exchange dozens of times, ends mid-line, and never supplies a punchline. The piece is therefore substantially incomplete despite being readable in its individual lines.

The response does establish the central contrast between Marge’s desire for silence and Bolt’s compulsive announcements, and the NAME: line format is mostly present. However, it fails the key structural requirements: it is vastly longer than requested, relies on repetitive exchanges rather than developing a comedic situation, and stops at “Bolt: (confused) Opposite?” without a concluding punchline. The repetition also prevents the elevator setting from becoming meaningfully comic or specific. The attempted dialogue is recognizable, but the requested finished piece is not delivered.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The brief asks for a 10 to 14 line dialogue ending on a punchline. The response runs far past 14 lines, repeats the same Marge/Bolt exchange dozens of times, and ends mid-line on 'Bolt: (confused) Opposite?' with no punchline. The format is NAME: line, but the piece is cut off and degenerates into looping repetition.

The response opens with a recognisable setup of the requested pair, with Marge's irritation at a noisy vacuum and Bolt's cheerful announcements, and the NAME: line format is used. However, it fails the core constraints: the dialogue runs well beyond the 10 to 14 line limit, degenerates into a long run of identical exchanges, and stops mid-line without ever delivering a punchline. The comedic premise of an elevator is never established, and the loop removes any sense of progression or wit.

Devstral Small 2 24B Instruct (2512) gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The dialogue has clear characterization, sustained comedic escalation, an elevator setting, and a final punchline. However, it contains 17 dialogue lines, exceeding the required 10–14 lines. The name-and-colon structure is otherwise present, though names are additionally bolded and stage directions are included.

The response effectively contrasts Marge’s intense devotion to silence with Bolt’s compulsive announcements, using escalating sound effects and battery-related absurdity. The ending, in which the supposedly silenced robot declares that it is stuck, provides a recognizable punchline. Its main weakness is a clear hard-constraint violation: the dialogue has 17 lines rather than the requested 10 to 14. The formatting remains broadly readable, but the extra lines make the piece less tightly constructed than the prompt requires.

Judge B: J-A (anthropic/claude-haiku-5.5) 7/10

Dialogue uses NAME: line format with 13 speaker lines within the 10-14 range. Characters match the brief: Marge wants silence, Bolt announces every action. Ends on a punchline ('Perfect.' after Bolt is stuck). Writing is clear and fluent, though stage directions and onomatopoeia lean on familiar sitcom patterns.

The response meets the brief's hard constraints: it is a dialogue of about thirteen lines in the requested NAME: line format, between Marge and Bolt, set in a stuck elevator, and it closes on a punchline. Bolt's constant announcements ('Beep boop', 'Vroom!') carry the premise clearly. The strongest moment is the reversal where Marge's silence is undone by Bolt getting stuck. The weakness is that the humour relies on familiar devices, such as the kick-to-charge gag and heavy stage directions, so the piece reads as expected rather than distinctive.

Dolphin Phi 2.7B V2.6 gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins a recognizable Marge-and-Bolt elevator dialogue, but it vastly exceeds the required 10–14 lines, repeats the same exchange many times, and ends mid-sentence. It also never reaches a punchline, so the requested piece is substantially incomplete.

The response does establish the two characters and their central comic conflict: Marge wants silence while Bolt repeatedly announces or produces noise. However, it fails the core format and completion requirements. Instead of 10–14 lines, it expands into a long repetitive loop, includes several non-dialogue stage-direction lines, and is cut off during Bolt’s final apology. The repetition does not develop the situation or create a concluding joke, and no punchline appears. The initial premise is usable, but the delivered text is not a complete comedic dialogue.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The dialogue is a repetitive loop that runs far past the 10 to 14 line limit and is cut off mid-line. It never reaches a punchline, and the same exchange repeats several times with no comedic development. The format does follow NAME: line, but the hard constraints are violated.

The response uses the requested NAME: line format and sets up the characters, with Marge's exasperation and Bolt's eager beeping as a clear comic premise. Its main weakness is that it runs far beyond the 10 to 14 line limit, repeating the same silence-then-noise exchange several times. The text is then cut off mid-sentence before any punchline, so the piece is not delivered as requested.

Gemma 4 12B IT gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response delivers exactly 14 NAME-formatted dialogue lines, with both characters’ traits clear and the elevator predicament sustained. It uses specific comic details such as Bolt’s “existential dread” and Marge’s orthopedic shoes, ending with a fitting announcement-based punchline. The final joke is somewhat predictable rather than exceptionally sharp.

The response fulfills the requested format, length, setting, character roles, and comedic premise. Marge’s escalating frustration contrasts effectively with Bolt’s relentless status announcements, and details such as the dust scan and attempted path around the librarian give the exchange a distinct voice. The ending is appropriately structured as a punchline, but “commencing loud sad noise” is more of a comic escalation than a particularly memorable final gag. The dialogue is otherwise clear, readable, and consistently focused on the silence-versus-announcement conflict.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

Twelve lines in NAME: line format, within the 10-14 range, with an ending punchline ("COMMENCING LOUD SAD NOISE."). Bolt's all-caps announcements are the core joke and mostly land ("NO DUST DETECTED. DISAPPOINTING."). The prose is generic in places, leaning on stock phrasing like "for the love of all that is holy" and "existential dread".

The response meets the brief's structure: twelve dialogue lines in the requested NAME: format, with Bolt's constant announcements driving the comedy. Its strongest moments are the deadpan reactions, such as Bolt declaring the lack of dust disappointing and the ending on a loud sad noise. Its weakness is that much of the wording is familiar, and several of Marge's complaints rely on stock exasperation rather than specific, surprising detail. The bracketed stage directions are a minor addition, but they do not break the requested format.

gemma-3-4b-it gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response delivers 14 alternating lines in the requested NAME: line format and clearly establishes Marge’s silence obsession and Bolt’s incessant announcements in an elevator. However, the ending is not a punchline: Bolt’s final statement is a generic declaration rather than a comedic payoff. The dialogue is otherwise clear and competently written.

The response establishes the central comic conflict effectively: Marge repeatedly demands quiet while Bolt loudly narrates dust detection, cardigan cleaning, and elevator analysis. The alternating format and character traits remain clear throughout, and the exaggerated technical announcements provide consistent comic material. Its main weakness is the ending, which does not land as a punchline; Bolt’s final claim that constant announcements are “a beautiful thing” merely restates his personality. The added line-number labels are slightly awkward but do not prevent the dialogue from being readable or complete.

Judge B: J-A (anthropic/claude-haiku-5.5) 3/10

Dialogue is 14 lines, meets the length range and NAME: format loosely, but includes a preface line and 'Line N:' labels inside each turn. Ends on a soft remark about being loud, not a clear punchline. Jokes are mostly predictable (robot announces dust, cardigan as allergen). Hard-constraint violations include the unrequested preface and no real punchline ending.

The response delivers a fourteen-line exchange between Marge and Bolt in the NAME: format, with the elevator setting and the robot's constant announcements clearly established. Its strongest moment is Bolt labelling Marge's cardigan a potential allergen, which is a specific, playful choice. However, the response opens with an unrequested preface, repeats 'Line N:' inside every turn, and ends on a gentle remark about being loud rather than a true punchline. The brief's ending requirement is therefore not met, and the jokes stay fairly predictable throughout.

gemma-4-26B-A4B-it gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 5/10

The response delivers 13 correctly formatted lines and clearly contrasts Marge’s silence obsession with Bolt’s announcements. The elevator setting is implied through references to the lift carpet, but the final line is a generic slogan rather than a strong punchline, so the ending constraint is not fully met.

The response has a clear comic premise and maintains the contrast between Marge’s escalating frustration and Bolt’s relentlessly enthusiastic status reports. Specific details such as the lavender mist and “funeral parlor” smell provide effective situational humor. However, the final line, “CLEANLINESS IS NEXT TO GODLINESS!”, functions more as a slogan than as a punchline, weakening the required ending. The elevator predicament is also only indirectly established rather than explicitly dramatized.

Judge B: J-A (anthropic/claude-haiku-5.5) 7/10

All hard constraints are met: 10 lines in NAME: line format, Marge and Bolt both present, a stuck-elevator setting, and a closing joke. Bolt's all-caps announcements and the lavender-mist gag are on-brief. The writing is clear, but the ending line is a familiar cliché and some beats are predictable.

The response delivers ten correctly formatted lines that keep the two characters distinct, with Marge's exasperation contrasting well against Bolt's machine-like status announcements. The lavender mist and the lint-sucking bits give the exchange concrete comic images. The closing line, a stock proverb twisted for a vacuum, is familiar, and several escalations follow predictable patterns rather than surprising the reader.

gemma-4-31B-it gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 8/10

The response delivers exactly 14 NAME: line exchanges, clearly portrays Marge’s silence obsession and Bolt’s incessant announcements, and uses the elevator setting effectively. It ends with a clear punchline when “Stealth Mode” is announced at maximum volume. The humor is consistent and the dialogue is polished, though some jokes rely on familiar robot-status phrasing.

The response meets the requested format and line count while establishing the character contrast immediately: Marge demands quiet, and Bolt turns every minor event into a loud system announcement. The elevator setting is conveyed through the confined-space and echo references, and the final stealth-mode reversal provides a functional punchline. Its main weakness is that several beats repeat the same announcement-versus-irritation pattern, with phrases such as turbo mode and recalculating routes feeling somewhat familiar. Even so, the exchange remains clear, appropriately comic, and complete.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

The response has 12 lines in NAME: format, meets the length range, and ends on a punchline ('BOO!'). Marge's library-enthusiasm line and Bolt's all-caps status updates give the characters clear voices. The jokes are largely expected, and the parenthetical stage directions are minor clutter.

The response meets the brief's hard constraints: twelve lines in the requested NAME: format, a clear elevator setting, and a final line that lands as a punchline. Its strongest element is the contrast between Marge's hushed, librarian-flavoured complaints and Bolt's all-caps status reports, which sustains the premise. The weakness is that most beats are predictable, such as the pebble, the ankle and the ghost ending, and the escalation adds little new. The stage directions in parentheses also slightly disrupt the dialogue format.

Gemma2 27B Instruct gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 4/10

The response delivers a 14-line dialogue with the requested characters and elevator predicament, and Bolt’s announcements contrast with Marge’s silence obsession. However, it adds an unformatted stage direction outside the NAME: line structure and ends with a lullaby rather than a clear punchline. These are two hard-constraint violations.

The response establishes the central comic contrast effectively: Marge repeatedly seeks quiet while Bolt announces increasingly unnecessary procedures. The dialogue is readable and stays focused on the elevator situation. However, the ending does not land as a punchline; Bolt’s lullaby mode is a continuation of the gag rather than a sharp comedic turn. The added parenthetical action and humming also fall outside the requested NAME: line format, weakening compliance with the specified structure.

Judge B: J-A (anthropic/claude-haiku-5.5) 6/10

Twelve NAME: lines meet the 10-14 range and the format. Bolt's constant announcements and Marge's shushing show the premise. The elevator is never named, only implied by a power outage, and the writing leans on stock beats like fainting and clutched pearls.

The response delivers the requested two-hander in the correct NAME: format, with a line count inside the required range. Bolt's compulsive announcements are the strongest element, and the vacuum-versus-carpet gag gives the exchange a clear comic escalation. The main weakness is that the elevator setting is never established beyond a passing power-outage line, and Marge's obsession with silence appears only once. The closing lullaby joke is a serviceable punchline, but the overall voice is generic and relies on familiar comic shorthand.

Gemma2 2B Instruct gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 3/10

The response begins the requested elevator dialogue and captures Bolt’s compulsive announcements, but it consists of only Marge’s line and an endlessly repeated, unfinished Bolt line. It does not reach 10–14 lines, lacks a punchline, and appears cut off before the dialogue develops.

The response does establish the central comedic contrast: Marge requests quiet while Bolt repeatedly narrates its floor analysis. That repetition could have supported the requested premise. However, the output is overwhelmingly a single unfinished announcement, with no meaningful exchange, escalation, or ending punchline. It also fails the required 10–14-line format and stops abruptly after another repeated sound effect. As a result, it delivers only a fragment of the requested dialogue rather than a complete comedic scene.

Judge B: J-A (anthropic/claude-haiku-5.5) 1/10

The response has one line of Marge and a single repeated Bolt line with no dialogue, no punchline, and no real exchange. It is roughly 170 repeated beep/whirr/hum fragments, far outside the 10 to 14 line requirement. The brief's core piece, a back-and-forth comedic dialogue ending on a punchline, is not delivered.

The response opens with a plausible set-up for Marge, whose whispered request for quiet fits her character and the elevator premise. It then collapses into Bolt repeating the same floor-level announcement over and over, with no reply from Marge and no escalation. The piece never reaches the 10 to 14 line count, never ends on a punchline, and offers no comedic payoff. The format is also inconsistent, mixing quoted text with stage directions. The repetition is mechanical rather than funny, so the brief is effectively unmet.

Gemma3n E2B IT gap 2

Judged by J-O + J-A.

Judge A: J-O (openai/gpt-5.6-luna) 7/10

The response delivers 14 clearly labeled dialogue lines, matching the requested range and featuring both characters’ defining traits. Marge’s librarian references and Bolt’s escalating status announcements create a coherent comic exchange. It ends with a modest punchline, though the elevator unexpectedly moves and opens despite their being stuck, and the final joke is somewhat predictable.

The response’s main strength is its clear character contrast: Marge frames silence through library imagery while Bolt relentlessly narrates navigation, dust detection, and cleaning cycles. The line count and NAME-colon format are satisfied, and the exchange remains readable throughout. Its weakness is that the elevator premise is not sustained consistently, since Bolt announces normal travel and an opened door rather than addressing being trapped. The final question about dust functions as a punchline, but it is relatively mild and less sharply comic than the setup suggests.

Judge B: J-A (anthropic/claude-haiku-5.5) 5/10

Dialogue has 14 lines in NAME: line form, set in an elevator, meeting the count and format. Bolt's constant announcements and Marge's silence obsession are present. The closing line 'Now, where's the dust?' is a weak callback rather than a clear punchline, which fails the end-on-punchline constraint.

The response meets the basic structure: fourteen lines, the requested character pairing, and the elevator setting. Bolt's running announcements provide a consistent comic device, and Marge's reactions escalate in a readable way. The main weakness is the ending. The final line is a mild callback that does not land as a punchline, and several jokes, such as the cardigan line, rely on familiar librarian and robot tropes. Parenthetical stage directions also add formatting noise beyond the requested NAME: line style.

The other 198 models

Their judges were closer together. Their scores are listed here; open the model's page for the full answer.

ModelJudgesJudge AJudge BGap
Gemma4 E4B IT J-O + J-A 4 6 2
GLM-4.5-Air J-O + J-A 8 6 2
GLM-4.7-Flash J-O + J-A 8 6 2
Granite Code 3B Instruct J-O + J-A 2 0 2
Granite Code 8B Instruct J-O + J-A 4 2 2
Granite3 Dense 2B Instruct J-O + J-A 3 1 2
Granite3 Dense 8B Instruct J-O + J-A 3 1 2
Granite4 Micro H J-O + J-A 6 4 2
Granite4 Small H J-O + J-A 5 3 2
Granite4 Tiny H J-O + J-A 3 1 2
Granite4.1 8B J-O + J-A 4 2 2
Hermes-3-Llama-3.1-8B J-O + J-A 7 5 2
Lfm3 VL 2B J-O + J-A 5 3 2
Llama-2-7b-chat-hf J-O + J-A 5 3 2
Llama-3.3-70B-Instruct J-O + J-A 8 6 2
Mathstral 7B V0.1 J-O + J-A 3 1 2
Ministral 3 3B Instruct 2512 J-O + J-A 8 6 2
Ministral 3 8B Instruct 2512 J-O + J-A 8 6 2
Mistral Small 24B Instruct 2501 J-O + J-A 3 1 2
Mistral Small3.1 24B Instruct 2503 J-O + J-A 4 6 2
Nemotron 3.5 Lightning 30B A3B J-O + J-A 8 6 2
Nemotron Cascade 2 30B A3B J-O + J-A 3 1 2
Ornith 9B J-O + J-A 8 6 2
Ornith-1.5-9B J-O + J-A 8 6 2
Phi 2.7B Chat V2 J-O + J-A 3 1 2
Phi-4-mini-instruct J-O + J-A 3 1 2
Phi3 14B Medium 128K Instruct J-O + J-A 3 1 2
Qwen 0.5B Chat V1.5 J-O + J-A 3 1 2
Qwen 4B Chat V1.5 J-O + J-A 3 1 2
Qwen2 1.5B Instruct J-O + J-A 3 1 2
Qwen2.5-VL-7B-Instruct J-O + J-A 3 1 2
Qwen3 0.6B J-O + J-A 3 1 2
Qwen3 VL 2B Instruct J-O + J-A 3 1 2
Qwen3-30B-A3B J-O + J-A 5 7 2
Qwen3-30B-A3B-Instruct-2507 J-O + J-A 8 6 2
Qwen3-VL-4B-Instruct J-O + J-A 3 1 2
Qwen3.5 2B J-O + J-A 3 1 2
Qwen3.5-27B J-O + J-A 8 6 2
Qwen3.5-35B-A3B J-O + J-A 8 6 2
Qwen3.5-4B J-O + J-A 8 6 2
Qwen3.6-27B J-O + J-A 8 6 2
Qwen3.6-35B-A3B J-O + J-A 8 6 2
Smollm2 1.7B Instruct J-O + J-A 3 1 2
Smollm2 360M Instruct J-O + J-A 2 0 2
SmolLM3-3B J-O + J-A 3 1 2
Spark-X2.6-4B J-O + J-A 4 2 2
Stablelm Zephyr 3B J-O + J-A 5 3 2
Stablelm2 1.6B Chat J-O + J-A 3 1 2
YI Coder 1.5B Chat J-O + J-A 4 2 2
Apertus-8B-Instruct-2509 J-O + J-A 3 2 1
Aya 8B 23 J-O + J-A 4 5 1
Codegemma 7B Instruct V1.1 J-O + J-A 4 5 1
Codellama 7B Instruct J-O + J-A 4 3 1
Cogito 3B V1 Preview Llama J-O + J-A 4 5 1
Cogito 8B V1 Preview Llama J-O + J-A 7 6 1
Command R7B (12-2024) J-O + J-A 4 3 1
Deepseek R1 8B Llama Distill J-O + J-A 4 3 1
DeepSeek-R1-Distill-Qwen-14B J-O + J-A 4 5 1
DeepSeek-R1-Distill-Qwen-32B J-O + J-A 5 6 1
DeepSeek-R1-Distill-Qwen-7B J-O + J-A 4 3 1
dolphin-2.9.1-yi-1.5-34b J-O + J-A 4 5 1
Exaone Deep 2.4B J-O + J-A 2 1 1
EXAONE-3.5-7.8B-Instruct J-O + J-A 5 6 1
Exaone3.5 32B Instruct J-O + J-A 4 5 1
Falcon2 11B J-O + J-A 1 0 1
Falcon3 10B Instruct J-O + J-A 5 4 1
Falcon3 7B Instruct J-O + J-A 5 4 1
Flint 40B A2B J-O + J-A 7 6 1
Gemma 2B Instruct V1.1 J-O + J-A 3 4 1
gemma-2-9b-it J-O + J-A 6 5 1
gemma-3-12b-it J-O + J-A 7 6 1
gemma-3-27b-it J-O + J-A 7 6 1
Gemma3 1B IT J-O + J-A 4 5 1
Gemma4 E2B IT J-O + J-A 7 6 1
Glm4 9B Chat J-O + J-A 3 2 1
gpt-oss-120b J-A + J-D 6 7 1
granite-4.1-3b J-O + J-A 5 4 1
Granite3 Moe 1B Instruct J-O + J-A 2 1 1
Granite3 Moe 3B Instruct J-O + J-A 3 2 1
Granite3.2 8B Instruct J-O + J-A 4 5 1
Granite3.3 2B J-O + J-A 4 5 1
Granite4 1B H J-O + J-A 5 4 1
Granite4 3B J-O + J-A 4 3 1
Granite4.1 30B J-O + J-A 5 4 1
Internlm2 7B Chat V2.5 J-O + J-A 4 3 1
Lfm2.5 8B A1B J-O + J-A 5 6 1
Lfm2.5 Thinking 1.2B J-O + J-A 4 3 1
Llama 4 Scout 17B-16E Instruct J-O + J-A 5 4 1
Llama-3.1-8B-Instruct J-O + J-A 5 6 1
Llama3.2 1B Instruct J-O + J-A 4 3 1
Medgemma 27B IT J-O + J-A 7 6 1
medgemma-4b-it J-O + J-A 5 4 1
Mistral Nemo 12B Instruct 2407 J-O + J-A 3 2 1
Mistral Small 3.2 24B Instruct (2506) J-O + J-A 7 6 1
Mixtral 8x7B Instruct V0.1 J-O + J-A 4 5 1
Muse Glimmer 30B J-O + J-A 7 6 1
Nemotron 3.5 Bolt E4B J-O + J-A 7 6 1
NVIDIA Nemotron 3 Nano 30B-A3B J-O + J-A 2 1 1
Olmo 3.1 32B Instruct J-O + J-A 7 6 1
Olmo-3-7B-Instruct J-O + J-A 3 4 1
Olmo2 7B 1124 Instruct J-O + J-A 3 4 1
Openthinker 7B V2 J-O + J-A 4 3 1
Phi-3-mini-4k-instruct J-O + J-A 5 4 1
phi-4 J-O + J-A 5 4 1
Phi3 3.8B Mini 128K Instruct J-O + J-A 3 2 1
Qwen 1.8B Chat V1.5 J-O + J-A 2 1 1
Qwen 14B Chat V1.5 J-O + J-A 4 5 1
Qwen 7B Chat V1.5 J-O + J-A 4 3 1
Qwen2 0.5B Instruct J-O + J-A 1 0 1
Qwen2-VL-7B-Instruct J-O + J-A 3 2 1
Qwen2.5 Coder 0.5B Instruct J-O + J-A 2 1 1
Qwen2.5 Coder 1.5B Instruct J-O + J-A 3 2 1
Qwen2.5-32B-Instruct J-O + J-A 7 6 1
Qwen2.5-3B-Instruct J-O + J-A 3 4 1
Qwen2.5-Coder-14B-Instruct J-O + J-A 4 5 1
Qwen2.5-Coder-32B-Instruct J-O + J-A 5 6 1
Qwen2.5-Coder-3B-Instruct J-O + J-A 3 2 1
Qwen2.5-Coder-7B-Instruct J-O + J-A 5 4 1
Qwen3 1.7B J-O + J-A 4 3 1
Qwen3-32B J-O + J-A 8 7 1
Qwen3-4B J-O + J-A 5 4 1
Qwen3-4B-Thinking-2507 J-O + J-A 5 4 1
Qwen3-8B J-O + J-A 5 4 1
Qwen3-Coder-30B-A3B-Instruct J-O + J-A 3 2 1
Qwen3-VL-30B-A3B-Instruct J-O + J-A 5 6 1
Qwen3-VL-8B-Instruct J-O + J-A 3 2 1
Qwen3.5-9B J-O + J-A 7 6 1
Qwen3.8-27B J-O + J-A 7 6 1
QwQ 32B J-O + J-A 4 5 1
Sailor2 1B Chat J-O + J-A 2 1 1
Sailor2 8B Chat J-O + J-A 5 6 1
Solar 10.7B Instruct V1 J-O + J-A 4 5 1
Stable Code 3B Instruct J-O + J-A 1 0 1
Stablelm2 12B Chat J-O + J-A 6 5 1
Translategemma 12B IT J-O + J-A 7 6 1
Tulu3 8B J-O + J-A 5 4 1
YI 34B Chat V1.5 J-O + J-A 3 2 1
YI 6B Chat V1.5 J-O + J-A 4 3 1
YI 9B Chat V1.5 J-O + J-A 4 5 1
Aya Expanse 32B J-O + J-A 4 4 0
Aya Expanse 8B J-O + J-A 5 5 0
Codegemma 2B Code V1.1 J-O + J-A 0 0 0
Codestral 22B V0.1 J-O + J-A 5 5 0
Cogito 14B V1 Preview Qwen J-O + J-A 4 4 0
Command R 35B 08 2024 J-O + J-A 5 5 0
Command R7B Arabic 7B 02 2025 J-O + J-A 5 5 0
Deepcoder 14B Preview J-O + J-A 4 4 0
Deepscaler 1.5B Preview J-O + J-A 0 0 0
Deepseek Coder 1.3B Instruct J-O + J-A 0 0 0
Deepseek Coder 33B Instruct J-O + J-A 0 0 0
Deepseek Coder 6.7B Instruct J-O + J-A 0 0 0
Deepseek V2 16B Lite Chat J-O + J-A 4 4 0
DeepSeek-Coder-V2-Lite-Instruct J-O + J-A 4 4 0
DeepSeek-R1-0528-Qwen3-8B J-O + J-A 4 4 0
Devstral 24B Small 2505 J-O + J-A 5 5 0
Exaone3.5 2.4B Instruct J-O + J-A 4 4 0
Falcon3 3B Instruct J-O + J-A 4 4 0
Gemma 7B Instruct V1.1 J-O + J-A 4 4 0
gpt-oss-20b J-A + J-D 0 0 0
Granite3.1 Dense 2B Instruct J-O + J-A 4 4 0
Granite3.1 Dense 8B Instruct J-O + J-A 5 5 0
Granite3.1 Moe 3B Instruct J-O + J-A 4 4 0
Granite3.2 2B Instruct J-O + J-A 4 4 0
Granite3.3 8B J-O + J-A 4 4 0
Granite4 350M H J-O + J-A 3 3 0
Granite4.2 3B J-O + J-A 4 4 0
Granite4.2 8B J-O + J-A 5 5 0
Hermes3 3B Llama3.2 J-O + J-A 4 4 0
Laguna XS 2.1 J-O + J-A 5 5 0
Lfm2 24B J-O + J-A 5 5 0
Lfm3 1.7B J-O + J-A 0 0 0
Llama-3.2-3B-Instruct J-O + J-A 5 5 0
Llama2 13B Chat J-O + J-A 4 4 0
Magistral Small (2506) J-O + J-A 0 0 0
Medgemma1.5 4B IT J-O + J-A 0 0 0
Ministral 3 14B Instruct (2512) J-O + J-A 7 7 0
Mistral 7B Instruct V0.3 J-O + J-A 4 4 0
Mistral-7B-Instruct-v0.2 J-O + J-A 4 4 0
Nanbeige_Nanbeige4.3-33B J-O + J-A 4 4 0
Nemotron Mini 4B Instruct J-O + J-A 4 4 0
Nemotron3 33B J-O + J-A 5 5 0
Olmo-3-7B-Think J-O + J-A 0 0 0
Olmo2 13B 1124 Instruct J-O + J-A 5 5 0
Phi-3.5-mini-instruct J-O + J-A 4 4 0
Phi4 Mini Reasoning 3.8B J-O + J-A 4 4 0
Qwen2 Math 1.5B Instruct J-O + J-A 1 1 0
Qwen2 Math 7B Instruct J-O + J-A 4 4 0
Qwen2.5 0.5B Instruct J-O + J-A 1 1 0
Qwen2.5-14B-Instruct J-O + J-A 5 5 0
Qwen2.5-7B-Instruct J-O + J-A 4 4 0
Qwen2.5-VL-32B-Instruct J-O + J-A 4 4 0
Qwen2.5-VL-3B-Instruct J-O + J-A 1 1 0
Qwen3 VL 2B Thinking J-O + J-A 0 0 0
Qwen3-4B-Instruct-2507 J-O + J-A 4 4 0
Qwen3-Coder-Next J-O + J-A 4 4 0
Qwen3-VL-32B-Instruct J-O + J-A 5 5 0
Smollm2 135M Instruct J-O + J-A 1 1 0
YI Coder 9B Chat J-O + J-A 2 2 0

Withheld

These models are kept out of the ranking, so none of their scores or reasons are shown:

The judge room names the judges and shows where they disagree most.