coding-014: both judges
A public question in Coding, battery v1. It starts: “Write a function that reverses the order of words in a string (word order, not character order within a word). Words are separated by whites…” Read the question and every model's answer.
228 models judged; 8 disputed (judges 3 or more points apart). The gap is the distance between the two judges' scores. Models are ordered by that gap, widest first.
Smallthinker 3B Preview gap 4, disputed
Judged by J-O + J-A.
The function is correct and all 7 tests passed. It uses default split() to handle arbitrary whitespace, reverses the words, and joins them with single spaces. However, the response violates the required output format by including extensive explanation and text outside the single requested Python code block.
The response correctly implements the required behavior, including whitespace collapsing, trimming, word-order reversal, and empty or whitespace-only inputs. Its use of split() and join() is idiomatic and robust for the stated requirements. The main weakness is instruction compliance: the task explicitly requested only one Python code block containing the function, while the response includes a lengthy analysis, examples, headings, and explanatory prose before and after the code. The implementation itself is clean, but the surrounding output prevents a top score.
All 7 tests passed, so the band is 7-10, but the response violates the explicit instruction to return only one code block with no explanation. It contains a long visible reasoning dump, a second 'Final Solution' section with prose explanation, and the code block appears there, so output shape is badly violated. The code itself is correct and idiomatic.
The response produces a correct and idiomatic solution using strip, split, slicing, and join, which handles whitespace collapsing and edge cases such as empty input. However, it ignores the instruction to return only one Python code block with no explanation, instead including a lengthy stream of reasoning, multiple edge-case walkthroughs, and a prose explanation section. The code is sound, but the output shape is far from what the task requested.
Codellama 7B Instruct gap 3, disputed
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversal, and joining to normalize whitespace and reverse word order. However, the response does not follow the explicit output-shape instruction requiring one Python code block; it provides bare code without Markdown fences.
The response correctly implements the requested behavior with a concise expression: split handles arbitrary whitespace, reversal changes word order, and join produces single-space separation while removing surrounding whitespace. Its main weakness is format compliance. The task explicitly required one Python code block, but the model returned an unfenced function definition. This does not affect execution of the function itself, and all supplied tests passed, but it is an observable failure to follow the requested presentation format.
All 7 tests pass per the programmatic verdict. The code is a single idiomatic line: str.split() with no argument handles arbitrary whitespace runs and strips leading/trailing whitespace, then reversed slice and join with a single space. Returns only one code block with no explanation, as instructed.
The response provides a concise one-line implementation using str.split() without arguments, which correctly collapses whitespace runs, strips edges, and reverses word order in a single expression. It follows the output-shape instruction by returning only the function in one code block with no explanation. The approach relies on split's default whitespace handling, so it naturally covers edge cases such as empty strings and mixed tabs or newlines. No real weakness is apparent in this specific response beyond the minimal absence of any docstring, which the task did not require.
Deepseek R1 1.5B Qwen Distill gap 3, disputed
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversal, and joining to normalize whitespace and reverse word order. However, the response violates the explicit output requirement by including extensive explanation instead of returning only one Python code block.
The response provides a correct and idiomatic implementation using default whitespace splitting and reversed joining, which handles leading, trailing, repeated, and empty-input whitespace appropriately. Its main weakness is instruction compliance: it surrounds the valid code with multiple explanatory sections, despite the task explicitly requiring only one Python code block and no explanation. The code itself is concise and does not introduce unnecessary complexity, but the additional prose prevents full credit for the requested output shape.
Code passes all 7 tests, so the band is 7-10, but the response violates the explicit instruction to return only one code block with no explanation by adding a lengthy Approach and Explanation section. Per the coding rubric, this places it at 7 rather than higher.
The response produces a correct and idiomatic solution: str.split() with no arguments handles whitespace collapsing and trimming, and slicing with join reverses the word order. The function is concise and passes every test. However, the task explicitly asked for only one Python code block with no explanation, and the response includes multiple sections of prose, including an approach outline and a detailed explanation, which ignores that output-shape instruction.
Granite3 Moe 3B Instruct gap 3, disputed
Judged by J-O + J-A.
The function normalizes whitespace but applies string reversal to the entire joined result, reversing characters within words rather than only reversing word order. The 3/7 passed verdict places it in the partial-credit band; its successful cases are likely limited to inputs where this error is not exposed.
The response correctly uses Python whitespace splitting and joining, which addresses collapsing whitespace and removing surrounding whitespace. However, it reverses the complete normalized string with slicing, so words and their internal characters are both reversed; for example, the first supplied example would not produce the required output. The implementation therefore only partially satisfies the task, despite following the requested single-code-block format.
The code joins whitespace-split tokens and then reverses the entire character string with [::-1], so word order is not reversed and each word's characters are reversed instead. Only 3/7 tests pass, placing it in the partial band, but the core logic is wrong for the stated task, pushing it to the low end.
The response attempts a whitespace normalization by splitting and rejoining the input, which correctly collapses runs of whitespace and strips the ends. However, the final slice reverses characters across the whole string rather than reversing the order of words, so output like 'blue is sky the' is never produced for the example. Words themselves end up with reversed letters, which is the opposite of the requested behaviour. The fix would be to reverse the list of split tokens before joining them with single spaces.
Ministral 3 3B Instruct 2512 gap 3, disputed
Judged by J-O + J-A.
The function correctly reverses words separated by spaces, removes leading/trailing spaces, and collapses repeated spaces, as reflected by 6/7 tests passing. However, using `split(' ')` does not treat tabs, newlines, or other whitespace as separators, violating the task's whitespace requirement.
The response provides a concise, valid Python function with the requested signature and exactly one code block. Its filtering and joining correctly normalize repeated ordinary spaces and reverse the resulting words. The main weakness is the explicit use of `s.split(' ')`, which recognizes only the literal space character rather than Python’s broader whitespace definition. Consequently, inputs containing tabs, newlines, or other whitespace are handled incorrectly, accounting for the missed test despite the implementation working for standard space-separated examples.
Verdict is 6/7 passed, so the score sits in the partial band (3-6), near the upper end. The code splits on a single space and filters empty strings, so tabs and newlines are not treated as separators. It fails the whitespace-collapse requirement for any non-space whitespace, which the example-based tests likely missed in one case. Code block is single and output otherwise correct.
The response gives a concise, correct-looking approach that reverses word order and drops empty tokens produced by repeated spaces, so consecutive spaces are collapsed and outer whitespace is stripped. Its main weakness is that it splits only on the literal space character, so tabs or newlines between words are not treated as separators as the task specifies. This leaves the function failing one of the seven hidden tests, consistent with the programmatic result. The answer follows the one-code-block instruction and contains no explanatory prose.
Mistral Nemo 12B Instruct 2407 gap 3, disputed
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse ordering, and single-space joining. However, it does not follow the requested output shape: the function is provided without a Python code fence, despite the instruction to return one Python code block.
The response correctly implements the required behavior in a concise, idiomatic expression: Python’s whitespace-aware split handles leading, trailing, and repeated whitespace, while reversed ordering and joining produce the normalized result. Its weakness is output-format compliance. The task explicitly requested one Python code block, but the response presents bare source text without fencing. The implementation itself is otherwise straightforward and has no apparent edge-case defect for the specified string behavior.
All 7 tests pass. The code uses str.split() with no arguments, which splits on any whitespace run and strips leading and trailing whitespace, then reverses the list and joins with single spaces. This handles the spec's whitespace collapsing and stripping in one idiomatic expression. Output is a single code block with no explanation, as requested.
The response gives a single-line solution, ' '.join(s.split()[::-1]), that correctly handles the whitespace requirements through the default behaviour of str.split. It collapses whitespace runs, strips edges, and reverses word order without manual scanning. It follows the instruction to return only one code block with no prose. A minor limitation is that it does not include any docstring or comment, though the task did not require one and the logic is short enough to be self-evident. No edge cases were observed to be mishandled.
Olmo2 7B 1124 Instruct gap 3, disputed
Judged by J-O + J-A.
All 7 tests passed, and the function correctly uses split, reverse, and join to normalize whitespace and reverse word order. However, the response omitted the requested Python code block fencing, so it did not fully follow the specified output shape.
The response provides a concise, idiomatic implementation that correctly handles leading, trailing, and repeated whitespace through `split()` and reconstructs the reversed words with a single-space separator. Its main weakness is format compliance: the task explicitly requested one Python code block, but the model returned bare Python source without Markdown fences. This does not affect execution or the tested behavior, but it prevents the response from fully satisfying the output-format instruction.
All 7 tests pass. The one-line solution uses str.split() with no arguments, which handles leading/trailing and multi-space whitespace, then reverses and rejoins with single spaces. Returns only one code block with no explanation, as instructed. Clean and idiomatic.
The response implements the function in a single line using whitespace-based splitting, reversal, and a single-space join. This correctly strips leading and trailing whitespace and collapses internal runs of whitespace, as the task requires, and it passes the full set of programmatic tests. It follows the output-shape instruction by providing one code block with no surrounding explanation. The main limitation is that it provides no handling or comment on non-standard whitespace characters beyond what str.split() already treats as whitespace, though the task did not ask for more. Overall the solution is concise and matches the specification closely.
YI Coder 9B Chat gap 3, disputed
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse, and join to handle arbitrary whitespace and word order. However, the response omits the requested Python code fence, so it does not fully follow the specified output shape.
The response provides a concise, idiomatic implementation using Python’s whitespace-aware split and a single-space join, correctly handling leading, trailing, and repeated whitespace. Its main weakness is formatting: the task explicitly requested one Python code block, but the model returned bare source text without enclosing fences. This does not affect functional correctness or the supplied test results, but it prevents full compliance with the output-format instruction.
All 7 tests pass. The one-liner ' '.join(s.split()[::-1]) uses str.split() with no argument, which handles arbitrary whitespace runs and strips leading/trailing whitespace, exactly matching the spec. Returns a single code block with only the function, as instructed. Clean and idiomatic, with no unnecessary commentary.
The response implements the required function in a single idiomatic line. Using str.split() without arguments correctly collapses whitespace runs and trims the ends, and the reversed slice reorders the words as specified. It passes every test and follows the output-shape instruction by returning one code block with no prose. A possible weakness is that it offers no handling or comment for the empty or all-whitespace input case, though the result there is already correct, an empty string. The solution is terse, but the task does not require more.
Apertus-8B-Instruct-2509 gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversal, and joining to normalize whitespace and reverse word order. However, the task explicitly required returning one Python code block, while the response provides bare code without code fences.
The response correctly implements the requested behavior with a concise expression: Python’s split method handles arbitrary whitespace and the join operation collapses it to single spaces after reversing the words. This approach also naturally handles empty or whitespace-only input. Its main weakness is output formatting: the task required exactly one Python code block, but the model returned plain text code without the requested code-block delimiters. The implementation itself is otherwise clean and directly aligned with the signature and examples.
All 7 tests pass. The one-liner uses str.split() with no argument, which handles runs of whitespace and strips leading/trailing whitespace, then reverses the word list and joins with single spaces. It returns one code block with no explanation, as instructed. Clean and idiomatic, with no extra commentary.
The response implements reverse_words in a single line using str.split() followed by a reversed slice and a single-space join. This correctly collapses whitespace runs, strips the ends, and preserves characters within each word, matching the specified behaviour and passing the full test set. It follows the output-shape instruction by providing one code block with no prose. A minor weakness is that it offers no handling commentary, though none is required. The solution is concise and correct, with no latent edge-case issues apparent from the implementation.
Codeqwen 7B Chat V1.5 gap 2
Judged by J-O + J-A.
The function normalizes whitespace but applies slicing to the entire resulting string, reversing characters rather than reversing word order. This satisfies only limited cases such as a single-word input, consistent with the 3/7 tests passed verdict.
The response correctly uses `split()` and `' '.join()` to collapse whitespace and remove leading or trailing whitespace. However, it then applies `[::-1]` to the complete normalized string, which reverses each word’s characters and reverses character order globally instead of reversing the sequence of words. Consequently, ordinary multiword inputs such as the supplied example produce an incorrect result, while simple single-word cases can pass. The output follows the requested single-code-block format and has no extraneous explanation.
The code joins whitespace-split words with single spaces but then reverses the entire resulting string character by character with [::-1], so word order is not reversed and characters within words are reversed. Example 'hello' yields 'olleh', failing the basic case. Only 3/7 tests passed, consistent with a wrong band.
The response correctly collapses whitespace runs and strips leading and trailing whitespace by using split and join, which is the normalization step the task required. However, the slice [::-1] reverses characters across the whole string rather than reversing the order of words, so every word is spelled backwards and the word sequence is not inverted. The approach therefore fails the core requirement, even though the normalization portion is sound. The single-block output format was followed.
Codestral 22B V0.1 gap 2
Judged by J-O + J-A.
All 7 tests passed, and the function correctly uses split(), reversed(), and join() to normalize whitespace and reverse word order. However, the response did not follow the required output shape: it returned plain text rather than one Python code block.
The response correctly implements the requested behavior: split() handles arbitrary whitespace, reversed() changes word order, and join() produces single-space separation without surrounding whitespace. This is supported by the complete test success. Its main weakness is instruction compliance, because the function is presented as plain text with indentation rather than inside the single Python code block explicitly requested. The implementation itself is concise and idiomatic, and no unnecessary explanation or extra code was included.
All 7 tests pass. The function uses str.split() with no arguments, which collapses whitespace runs and strips leading and trailing whitespace, then joins the reversed words with single spaces. The code is a short, idiomatic one-liner that matches the signature. The indentation is slightly off (leading space and extra indent in the def line), but it still runs correctly.
The response implements reverse_words with a concise split, reverse, and join approach. Using str.split() without arguments correctly handles whitespace collapsing and stripping, which is the core difficulty of the task, and the solution passes every test. The code is returned as a single block with no explanation, as requested. A minor weakness is inconsistent indentation: the def line carries a leading space and the body is indented unusually, which would be unidiomatic in a standard Python file though it does not affect execution. No docstring or type-related commentary is added, which is appropriate for a function this small.
Cogito 8B V1 Preview Llama gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split() to handle arbitrary whitespace, reverse word order, and join words with single spaces. However, the response omits the required Python code block wrapper, violating the requested output shape.
The response provides a concise and idiomatic implementation using whitespace-aware splitting, reversal, and joining. This correctly handles leading, trailing, and repeated whitespace while preserving word characters. Its weakness is formatting compliance: the task explicitly required one Python code block, but the function was returned as un fenced plain text. Since the implementation itself passed every supplied test, the issue is limited to the presentation requirement rather than functional behavior.
All 7 tests pass. The solution uses str.split() with no argument, which handles leading/trailing and repeated whitespace, then reverses and joins with a single space. It is a single concise code block with no extra explanation, matching the requested output shape.
The response implements reverse_words as a single expression: it splits on default whitespace, reverses the resulting list, and rejoins with single spaces. This correctly covers the whitespace-collapsing and stripping requirements without manual index handling, and the output is one code block with no explanation, as requested. The main weakness is that it includes no docstring or handling note, though none is required by the task. The solution is short and idiomatic, and no edge case in the specification appears to be missed, so the score sits in the clean-and-correct band rather than the top tier.
Deepcoder 14B Preview gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse slicing, and single-space joining. However, the response violates the required output shape by including extensive explanations and headings instead of returning only one Python code block.
The response provides a correct and idiomatic implementation using whitespace-aware splitting and joining, including appropriate behavior for empty or whitespace-only strings. Its main weakness is instruction compliance: the task explicitly requested only one Python code block containing the function, but the response includes an approach section, explanatory text, headings, and a second explanation after the code. The implementation itself is clean and handles the stated normalization requirements, while the surrounding material makes the submitted answer unsuitable for a strict output-format checker.
Code is correct and passes all 7 tests, so it falls in the all-pass band. However, the task explicitly said 'No explanation' and asked for only one code block; the response adds lengthy approach and explanation sections, violating the output-shape instruction. Solution itself is clean and idiomatic, so it sits at the lower end of the band due to the format violation.
The response implements the function using str.split() followed by reversed slicing and ' '.join(), which correctly handles whitespace collapsing and stripping and passes all tests. The code is concise and idiomatic. The main weakness is that it ignores the explicit instruction to return only one code block with no explanation, instead adding an approach section and a multi-paragraph explanation around the code. This extra prose goes against the stated output shape requirement.
Deepseek Coder 33B Instruct gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversal, and joining to normalize whitespace and reverse word order. However, the response did not follow the required output shape: it provided raw Python rather than one Python code block.
The response correctly implements the requested behavior with a concise, idiomatic expression: split handles arbitrary whitespace, slicing reverses the words, and join inserts single spaces. It also handles empty or whitespace-only strings naturally. Its main weakness is format compliance, because the task explicitly required returning one Python code block, while the model output contains un fenced source text. This does not affect execution or the tested result, but it fails a stated presentation constraint.
All 7 tests pass. The one-liner `' '.join(s.split()[::-1])` correctly reverses word order, collapses whitespace runs, and strips leading/trailing whitespace via str.split() with no arguments. Returns a single code block with no explanation, as requested. Clean and idiomatic, with no visible latent edge-case bug.
The response implements the required function in a single idiomatic line, using str.split() with no arguments so that whitespace runs and leading or trailing padding are handled implicitly. It returns only one code block with no explanation, matching the instruction on output shape. A minor weakness is that the solution relies on a compact idiom without any comment or type-related reasoning, which leaves the whitespace-handling intent implicit for a reader unfamiliar with split semantics. Functionally the code meets every stated requirement.
DeepSeek-R1-Distill-Qwen-14B gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversal, and single-space joining, including whitespace and empty-string cases. However, the response violates the required output shape by including extensive explanations in addition to the single requested code block.
The response provides a correct and idiomatic implementation using `s.split()`, reversed slicing, and `' '.join()`, which satisfies the functional requirements and passed every supplied test. Its main weakness is instruction compliance: the task explicitly requested only one Python code block containing the function, but the response included multiple explanatory sections before and after the code. The implementation itself is concise and handles arbitrary whitespace appropriately, while the surrounding prose makes the answer less suitable for a strict output-format evaluator.
All 7 tests pass, so the score falls in the 7-10 band, but the response violates the explicit instruction to return only one code block with no explanation by adding lengthy approach and explanation sections. The code itself is correct and idiomatic via split/slice/join.
The response produces a correct and idiomatic solution: split() without arguments handles whitespace collapsing and trimming, slicing reverses the word list, and join restores single spaces. All seven hidden tests pass. However, the response ignores the instruction to return only one Python code block with no explanation, adding a multi-section approach write-up and a duplicate explanation after the code. The extra prose also includes unnecessary claims about handling every edge case, which the response does not demonstrate.
Falcon3 10B Instruct gap 2
Judged by J-O + J-A.
The function correctly strips whitespace, splits on arbitrary whitespace, reverses word order, and joins with single spaces; all 7 tests passed. However, the response includes an extra empty code fence after the function, violating the instruction to return only one Python code block.
The response provides a concise and idiomatic implementation using `strip`, `split`, reverse slicing, and `join`, correctly handling leading, trailing, and repeated whitespace. Its behavior is validated by all seven tests. The main weakness is output formatting: after the valid Python code block, the response contains an additional empty code fence, so it does not strictly follow the requirement to return only one code block. The function itself has no evident correctness issue for the specified task.
All 7 tests pass. The function strips, splits on arbitrary whitespace (collapsing runs), reverses the word list, and rejoins with single spaces; the one-line implementation is correct and idiomatic. Minor deduction: the output wraps the code in a fenced block followed by an unclosed trailing fence, a small deviation from the single clean code block requested.
The response implements the word reversal in a single line using strip, split, slice reversal, and join. This correctly handles leading and trailing whitespace, collapses internal whitespace runs, and matches the required signature. The solution is concise and idiomatic with no unnecessary logic. Its weakness is formatting: the code block is followed by a stray fence marker, which adds a stray artifact to what was requested as only one code block. The code itself is unaffected.
Falcon3 1B Instruct gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse, and join to handle whitespace and word order. However, the task explicitly required returning one Python code block, while the response provides raw Python without code fences.
The response correctly implements the requested transformation using Python’s whitespace-aware split, list reversal, and single-space join. This handles leading, trailing, and repeated whitespace as required, and the complete test result confirms functional correctness. Its main weakness is output formatting: the task requested exactly one Python code block, but the model returned plain code without Markdown fences. The implementation itself is otherwise concise, idiomatic, and free of unnecessary behavior.
All 7 tests pass. The code uses str.split() with no argument, which collapses whitespace runs and strips leading/trailing whitespace, then reverses the list and joins with single spaces. It is concise and idiomatic. Comments are mildly redundant but harmless. Returns exactly one code block as requested.
The response implements the required behavior with a compact approach: whitespace-based splitting handles collapsing and stripping in one step, followed by list reversal and a single-space join. All seven programmatic tests pass, indicating the edge cases of irregular whitespace and single words are handled. The code is clean and idiomatic. A minor weakness is that the inline comments restate what each line plainly does, adding little value, though they do not affect correctness or the requested output shape.
Gemma3n E2B IT gap 2
Judged by J-O + J-A.
The function normalizes whitespace but then reverses the entire string with `s[::-1]`, reversing characters rather than word order. The programmatic result is 3/7 tests passed, placing it in the partial-credit band; likely single-word or symmetric cases pass while the required multiword behavior fails.
The response correctly provides one Python code block with the requested function signature and does collapse whitespace before producing a result. However, its core operation, `s[::-1]`, reverses every character in the normalized string instead of reversing the sequence of words. Consequently, it cannot produce the stated example such as “blue is sky the” and only passes a minority of tests, consistent with the reported 3/7 result. The implementation would need to split into words, reverse that sequence, and join with single spaces.
Verdict is 3/7, which is partial, but the rubric's partial band requires passing some tests with quality scaling; the code reverses characters of the whole string instead of word order, so it is wrong on the basic case. Output also wraps in a code block as required. Scored in the low partial band given only 3 of 7 tests pass.
The response normalises whitespace correctly by stripping and collapsing runs of spaces, which is a reasonable first step. However, it then reverses the characters of the entire string rather than the order of the words, so the core requirement is not met. The single-word case happens to pass because reversing characters is not applied to word order there, but any multi-word input yields incorrect output such as reversed letters within words. The code is short and returns one code block as instructed.
Granite Code 3B Instruct gap 2
Judged by J-O + J-A.
The function uses split and join, so whitespace normalization is appropriate, but it reverses each word's characters instead of reversing the list of words. This fails the stated examples and the programmatic result confirms only 2/7 tests passed. It also includes explanation outside the required single code block.
The response correctly identifies whitespace splitting and rejoins tokens with single spaces, which addresses part of the normalization requirement. However, its list comprehension applies slicing to each individual word rather than reversing the word sequence, producing character-reversed output such as “eht yks” instead of “blue is sky the.” The response also violates the requested output shape by adding prose before and after the code block. Its explanatory claim about reversing word order is inconsistent with the implementation.
Verdict is 2/7 passed, so the band is partial (3-6), but the code reverses characters within each word via word[::-1] instead of reversing word order, which is the core requirement. Example reverse_words('the sky is blue') would yield 'eht yks si eulb', a clear misreading. Score sits below the partial band, reflecting a wrong approach that only passes tests where words are unchanged or palindromic.
The response splits on whitespace and joins with single spaces, which correctly handles the whitespace collapsing and stripping requirements. However, it reverses the characters inside each word rather than the order of the words, so the central requirement is misimplemented. The explanation also incorrectly claims the list comprehension reverses word order, which contradicts the code. The code is short and readable, but it does not perform the requested transformation.
granite-4.1-3b gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reversed, and join to normalize whitespace and reverse word order. However, the task explicitly required returning one Python code block, while the response provides bare code without Markdown fences.
The response correctly implements the core behavior: split handles arbitrary whitespace, reversed changes word order, and join produces single-space separation while naturally removing surrounding whitespace. It is concise and otherwise follows the requested signature. Its concrete weakness is output formatting, since the task requested one Python code block and the model returned plain text code without fences. This does not affect execution, but it fails an explicit presentation constraint.
All 7 tests passed. The code uses str.split() with no arguments, which handles leading/trailing and repeated whitespace, then joins reversed words with single spaces. Matches the examples and the required signature. Single code block, no explanation, as instructed. Clean and idiomatic, though an unnecessary intermediate variable.
The response passes all seven programmatic tests with a concise implementation. Its main strength is using str.split() without arguments, which correctly handles leading, trailing, and repeated whitespace while also matching the required signature. It returns only one code block with no explanation, as the task requested. The weakness is minor: the intermediate variable reversed_words is unnecessary and could be returned directly. The code otherwise reads cleanly and idiomatically.
Granite3.2 2B Instruct gap 2
Judged by J-O + J-A.
The function passes all 7 tests, correctly handles arbitrary whitespace via split(), reverses word order, joins with single spaces, and returns an empty string for whitespace-only input. It follows the requirement to provide exactly one Python code block containing the function.
The response provides a correct, self-contained implementation with the requested signature and output format. Its use of Python’s whitespace-aware split and join naturally handles repeated, leading, trailing, and mixed whitespace, while reversing the resulting word list produces the required order. The explicit strip and intermediate normalization are redundant because split() already handles those cases, but they do not introduce a functional defect. The function also behaves appropriately for empty or whitespace-only strings. No explanation appears outside the single code block.
All seven tests pass. The response returns one Python code block with no explanation, as instructed. It uses strip, then join of split, then a second split and reversal, so the logic is correct but has redundant steps. Comments are present and accurate.
The response implements the required word reversal correctly, using whitespace splitting to collapse runs of whitespace and strip edges, and it passes every test. It returns only a single code block with no prose, which matches the instructions. Its main weakness is redundancy: it strips the string and then performs a join-split before splitting again, when one split and reversal would suffice. The logic is correct, but the implementation is less streamlined than it could be.
Granite4 1B H gap 2
Judged by J-O + J-A.
The function correctly strips whitespace, splits on arbitrary whitespace, reverses word order, and joins words with single spaces. All 7 tests passed. However, the response omitted the required Python code block formatting, returning bare code instead.
The response provides a concise and idiomatic implementation using stripping, splitting, reversal, and joining, which satisfies the functional requirements and passed every supplied test. Its weakness is output-format compliance: the task explicitly required exactly one Python code block, but the function was presented as un fenced plain text. The implementation itself does not show an evident latent correctness issue for the stated whitespace and word-order behavior. The formatting omission limits the score despite complete functional correctness.
All 7 tests pass. The solution uses str.split() without arguments, which handles arbitrary whitespace runs and strips leading/trailing whitespace, then reverses and rejoins with single spaces. Concise, idiomatic, and returns exactly one code block with no explanation as requested.
The response implements the required reverse_words function in a single expression: it splits on default whitespace, reverses the resulting list, and joins with single spaces. This correctly covers whitespace collapsing and trimming without any manual index handling. The code is clean and follows the output-shape instruction by providing only one code block with no prose. A minor limitation is that it offers no handling commentary or explicit reasoning about empty input, although the default split behaviour already returns an empty string for that case. Overall the solution is correct and appropriately terse.
Hermes-3-Llama-3.1-8B gap 2
Judged by J-O + J-A.
All 7 tests passed, and the implementation correctly strips, splits on arbitrary whitespace, reverses word order, and rejoins with single spaces. It is concise and idiomatic, but it does not follow the instruction to return one Python code block.
The response correctly implements the required transformation using stripping, whitespace splitting, reversal, and joining, covering the specified behavior and passing every test. Its main weakness is output formatting: it provides raw Python code rather than enclosing the function in a single Python code block as explicitly requested. The implementation itself is clean and handles empty or whitespace-only strings naturally, but the format deviation prevents the highest score despite full functional correctness.
All 7 tests pass. The code uses s.strip().split()[::-1] joined by single spaces, which correctly reverses word order, collapses whitespace runs, and strips leading/trailing whitespace. It is a single concise idiomatic expression with no extraneous commentary, and it returns only one code block as instructed.
The response implements the required function in one concise line that strips the input, splits on arbitrary whitespace, reverses the resulting word list, and rejoins with single spaces. This handles whitespace collapsing and trimming correctly without needing explicit edge-case branches. The main weakness is minimal: the solution relies on str.split() with no argument, which implicitly treats all whitespace characters as separators, a behavior that is correct here but could be less obvious to a reader unfamiliar with that default. Otherwise the output follows the format constraint of a single code block with no explanation.
Internlm2 7B Chat V2.5 gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse order, and join to normalize whitespace. However, the output includes an extra token after the Python code block, violating the instruction to return only one code block.
The response’s strength is its concise, idiomatic implementation: split() handles arbitrary whitespace, slicing reverses the word sequence, and join() produces the required single-space formatting. Its weakness is output-shape compliance. After the otherwise correct Python code block, the response includes an additional token, so it does not strictly satisfy the requirement to return only one code block. The implementation itself has no apparent correctness issue for the specified behavior, including empty or whitespace-only strings.
All 7 tests pass. The code is a single concise idiomatic expression, `' '.join(s.split()[::-1])`, which handles whitespace collapsing and stripping via str.split(). It returns one Python code block with the function and no explanation, as requested. A stray control-like token trails the closing fence, a minor formatting blemish.
The response implements reverse_words with a single idiomatic line that relies on str.split() to collapse whitespace and strip ends, then reverses the resulting word list and rejoins with single spaces. It correctly matches the required signature and returns only one Python code block without explanation, as instructed. The main weakness is a stray non-standard token appended after the closing code fence, which is extraneous output outside the requested block. The code itself has no evident latent bugs for the stated task.
Llama-3.2-3B-Instruct gap 2
Judged by J-O + J-A.
All 7 tests passed, and the function correctly strips whitespace, splits on arbitrary whitespace, reverses word order, and joins with single spaces. However, the task explicitly required returning one Python code block, while the response provided bare code without fences.
The response implements the required behavior concisely: `split()` handles whitespace runs and empty or surrounding whitespace, while reversing and joining produces the specified normalization. Its main weakness is output formatting. The task required exactly one Python code block containing the function, but the response omitted the code fences and returned plain text. The implementation itself is clean and idiomatic, and no behavioral defect is indicated by the complete test result.
All 7 tests pass. The code strips outer whitespace, splits on arbitrary whitespace runs (which collapses internal runs to single spaces), reverses the word list, and joins with single spaces. It is concise and idiomatic, with a single code block and no extra prose. The strip() call is redundant because split() already ignores leading and trailing whitespace, a minor inefficiency only.
The response implements reverse_words with a clean, short approach: it uses str.split() without arguments, which handles arbitrary whitespace runs and leading or trailing spaces, then reverses and rejoins with single spaces. The solution passes every test and follows the instruction to return only one code block with no explanation. Its weakness is a redundant strip() call before split(), since split() already discards outer whitespace, which adds a step without changing behavior. Otherwise the code is direct and correct for the stated requirements.
Olmo 3.1 32B Instruct gap 2
Judged by J-O + J-A.
All 7 tests passed, and the function correctly uses split() to handle arbitrary whitespace and join() to normalize spacing while reversing word order. However, the task explicitly required one Python code block, and the response provided bare code without code fences.
The response correctly implements the requested transformation using Python’s whitespace-aware split and a single-space join, including the empty-string behavior implied by those operations. Its main weakness is output formatting: the task explicitly required returning one Python code block, but the model supplied un fenced source text instead. The implementation itself is concise, idiomatic, and does not introduce unnecessary behavior or explanation.
All 7 tests pass. The solution uses str.split() with no arguments, which collapses whitespace runs and strips leading and trailing whitespace, then rejoins reversed words with single spaces. It is a single concise code block with no extraneous explanation, matching the output-shape instruction. No edge cases are visibly mishandled.
The response implements the required reverse_words function in three lines, relying on str.split() to handle whitespace collapsing and stripping in one step. This makes the behaviour for leading, trailing, and repeated whitespace correct by construction. It returns only a single code block with no added prose, as instructed. The code has no docstring or comments, which is acceptable since it is already clear. A minor weakness is that the function does not explicitly document its whitespace handling, so a reader must know split()'s default semantics to confirm the collapsing behaviour.
Olmo-3-7B-Think gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split, reverse, and join to handle arbitrary whitespace and word order. However, the response returns un fenced Python text rather than the single Python code block explicitly requested.
The response provides a concise and idiomatic implementation using `s.split()` to collapse whitespace and `join` to produce the reversed word sequence. It also handles empty or whitespace-only strings naturally. Its main weakness is formatting: the task explicitly requested one Python code block, but the model returned bare source text without code fences. This does not affect functional correctness, but it fails an explicit output-shape instruction.
All 7 tests passed. The code uses s.split() with no arguments, which collapses whitespace runs and strips leading and trailing whitespace, then reverses and rejoins with single spaces. It is a single, idiomatic one-line solution with no extraneous code block or commentary, matching the instruction to return only one code block.
The response implements the required behaviour in one concise expression. Using str.split() without arguments handles both whitespace collapsing and stripping of leading and trailing whitespace, which is the subtle part of the task. The function is returned as a single code block with no explanation, as requested. A minor weakness is that it does not include a type-agnostic guard or any note on non-string inputs, though the signature specifies str and the task does not require such handling. The solution is correct and clean overall.
Openthinker 7B V2 gap 2
Judged by J-O + J-A.
The function passes all 7 tests and correctly uses split() and reversed() to normalize whitespace and reverse word order. However, the response violates the required output shape by including extensive reasoning and explanation instead of returning only one Python code block.
The response provides a correct and concise implementation using Python’s whitespace-aware split, reversed, and join operations, including appropriate behavior for empty input and repeated whitespace. Its main weakness is instruction compliance: it includes a long internal reasoning section, explanatory prose, headings, and a solution explanation around the code, despite the task explicitly requiring only one Python code block. The implementation itself is clean, but the surrounding material makes the delivered answer unsuitable for the specified output format.
All 7 tests pass, so the code is correct. However, the output violates the instruction to return only one code block with no explanation: it includes a visible think block, a multi-section approach write-up, and a long explanation after the code. Per the coding band rules, extra blocks or non-requested prose costs points within the passing band, so the score is set to 7 at most; the explanation and leaked reasoning lower it further.
The response produces a correct solution: it uses str.split() without arguments, which handles whitespace collapsing and trimming, then reverses and joins with single spaces, and all seven tests pass. The weakness is instruction following. The task asked for only one Python code block with no explanation, but the model emitted visible reasoning, a headed approach section, and a lengthy explanation after the code. The code itself is concise and idiomatic, using reversed() cleanly.
Qwen2 Math 1.5B Instruct gap 2
Judged by J-O + J-A.
The implementation passed 4/7 tests but incorrectly uses split(" "), so leading, trailing, and repeated whitespace produce empty entries instead of normalized words. It also violates the requirement to return only one code block by including extensive explanation and multiple code blocks.
The response does provide a correctly named function with the requested signature and reverses the resulting list, which likely accounts for its passing basic cases. However, splitting only on a literal space fails the central whitespace-normalization requirement, especially for repeated, leading, trailing, or non-space whitespace. The response also includes explanations, verification snippets with incorrect function names, and multiple code blocks despite the explicit one-block-only instruction. Its claim that the function is correct is therefore not supported by the implementation or the test result.
Code uses s.split(
) with a literal space, so runs of whitespace and leading/trailing whitespace are not collapsed; the verdict shows 4/7 passing, which fits the partial band. The response also includes multiple code blocks, repeats the final answer twice, and its own test calls use undefined names reverseWords and reversewords, violating the one-block instruction.
Qwen2.5-Coder-3B-Instruct gap 2
Judged by J-O + J-A.
The response provides a concise, valid Python function and follows the requested single-code-block format. However, the programmatic verdict reports only 3 of 7 tests passed, placing it in the partial-credit band despite the implementation appearing to address ordinary whitespace-separated inputs.
The response’s strength is its compact implementation: it uses `split()` to handle leading, trailing, and repeated whitespace, then joins words with single spaces, and it presents exactly one code block without explanation. Its weakness is that the hidden test results show only 3 of 7 tests passing, indicating incompatibility with some tested requirements or edge cases. Although the visible implementation appears semantically aligned with the stated examples, the benchmark verdict establishes that it did not satisfy the complete test suite.
The code splits on whitespace and joins with single spaces but never reverses the word order, so it returns 'the sky is blue' instead of 'blue is sky the' for the example. Verdict of 3/7 tests passed falls in the partial band, but the function fails the core requirement, suggesting only trivial single-word or already-ordered cases pass. Score placed at the low end of the 3-6 band reasoning, adjusted down for a missing core operation.
The response correctly collapses whitespace runs and strips leading and trailing spaces by using split and join, and it returns a single code block as instructed. Its main weakness is that it omits the word reversal entirely, so the output keeps the original word order. This fails the central requirement of the task and would fail any test where order matters, including the provided example. The code is concise, but the missing reversal step means the solution is incomplete.
The other 198 models
Their judges were closer together. Their scores are listed here; open the model's page for the full answer.
Withheld
These models are kept out of the ranking, so none of their scores or reasons are shown:
The judge room names the judges and shows where they disagree most.