ReDoS Regex Rewrite: Same Answers, Linear Time: test results

Verdicts

Date

2026-10-09

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 22, checked by running every answer: the same yes or no as the original on a pool of short texts, and each near-miss input finished within 200 ms. It collapsed nested runs, made optional separators mandatory where the set allows, shrank unanchored runs to one character, worked out a glued repeat of digits and decimals, turned a backreference over runs of one letter into an even count, kept the flags, left seven linear patterns as they were or rewrote them to the same set, and refused both patterns whose backreference repeats arbitrary text.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 22, checked by running every answer, with the same rewrites, the same seven safe patterns kept and the same two refusals as Sonnet.

With and without the skill

Tested 2026-10-09.

Same request on both sides; it states the JSON shape and asks for a CANNOT line when no linear pattern gives the same answers. Without the skill both models already rewrote all thirteen hanging patterns correctly and kept the safe ones. Each missed one refusal: for a pattern whose first word must equal its last word they returned another pattern with the same backreference, which still is not linear.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-two JavaScript patterns used with test(), written by us: 13 that hang on a near-miss input (nested runs, optional separators, overlapping alternatives, a dot run in a repeat, a counted group, unanchored runs, a classic e-mail validator, Unicode words, a backreference over runs of one letter), 2 that no regular expression can replace (a backreference that repeats arbitrary text) and 7 safe controls. The accepted strings of each original are measured over an exhaustive pool of short texts plus edits of seeds; every answer must agree on all of them and finish each near-miss input within 200 ms. The case script proves that each trap really hangs and each control does not. The examples in the skill are deliberately not the test patterns. No check was widened. One run per model and pattern.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.