Skip to content

Prevent O(N^2) regex backtracking on long lines - #5601

Open
Xeicker wants to merge 2 commits into
masterfrom
fix-stackparser-redos
Open

Xeicker wants to merge 2 commits into
masterfrom
fix-stackparser-redos

Conversation

@Xeicker

@Xeicker Xeicker commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Bug: 582424297

Context

When a target exits printing a very long single line without newlines (for example, d8 throwing a JavaScript ReferenceError on a 100–400 KB identifier after a crash is fixed on trunk), CrashResult.is_crash() invokes StackParser.parse() twice per round and stalls for tens of minutes per round, causing progression_task to time out and leave fixed testcases open (similar to #3978):

  1. Seven sanitizer regexes (SAN_ABRT_REGEX, SAN_BREAKPOINT_REGEX, SAN_FPE_REGEX, SAN_ILL_REGEX, SAN_TRAP_REGEX, SAN_SEGV_REGEX, WINDOWS_SAN_ILL_REGEX) start with .*[a-zA-Z]+Sanitizer, which backtracks quadratically (O(N^2)) across long runs of ASCII letters on every non-matching line.
  2. StackParser.parse() calls ANDROID_SIGTRAP_REGEX.search(line) (r'.*signal.*\(SIGTRAP...') unconditionally before update_state_on_match, causing O(N^2) .search() scans across every starting offset of any long line (including non-alphabetic lines).

Fix description

  • Replace .*[a-zA-Z]+Sanitizer with .*[a-zA-Z]Sanitizer across those 7 regexes. Because leading .* is greedy, it already consumes all preceding letters on any match (preserving the exact same matched strings and capture groups), while making non-matching lines O(N).
  • Pass address_from_group=1 directly to update_state_on_match(ANDROID_SIGTRAP_REGEX, ...) so it uses .match(line) once instead of calling ANDROID_SIGTRAP_REGEX.search(line) first.

Testing

  • Unit tests: Added test_parse_long_line_no_dos in stack_analyzer_test.py (100 KB line parses in ~20 ms instead of ~80 s); verified test_mte_sigtrap_crash in src/clusterfuzz/tests/test.py and all appengine and core unit tests pass.
  • Validation (development environment): Noticed on real V8 testcases whose progression checks previously timed out (~42 min/round) on long single-line ReferenceError outputs, and verified that they now complete cleanly in seconds.

When a target exits printing a very long single line without newlines
(e.g. `d8` throwing a JavaScript `ReferenceError` on a 100-400 KB
identifier after a crash is fixed on trunk), `StackParser.parse()`
backtracks quadratically across eight regex checks:

1. Seven sanitizer regexes (`SAN_ABRT_REGEX`, `SAN_BREAKPOINT_REGEX`,
   `SAN_FPE_REGEX`, `SAN_ILL_REGEX`, `SAN_TRAP_REGEX`,
   `SAN_SEGV_REGEX`, `WINDOWS_SAN_ILL_REGEX`) start with
   `.*[a-zA-Z]+Sanitizer`, backtracking in `O(N^2)` across long runs of
   ASCII letters when the line does not match.
2. `StackParser.parse()` calls `ANDROID_SIGTRAP_REGEX.search(line)` on
   `r'.*signal.*\(SIGTRAP...'` before `update_state_on_match`, causing
   `O(N^2)` `.search()` scans across every starting offset of any long
   line.

Replace `.*[a-zA-Z]+Sanitizer` with `.*[a-zA-Z]Sanitizer` (greedy `.*`
already consumes preceding letters on any match, preserving identical
matches and capture groups while making non-matches `O(N)`) and pass
`address_from_group=1` directly to `update_state_on_match` for
`ANDROID_SIGTRAP_REGEX`.
@Xeicker
Xeicker requested a review from a team as a code owner November 13, 2026 18:00
@Xeicker
Xeicker requested a review from IvanBM18 November 13, 2026 18:07

@IvanBM18 IvanBM18 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Xeicker
Xeicker requested a review from smolkaj November 13, 2026 21:07

@smolkaj smolkaj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, awesome improvement!

In one place where actually changing the semantics of the regex because it contains a capture group -- let's double check that this is intendended and safe. Otherwise LGTM!

SAN_ABRT_REGEX = re.compile(r'.*[a-zA-Z]+Sanitizer: ABRT ')
SAN_BREAKPOINT_REGEX = re.compile(r'.*[a-zA-Z]+Sanitizer: breakpoint ')
# Match a single letter before 'Sanitizer' since greedy '.*' already consumes
# any preceding letters; '.*[a-zA-Z]+' causes O(N^2) backtracking on long lines.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this comment -- especially the reference to '.*[a-zA-Z]+' -- still make sense after this PR has been submitted?

I think not. I'd suggest simply removing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

SAN_ADDR_REGEX = re.compile(r'.*(ERROR: [a-zA-Z]+Sanitizer)[: ]*(.*) on '
r'(unknown address |address |)([xX0-9a-fA-F]+)')
SAN_SEGV_REGEX = re.compile(r'.*([a-zA-Z]+Sanitizer).*(SEGV|access-violation) '
SAN_SEGV_REGEX = re.compile(r'.*([a-zA-Z]Sanitizer).*(SEGV|access-violation) '

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the capture group, i.e. the part inside the parens.

Why is that safe? Are we using the capture today?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't really change the result since regex is greedy the ' will swallow all [a-zA-Z] that + would capture, so the final captured group is the same.
In any case this group is not used, only group 3

@smolkaj smolkaj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered if there is a decent way that we can actually test performance, so that a future performance regression would be caught and prevented? We'd want to make sure the test isn't flaky, since test execution times can have pretty high variance.

One potential idea: Could we perhaps run the O(n) code on inputs of size k and 100k, and check that the runtime grows by far less than 10'000x? Say < 1000x?

Not a strict blocker for this CL, could be done as a follow up or omitted if it turns out to be too tricky to implement and make reliable.

@Xeicker

Xeicker commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

Have you considered if there is a decent way that we can actually test performance, so that a future performance regression would be caught and prevented? We'd want to make sure the test isn't flaky, since test execution times can have pretty high variance.

One potential idea: Could we perhaps run the O(n) code on inputs of size k and 100k, and check that the runtime grows by far less than 10'000x? Say < 1000x?

Not a strict blocker for this CL, could be done as a follow up or omitted if it turns out to be too tricky to implement and make reliable.

Done, added the 1k vs 100k < 1000x check using min-of-3 time.process_time() runs plus a 5s ITIMER_VIRTUAL CPU-time cap (~125 ms median). Verified 0 flakes across 300+ runs (100 serial + 200 across 16 parallel workers), and 55/55 expected failures across 11 injected regressions (5 runs each for all 8 reverted regexes hitting the 5s cap, plus 3 sub-5s letter/digit/whitespace O(N^2) cases caught by < 1000x

@Xeicker
Xeicker requested a review from smolkaj November 18, 2026 23:34
@jardondiego

jardondiego commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Validation (development environment): Noticed on real V8 testcases whose progression checks previously timed out (~42 min/round) on long single-line ReferenceError outputs, and verified that they now complete cleanly in seconds.

I think it would be great if you could share more details on this. I think this is a great addition!

state,
new_type='Trap',
new_address=(mte_match.group(1) if mte_match else ''))
address_from_group=1)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be good to have a small comment here explaining the magical number 1.

@jardondiego

jardondiego commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

AFAIU src/clusterfuzz/tests/test.py does not run in CI or butler.py py_unittest. The SIGTRAP tests in stack_analyzer_test.py only cover GDB and V8, so ANDROID_SIGTRAP_REGEX has no positive presubmit test. What do you think about adding a small test in stack_analyzer_test.py for an Android SIGTRAP line verifying Trap and the extracted fault address?

Could be something similar to this...

def test_android_sigtrap_extracts_fault_address(self):
  """Tests that Android SIGTRAP lines set Trap and extract fault addr."""
  stacktrace = (
      'Fatal signal 5 (SIGTRAP), code 1 (TRAP_BRKPT), '
      'fault addr 0x5d9fdb4188 in tid 27246 (example_mte_fuz)\n'
      'signal 5 (SIGTRAP), code 1 (TRAP_BRKPT), '
      'fault addr 0x0000005d9fdb4188 (read)\n'
  )
  actual_state = StackParser().parse(stacktrace)
  self.assertEqual(actual_state.crash_type, 'Trap')
  self.assertEqual(actual_state.crash_address, '0x005d9fdb4188')

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants