Fail gracefully when a page hangs: add timeouts and recovery to the find step #300

Summary

If a page stops responding while it's being scanned, the find action can wait forever. URLs are scanned one at a time, so one unresponsive page blocks every URL after it. The run then continues until the GitHub Actions job timeout cancels it, which is 6 hours by default. When that happens, no findings are filed for any URL in the run.

We'd like the scanner to protect itself from hangs like this and fail gracefully:

• Limit how long any one URL or plugin can take.
• Recover from a stuck page or browser.
• Report clearly what didn't complete.
• Keep scanning the remaining URLs.

What we've observed

This has happened with the built-in reflow-scan plugin on some pages, after the plugin resizes the viewport to 320×256:

• The last log line before cancellation is Running plugin: reflow-scan. Nothing else is logged until the job is cancelled.
• Where it stalls varies:
  • With screenshots enabled, it stalls during the full-page screenshot taken for a finding.
  • With screenshots disabled, it stalls in the page.evaluate call that measures horizontal overflow.
• Playwright errors such as page.screenshot: Timeout 30000ms exceeded and page.evaluate: Target page, context or browser has been closed only show up when the job is cancelled. The screenshot's default timeout doesn't rescue the run in practice.

We haven't found the root cause of the page or browser hang. This issue is about making the scanner resilient whatever the cause, not just in reflow-scan.

Where there's currently no limit

Links are to main at abd3a74:

• URL loop: find/src/index.ts awaits findForUrl for each URL in turn, with no deadline per URL.
• Navigation: in findForUrl, page.goto and waitForSelectors run outside the inner try/catch. A navigation error or selector timeout on one URL therefore looks like it ends the whole find step, rather than just skipping that URL.
• Scans: plugins, axe, and AccessLint are awaited in turn with no overall deadline (findForUrl.ts#L59-L95). Cleanup awaits context.close() and browser.close(), which can also hang if the browser is stuck.
• Plugin calls: invokePlugin calls plugin.default directly, with no timeout.
• reflow-scan: the plugin awaits:
  • setViewportSize
  • two page.evaluate calls, which have no timeout option in Playwright
  • a viewport restore in finally, which can hang too and hide the original error
• Screenshots: generateScreenshots relies on Playwright's default timeout for full-page captures.

Incomplete scans look like clean scans

If a plugin errors or a scan is cut short, the findings collected so far are returned as if the URL had been fully scanned. For example, reflow-scan catches and logs its own errors. In the file step:

• Cached filings are reset and refilled from the current findings (updateFilingsWithNewFindings).
• A filing with no findings is treated as resolved (isResolvedFiling).

So adding timeouts by themselves could close open issues as resolved even though the problem was never re-checked. The same happens when a scan type is removed from the scans input. Failing gracefully needs to cover both the hang and how the results are handled afterwards.

Implementation tasks

The work is split into four focused sub-issues. Each owns its implementation, regression tests, and directly related documentation; the behavior below remains the overall acceptance target.

Dependencies and delivery: Plugin hardening and the completion contract can be developed in parallel. Runtime recovery owns the hard execution boundary; the plugin should not duplicate that browser-management logic. Safe reconciliation consumes the reporting contract and can be tested against fixtures before runtime recovery is finished.

Safety gate: Connect reporting and safe reconciliation before allowing a recovered, partially completed run to perform ordinary issue resolution. Until then, incomplete runs must fail visibly rather than provide a success-shaped empty findings file. This lets the tasks be developed separately without shipping false resolutions as an intermediate state.

Proposed behavior

• Deadline per URL: a configurable input with a sensible default, covering navigation, plugins, axe, AccessLint, screenshots, and cleanup.
• Deadlines per operation: for each plugin call and each screenshot, so one slow step doesn't use up the whole URL deadline.
• Real recovery: Promise.race alone can leave the browser stuck and make cleanup hang. On timeout:
  • force-close the context or browser, with a time limit on the close itself
  • kill the browser process if it doesn't close
  • relaunch if needed and continue with the next URL
• Isolate URL failures: a navigation, selector-wait, or plugin failure on one URL should be recorded for that URL without ending the rest of the run.
• Clear reporting: log a warning or annotation naming the URL, scanner or plugin, and the operation that timed out or failed. Expose a summary in the action outputs, such as the URLs and scans that didn't complete, so workflows can alert on them. Optionally, add a setting that fails the action when any scan is incomplete.
• Don't resolve what wasn't checked: track which scans completed for each URL. The file step should leave existing filings for an incomplete or skipped URL and scanner pair untouched instead of closing them. Issues for checks that completed and found nothing should still close as they do today.
• Restore the viewport safely: limit how long reflow-scan's viewport restore can take, and keep the original error if the restore fails.

Acceptance criteria

• Tests simulate a page.evaluate, page.screenshot, plugin call, and viewport restore that never return. Each one ends within the configured limit, later URLs are still scanned, and no browser processes are left behind.
• A navigation or selector-wait failure on one URL is reported, and the other URLs are still scanned.
• Timeouts and failures are clearly visible in logs and action outputs.
• Existing issues for a URL and scanner pair that timed out, errored, or was skipped stay open, while issues for checks that completed with no findings still close.
• Behavior and runtime for pages that don't hang are unchanged.

🤖 Assisted by Copilot