Skip to content

perf(ci): Split large Node integration test suites - #26420

Merged
nicohrubec merged 12 commits into
developfrom
feat/benchmark-split-node-ai-suites
Nov 17, 2026
Merged

nicohrubec merged 12 commits into
developfrom
feat/benchmark-split-node-ai-suites

Conversation

@nicohrubec

@nicohrubec nicohrubec commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

What

Split long Node integration suites into smaller files so the existing shards can distribute work more evenly. As an example, before we had single test files for vercel ai that took 2-3 minutes to execute. Playwright shards split by test files so this makes the load distribution very uneven. We should prefer smaller files.

Why

The slowest required Node integration test step fell 184→162s (−22s, 12%); the longest complete Node integration job fell 229→198s (−31s).

Runtime Slowest test step Shard range, before → after
Node 20 184→145s 82–184 → 105–145s
Node 22 178→162s 82–178 → 110–162s
Node 24 166→141s 67–166 → 94–141s
Node 26 145→134s 66–145 → 92–134s
Node 27 α (optional) 146→131s 65–146 → 90–131s

Baseline → final benchmark. Single-run measurements; test-step times exclude setup/queueing, complete-job times include setup/cleanup. Inventories and pass/skip states match.

Co-Authored-By: OpenAI Codex <codex@openai.com>
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Compared: e78d40c (PR test merge using develop at 1d3a542)

Against: 1d3a542 (develop) · base run #66400

✅ Baseline matches: this diff contains only this PR's changes.

Path Size % Change Change
@sentry/browser 30.1 kB - -
@sentry/browser - with treeshaking flags 28.3 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 28.2 kB - -
@sentry/browser (incl. Tracing) 52.28 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.29 kB - -
@sentry/browser (incl. Tracing, Profiling) 55.25 kB - -
@sentry/browser (incl. Tracing, Replay) 92.82 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 81.6 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 97.21 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 110.6 kB - -
@sentry/browser (incl. Feedback) 47.73 kB - -
@sentry/browser (incl. sendFeedback) 30.69 kB - -
@sentry/browser (incl. FeedbackAsync) 35.69 kB - -
@sentry/browser (incl. Metrics) 31.43 kB - -
@sentry/browser (incl. Logs) 31.71 kB - -
@sentry/browser (incl. Metrics & Logs) 32.37 kB - -
@sentry/react 32.11 kB - -
@sentry/react (incl. Tracing) 54.95 kB - -
@sentry/vue 38.29 kB - -
@sentry/vue (incl. Tracing) 55.24 kB - -
@sentry/svelte 30.12 kB - -
@sentry/remix (Remix 3 client bundle) 58.06 kB - -
CDN Bundle 31.87 kB - -
CDN Bundle (incl. Tracing) 52.7 kB - -
CDN Bundle (incl. Logs, Metrics) 34.67 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 55.26 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 75.7 kB - -
CDN Bundle (incl. Tracing, Replay) 90.72 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 93.33 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96.09 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.69 kB - -
CDN Bundle - uncompressed 92.98 kB - -
CDN Bundle (incl. Tracing) - uncompressed 155.45 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 101.33 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 163.08 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 231.44 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 275.61 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 283.24 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 287.4 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 295.02 kB - -
@sentry/nextjs (client) 58.57 kB - -
@sentry/sveltekit (client) 53.57 kB - -
@sentry/core/server 44.8 kB - -
@sentry/core/browser 14.4 kB - -
@sentry/node 151.36 kB -0.01% -6 B 🔽
@sentry/node/import (ESM hook with diagnostics-channel injection) 83.3 kB - -
@sentry/node - without tracing 88.56 kB -0.01% -6 B 🔽
@sentry/node - without channel injection 140.3 kB -0.01% -6 B 🔽
@sentry/aws-serverless 98.93 kB -0.01% -6 B 🔽
@sentry/cloudflare (withSentry) - minified 206.47 kB - -
@sentry/cloudflare (withSentry) 479.22 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 231.83 kB - -

View base workflow run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/workflows/build.yml Resolved
Comment thread .github/workflows/build.yml Resolved
Co-Authored-By: OpenAI Codex <codex@openai.com>
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Optional E2E tests

Some optional checks were not successful. They do not block the merge. 1 not successful, 62 successful. See the run.

Check Status
❌ CI: Optional E2E Tests / E2E nextjs-16-cacheComponents (cloudflare) Test (optional) Failing after 11m Details
Successful checks
Check Status
✅ CI: Optional E2E Tests / E2E cloudflare-mastra Test (optional) Successful in 44s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-cloudflare Test (optional) Successful in 38s Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions (latest) Test (optional) Successful in 31s Details
✅ CI: Optional E2E Tests / E2E debug-id-sourcemaps Test (optional) Successful in 40s Details
✅ CI: Optional E2E Tests / E2E cloudflare-think (ai v6) Test (optional) Successful in 59s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-router-7-framework-instrumentation Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (cloudflare) Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E cloudflare-think Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable (latest) Test (optional) Successful in 52s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-sourcemaps Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E solidstart-2 Test (optional) Successful in 52s Details
✅ CI: Optional E2E Tests / E2E node-eve Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (latest) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-tunnel Test (optional) Successful in 46s Details
✅ CI: Optional E2E Tests / E2E solidstart-2-tunnel Test (optional) Successful in 53s Details
✅ CI: Optional E2E Tests / E2E astro-7-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext, bun) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E supabase-nextjs Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E astro-7 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (static / nuxt generate) Test (optional) Successful in 34s Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (bun) Test (optional) Successful in 53s Details
✅ CI: Optional E2E Tests / E2E cloudflare-flue-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (latest) Test (optional) Successful in 46s Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (deno) Test (optional) Successful in 48s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-tunnel (cloudflare) Test (optional) Successful in 59s Details
✅ CI: Optional E2E Tests / E2E react-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E deno (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-eve (content off, latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-eve (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-orpc Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (turbo) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (latest) Test (optional) Successful in 59s Details
✅ CI: Optional E2E Tests / E2E cloudflare-workers-without-tracing-send-to-sentry Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E cloudflare-agent Test (optional) Successful in 37s Details
✅ CI: Optional E2E Tests / E2E node-flue (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (cloudflare) Test (optional) Successful in 38s Details
✅ CI: Optional E2E Tests / E2E cloudflare-workers-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-eve (orchestrion) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-mastra Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 (development) Test (optional) Successful in 52s Details
✅ CI: Optional E2E Tests / E2E nuxt-5 Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v1 Test (optional) Successful in 26s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions Test (optional) Successful in 38s Details
✅ CI: Optional E2E Tests / E2E node-mastra (latest) Test (optional) Successful in 57s Details
✅ CI: Optional E2E Tests / E2E node-flue Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E node-pi-durable Test (optional) Successful in 39s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-cloudflare Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 (kv-cache) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 Test (optional) Successful in 31s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (cloudflare, latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps Test (optional) Successful in 32s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (client source maps kept) Test (optional) Successful in 38s Details
✅ CI: Optional E2E Tests / E2E vue-3 (no Options API) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable (latest) Test (optional) Successful in 40s Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v2 Test (optional) Successful in 37s Details

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9252e89. Configure here.

Comment thread .github/workflows/build.yml Resolved
nicohrubec and others added 5 commits November 16, 2026 15:58
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
@nicohrubec
nicohrubec force-pushed the feat/benchmark-split-node-ai-suites branch from 809a3be to edd5fe4 Compare November 16, 2026 17:09
nicohrubec and others added 4 commits November 16, 2026 18:19
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
@nicohrubec
nicohrubec marked this pull request as ready for review November 17, 2026 09:25
@nicohrubec nicohrubec changed the title test(ci): Benchmark natural splits of long integration suites perf(ci): Benchmark natural splits of long integration suites Nov 17, 2026
@nicohrubec
nicohrubec requested a review from JPeer264 November 17, 2026 09:25

@JPeer264 JPeer264 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. I'd still prefer to have the assertions directly in the tests, rather than having exports and then use them in the tests.

}

describe('express tracing', () => {
export function registerExpressRoutingTests(): void {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: Is there any benefit of having that in named exports (other than keeping the LOC changes low)? I'd prefer to have them actually directly in the tests. IMO having request-data/test.ts and all its assertions is fine.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for vercel AI it makes sense to have it like this because v6 and v7 share all the assertions. other than that no so I'll restructure the remaining ones

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, now look at that mighty diff

Co-Authored-By: OpenAI Codex <codex@openai.com>
@nicohrubec nicohrubec changed the title perf(ci): Benchmark natural splits of long integration suites perf(ci): Split large Node integration test suites Nov 17, 2026
@nicohrubec
nicohrubec enabled auto-merge (squash) November 17, 2026 10:54
@nicohrubec
nicohrubec merged commit d586a5d into develop Nov 17, 2026
467 of 468 checks passed
@nicohrubec
nicohrubec deleted the feat/benchmark-split-node-ai-suites branch November 17, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants