Skip to content

perf(api): build related_fields' nested serializers once per page - #16992

Open
blakeaowens wants to merge 1 commit into
devfrom
perf/finding-nested-serializer-reuse
Open

blakeaowens wants to merge 1 commit into
devfrom
perf/finding-nested-serializer-reuse

Conversation

@blakeaowens

Copy link
Copy Markdown
Contributor

Description

A 1000-row GET /api/v2/findings/?related_fields=true page spent most of its wall time in Python, and with JIRA enabled it also ran two queries per finding that has a JIRA issue.

  • FindingSerializer.get_related_fields built a new FindingRelatedFieldsSerializer for every row, and that built a new FindingTestSerializer (three levels of nested serializers) and a new JIRAIssueSerializer for every row too. Building a serializer builds its whole field tree, and nothing in those trees depends on the row. Each is now built once per serializer instance (a cached_property) and reused for every row the list renders, the same pattern perf(api): load the relations of prefetched duplicate findings once per page #16789 used in the prefetcher. A page without related_fields=true still builds none of them.
  • related_fields.jira.url reads the issue's JIRA project and its instance. Nothing loaded them with the page, so each issue cost two queries. FindingViewSet.get_queryset now prefetches jira_issue__jira_project__jira_instance, only when related_fields=true, so plain pages run exactly the statements they ran before.

Response bodies are byte-identical before and after.

Measured on 1000 findings (a third with a JIRA issue linked to a project), median of 9 requests, before and after under the same load:

1000-row page before after
superuser, related_fields=true 9,090 ms, 700 statements 4,854 ms, 34 statements
product Owner, related_fields=true 8,894 ms, 706 statements 5,303 ms, 40 statements
superuser, plain 4,179 ms, 32 statements 4,257 ms, 32 statements

A profile of the before request put about 6.3 s (under the profiler) in DRF get_fields, 6,537 calls, almost all from get_test.

Test results

New unittests/test_apiv2_finding_related_fields_reuse.py:

  • each of the three nested serializers is built once for a 2-row page and for a 6-row page (on dev: twice for 2 rows),
  • related_fields adds the same statements to 2 rows as to 6, measured against the plain page (on dev: 4 and 12),
  • the payload is byte-identical to the per-row construction, with and without a JIRA issue,
  • the JIRA prefetch is not added to a plain page.

2 of the 5 fail on dev for those reasons, and all 5 pass with the change. The paired downstream suite (finding list query scaling and performance tests) passes unchanged.

Documentation

No documentation change: nothing a user sees or configures changes.

Checklist

  • Make sure to rebase your PR against the very latest dev.
  • Submit all PRs, features and bug fixes alike, against the dev branch.
  • Give a meaningful name to your PR, as it may end up being used in the release notes.
  • Your code is Ruff compliant (see ruff.toml).
  • Your code is python 3.13 compliant.
  • If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
  • Model changes must include the necessary migrations in the dojo/db_migrations folder.
  • Add applicable tests to the unit tests.
  • Add the proper label to categorize your PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HJqD5EJ6LWVNAPF4qew47G

A 1000-row ?related_fields=true findings page spent most of its time
building serializers: get_related_fields built a FindingRelatedFieldsSerializer
for every row, which built a FindingTestSerializer (three nested levels)
and a JIRAIssueSerializer for every row too. Each is now built once per
serializer instance and reused for every row.

related_fields' jira url read each issue's JIRA project and instance with
a query each, so a related_fields page now prefetches
jira_issue__jira_project__jira_instance. Plain pages are unchanged.

Response bodies are byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJqD5EJ6LWVNAPF4qew47G
@blakeaowens blakeaowens added this to the 3.5.300 milestone Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant