Skip to content

graph: rate limit the personal data export - #3973

Merged
butonic merged 3 commits into
opencloud-eu:mainfrom
pbleser-oc:feat/graph-export-personal-data-rate-limit
Nov 13, 2026
Merged

butonic merged 3 commits into
opencloud-eu:mainfrom
pbleser-oc:feat/graph-export-personal-data-rate-limit

Conversation

@pbleser-oc

Copy link
Copy Markdown
Member

Limits how many personal data exports a user can start: 5 per minute by default, configurable with GRAPH_EXPORT_PERSONAL_DATA_RATE_LIMIT (0 disables the limit). Requests beyond it get 429 Too Many Requests with a Retry-After header.

Changes

  • RequireOwnUserID (graph middleware): checks that users only export their own data before the rate limit runs, so that requests for other users don't count against any limit. The route passes the name of its user id parameter. They still get a 400, now with a JSON error body instead of plain text. A code comment notes that 403 would arguably fit better; it stays 400 so that clients aren't affected.
  • The handler keeps its own check as a safety net, so that the data of one user can never be exported to another, also when the route is changed by mistake. If it catches a request, it logs an error, since the middleware must be missing.
  • RateLimitPerAuthenticatedUser (pkg/middleware): a reusable rate limit that counts requests by the id of the authenticated user, not by the URL, so that variations of the URL (e.g. a trailing slash, which the router strips for routing but which stays in the URL) can't get around it. It requires an authenticated user in the context and answers 401 without one.
  • The counters are kept in memory by every instance of the graph service: with several instances, a user can start up to the limit on each of them. The README says so.

Not part of this PR: limiting how many exports run at the same time, see #3970.

New dependency

This adds github.com/go-chi/httprate v0.16.1 (MIT), by the authors of the chi router that graph already uses.

  • Why not our own limiter: counting requests in sliding time windows correctly under concurrency is easy to get wrong, and httprate is maintained and tested for exactly that. Our part is only choosing the key (the user id) and the error response.
  • Modules it pulls in: none new. It needs github.com/zeebo/xxh3 and github.com/klauspost/cpuid/v2, which are already required at newer versions.
  • Vulnerabilities: no advisories on GitHub or OSV for httprate or the two modules above, and govulncheck reports nothing in them.
  • Open issues upstream that touch our use, neither a vulnerability: a request that waits for the limiter's lock across a window boundary can make the in-memory counter drop counts (OnLimit can discard local counters when a queued request crosses a window boundary go-chi/httprate#66, fix in Rebrand settings #67), which lets a few extra requests through in rare timings; and Retry-After is always the full window rather than the remaining time (Retry-After header should maybe not be static go-chi/httprate#47, fix in webfinger #64).

Reviewers can skip vendor/ in the diff.

Tests

  • Unit tests for both middlewares and for the handler's own check without any middleware, including URL variants (escapes, a zero-width space, another Unicode normalization form, case) and a missing user.
  • Tests through the graph router: requests for other users don't consume any budget, URL variants share one budget, and a limit of 0 disables it. The ordering test fails when the two middlewares are swapped.
  • apiGraph/userGDPRExport.feature passes on posix and decomposed.

AI assistance: Claude Code (claude-opus-5-5) analysed the options, wrote the first draft of the code, the tests and this description, and checked the dependency for licenses and vulnerabilities.

@codacy-production

codacy-production Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 44 complexity

Metric Results
Complexity 44

View in Codacy

🟢 Coverage 92.75% diff coverage

Metric Results
Coverage variation Report missing for 4bb1fee1
Diff coverage ✅ 92.75% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (4bb1fee) Report Missing Report Missing Report Missing
Head commit (0169f3b) 94146 27948 29.69%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#3973) 69 64 92.75%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Comment thread pkg/middleware/ratelimit.go Resolved
@butonic
butonic deleted the branch opencloud-eu:workpagages November 13, 2026 16:53
@butonic butonic closed this Nov 13, 2026
@butonic butonic reopened this Nov 13, 2026
@butonic
butonic changed the base branch from workpagages to main November 13, 2026 17:06
@butonic
butonic enabled auto-merge November 13, 2026 17:20
Check that a user exports only their own personal data in a new
middleware, RequireOwnUserID, so that the check can run before other
middlewares on the route, such as a rate limit. The route passes the
name of its user id parameter to it.

The handler keeps its own check as well, as a safety net: exporting the
data of one user to another must never happen, also when the route is
changed by mistake. When that check catches a request, it logs an error,
since the middleware must be missing.

Both unescape the user id from the path, like the other handlers of the
users endpoint do, and reject an empty one. The middleware answers 401
when there is no authenticated user. Requests for other users still get
a 400, now with a JSON error body instead of a plain text one.

Assisted-by: Claude Code:claude-opus-5-5
Each personal data export collects all events of the user and writes
them into the user's personal space, so starting exports over and over
puts load on the system. Limit how many exports a user can start to 5
per minute, configurable with GRAPH_EXPORT_PERSONAL_DATA_RATE_LIMIT
(0 disables the limit). Requests beyond it get a 429 with a
Retry-After header.

The limit uses a new middleware, RateLimitPerAuthenticatedUser. It
counts requests by the id of the authenticated user rather than by the
URL, so that variations of the URL, such as a trailing slash, can't get
around it. It comes after the check that users only export their own
data, so that requests for other users don't count against any limit.
The counters are kept in memory by every instance of the graph service.

This adds the dependency github.com/go-chi/httprate v0.16.1 (MIT), from
the authors of the chi router that the service already uses. It is used
instead of a limiter of our own, because counting requests in sliding
time windows correctly under concurrency is easy to get wrong, and the
library is maintained and tested for exactly that. It brings no further
modules: the two it needs are already required. There are no known
vulnerabilities in it, and govulncheck reports none.

Assisted-by: Claude Code:claude-opus-5-5
A request to export the personal data beyond the rate limit got the
plain text 429 of httprate, while the other errors of graph, also those
of the same route, are JSON with an error code.

RateLimitPerAuthenticatedUser takes a handler for the response beyond
the limit now, so that a service can answer in its own format; nil
keeps the plain text 429. Graph answers with activityLimitReached. The
Retry-After header is set before the handler is called, so it stays.

Assisted-by: Claude Code:claude-opus-5-5
@butonic
butonic force-pushed the feat/graph-export-personal-data-rate-limit branch from 9ad2c55 to 0169f3b Compare November 13, 2026 17:26
@butonic
butonic merged commit 8ba2c3c into opencloud-eu:main Nov 13, 2026
59 of 60 checks passed
@pbleser-oc
pbleser-oc deleted the feat/graph-export-personal-data-rate-limit branch November 13, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants