Skip to content

perf(mobile): save the open-channel directory and catch it up - #9513

Open
brow wants to merge 4 commits into
mainfrom
tomb/mobile-saved-directory
Open

brow wants to merge 4 commits into
mainfrom
tomb/mobile-saved-directory

Conversation

@brow

@brow brow commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Search and invite recovery read a community's open-channel directory (every open, listed channel's kind 39000 metadata) from the relay, and match names on the device. Until now the app kept that directory in memory for 30 seconds only, so every search session after that, and the first search after every launch, downloaded the whole directory again. This follows #9466 directory-only search load and #9457 directory keeps message times, which made each of those downloads cheaper; this one makes most of them unnecessary.

Now the directory is saved in a file per community and account, next to the image cache, and search reads it from there:

  • First read. The first time search opens, the app downloads the whole directory, as before, and saves it.
  • Catch-up. When search opens later, it shows the saved copy at once. At most once a minute, it asks the relay in the background only for directory changes since the newest saved one (less a minute of overlap), and merges them in. A catch-up that brings back nothing new saves nothing and leaves the list alone, and a search opened while a background read is still running shares it rather than starting another.
  • Daily full read. Once the last full read is a day old, the next check downloads the whole directory in the background and replaces the saved copy, which drops channels that became private or unlisted or were deleted. A catch-up can't see those: the relay never serves them to a non-member.
  • Gone on open. Opening a saved result that the relay no longer serves (private or deleted) drops it from the directory at once, and a background read already in flight does not bring it back.
  • Invite recovery still downloads the whole directory, as on main: it needs fresh memberships, and its read also refreshes the saved copy.
  • Cleanup. Signing out of a community, replacing its account, or leaving it deletes the file.

The saved events omit signatures (the relay signed them; the copy is a cache of what it served). The file's JSON is encoded and decoded off the UI isolate, and an unavailable caches directory (or one that takes over 3 seconds) just means no saved copy. A file that is unreadable, from another version, or over 20,000 channels or 16 MiB is discarded and read again from the relay. There is no relay change. CONTEXT.md gains an "Open-channel directory" entry.

Validation

Workflow (small community, 63 open channels) Directory bytes before Directory bytes after
First search after install 39.5 KB (2 requests) 39.5 KB (2 requests)
Search again 2 minutes later 39.5 KB (2 requests) 0.6 KB (2 requests)
Relaunch, then search 39.5 KB (2 requests) 0.6 KB (2 requests)

Measured on an iOS simulator signed in to a test community, with a temporary local counter on each relay query response (not part of this PR), main at 6974237 vs this branch. The full read grows with the number of open channels (about 600 bytes each); the catch-up grows only with what changed. Search results were identical on both builds, and opening an unjoined result from the saved directory still showed its preview with Join.

Tests:

  • A journey test drives the real channel list against a fake relay that honors since, with a real file and a fake clock: the first reader saves the directory without signatures; more readers within a minute ask nothing; a minute later a reader asks only for changes since the newest saved one less a minute; after a relaunch the saved copy serves at once and the catch-up starts from it; a channel that turned private stays until the daily full read replaces the copy; the search status never shows loading for the background reads; a catch-up of only held events leaves the list identical and the file unwritten; a second search a minute into a slow full read shares it; and opening a channel the relay no longer serves drops it, even with a read in flight. Each guard was checked by breaking it: removing the once-a-minute limit, dropping since, never doing the full read, merging the full read instead of replacing, removing the drop-on-open, the no-change check, the shared in-flight read, or the in-flight drop fence each fail the test.
  • Storage tests: the round trip omits signatures; unusable data is discarded; a directory over the bounds leaves no stale copy; a closed store neither reads nor writes; cleanup keeps only each community's current account.
  • The community test now also checks that replacing an account deletes the old account's saved directory (fails without the cleanup call).
  • Full mobile test suite and flutter analyze pass.

No visible change, so no screenshots: the same results appear, just without a download on later searches.

Regressions

  • A channel that turns private or is deleted can still appear in search for up to a day, until the daily full read, unless you open it (opening drops it at once, and the relay withholds its content). Before, it disappeared from the next search at least 30 seconds later.
  • A channel that turns unlisted can also appear in search for up to a day. Opening it does not drop it, and its content stays readable, because the relay serves an unlisted open channel to anyone who names it.
  • A channel created while search is open appears the next time search opens, not while it stays open. That was already the case within the 30-second window.
  • The saved file costs some disk per community (about 30 KB for 63 channels, roughly 0.5 KB per channel), deleted on sign-out or leaving.

square-tomb and others added 2 commits November 14, 2026 17:25
Search and invite recovery read the community's whole open-channel
directory (kind 39000) from the relay. The list kept it in memory for
30 seconds, so every search session after that, and every launch,
downloaded the whole directory again.

Save the directory in a file per community and account, next to the
media cache. Readers serve the saved copy at once. When a reader
opens, at most once a minute, the app asks the relay only for
directory changes since the newest saved one (less a minute of
overlap). Once the last full read is a day old, the next check reads
the whole directory in the background and replaces the copy, which
drops channels that became private, unlisted or were deleted. Opening
a channel the relay no longer serves drops it at once. Signing out or
leaving the community deletes the file.

Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
Signed-off-by: Tom Brow <tomb@squareup.com>
Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
Signed-off-by: Tom Brow <tomb@squareup.com>
@brow

brow commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T02:38:06.051072Z e3a4400 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 7307d0f0ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

square-tomb and others added 2 commits November 14, 2026 17:49
Review fixes for the saved open-channel directory:

- Encode, measure and decode the file's JSON in Isolate.run; the UI
  isolate only reads and writes bytes.
- Readers opening while a background directory read is in flight share
  it, so a full read slower than the check interval is not displaced and
  restarted.
- A catch-up that returns only held events (the overlap always returns
  the newest) saves nothing and leaves the list untouched.
- A read in flight when a channel is dropped on open no longer brings it
  back: its response predates the drop.
- Resolving the file waits at most 3 s and treats failure as no saved
  copy, as the image cache does.

Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
Signed-off-by: Tom Brow <tomb@squareup.com>
Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
Signed-off-by: Tom Brow <tomb@squareup.com>
@brow
brow marked this pull request as ready for review November 15, 2026 02:35
@brow
brow requested a review from a team as a code owner November 15, 2026 02:35

This branch was successfully deployed

No deployments
codex-review — e3a4400d Deployed Nov 15, 2026 by brow via Run Codex Security Review #15223
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants