Skip to content

fix(database): one row per file β€” unique (path, file_owner) index on filelist - #18001

Open
PeterPablo wants to merge 1 commit into
nextcloud:masterfrom
PeterPablo:fix/filelist-unique-path
Open

PeterPablo wants to merge 1 commit into
nextcloud:masterfrom
PeterPablo:fix/filelist-unique-path

Conversation

@PeterPablo

Copy link
Copy Markdown

What this changes

filelist has no unique index, and the query behind the file list (SELECT … FROM filelist WHERE parent = ?) has neither an owner filter nor a DISTINCT. A second row for one path is therefore accepted silently, and both rows are then served to whoever asks β€” which a Storage Access Framework consumer shows as the same folder twice. In #17051 the duplicates were traced through the provider cursor: the client maps that cursor 1:1, so the doubling cannot be created client-side.

A lookup in code cannot make that invariant hold β€” and it does not help anyone whose duplicates are already in the table, where clearing app data was the only cure. So the database enforces it.

Migration 106 β†’ 107

  • collapses the existing duplicates, keeping the oldest _id per (path, file_owner); rows without a path are left alone (SQLite counts NULLs as distinct, and they have no identity to compare)
  • then creates UNIQUE INDEX idx_filelist_path_owner ON filelist (path, file_owner)

FileEntity declares the same index. Room builds a fresh database from the entity, so without that declaration every new install would start out unprotected, whatever the migration does.

upsertSingleFile falls back to the stored row when its insert is refused by the index, instead of returning a URI with -1 in it. The existing look-up-then-update path is unchanged, and rows with a NULL path behave exactly as before.

Why a constraint and not just a code fix

master already looks the row up before writing (upsertSingleFile, from #17941), which narrows the window β€” but two writers can still pass the lookup, and, more importantly, nothing repairs a table that already holds the duplicates. That gap is the one the reporters are sitting in.

Verification

:app:compileGenericDebugKotlin success
generated app/schemas/…/107.json version 107, idx_filelist_path_owner, unique: true
Migration106to107Test (androidTest, API 36 emulator) seeds three twins, one row of another account and two path-less rows, runs Room's runMigrationsAndValidate(…, 107, …), asserts: two rows for the parent, both path-less rows kept, the oldest _id survives, and a further twin insert is refused
existing FileContentProviderTests 12/12 still green
:app:testGenericDebugUnitTest 634 tests, 0 failures
detekt, spotlessKotlinCheck clean

Worth a maintainer's eye

  • FileDao.upsert is a Room @Upsert whose fallback updates by primary key, so a write that collides on the new index returns -1 instead of raising. Harmless β€” the duplicate state it needs is what the migration removes β€” but it could look up by (path, file_owner) as well.
  • FileContentProvider.update() uses SQLiteDatabase.CONFLICT_REPLACE; an UPDATE that moves a row onto a path already stored can now delete the conflicting row instead of raising. In the duplicate state that is the outcome we want, but it is a behaviour change and deserves a look.
  • The "shows no content at all" symptom in Navigating files/folders through other apps produces double listings or shows no content at all.Β #17051 is most likely the same doubling (a picker oscillating between two rows for one folder); a reporter should confirm once this lands.

Fixes #17051

#AI-ASSISTED

@PeterPablo
PeterPablo marked this pull request as ready for review November 9, 2026 20:58
`filelist` had no unique index, and the query that feeds the file list
(`WHERE parent = ?`) has neither an owner filter nor a DISTINCT. Every write
path used an INSERT with some CONFLICT_* clause, which had nothing to conflict
with, so a second row for the same path was accepted silently β€” and then both
rows were handed out. A SAF consumer shows that as the same folder twice.

Observed with two different SAF clients on the same tree (the file manager
Twig and Material Files): duplicate listings in one, a truncated or empty
listing in the other, and a trace of the duplicates through the provider
cursor, which maps 1:1 onto the rows.

A code-side lookup cannot make the invariant hold, and β€” unlike a code fix β€”
it does not help anybody who already has the duplicates, short of clearing the
app data. So make it an invariant of the database:

* Migration 106 β†’ 107 collapses the twins (oldest `_id` per path/owner wins,
  rows without a path are left alone) and then creates the unique index.
* The same index is declared on `FileEntity`: Room builds a fresh database from
  the entity, so the migration alone would leave new installs unprotected.
* `upsertSingleFile` now falls back to the stored row when its insert is
  refused, instead of returning a URI with `-1` in it.

Verified on an Android 16 emulator: the migration test seeds three twins, a row
of another account and two path-less rows, runs Room's
`runMigrationsAndValidate(…, 107, …)` and asserts the outcome; the existing
`FileContentProviderTests` (12 tests) and the JVM unit tests (634) stay green.

Assisted-by: GitHub Copilot (Copilot SDK in VS Code)
Fixes: nextcloud#17051
Signed-off-by: PeterPablo <4340267+PeterPablo@users.noreply.github.com>
@PeterPablo
PeterPablo force-pushed the fix/filelist-unique-path branch from 99203dd to 1284892 Compare November 9, 2026 20:59
@PeterPablo

Copy link
Copy Markdown
Author

disclaimer: lot of LLM at play and first time for me touching on this nextcloud/android code base. I will conduct the test that made me first comment on #17051. anyhow, even if you do not want this kind of AI generated PR, it might give ideas on where the issue that multiple people commented on originates.

@PeterPablo

Copy link
Copy Markdown
Author

Test report from a device β€” the symptom is gone

Setup: the qa flavour built from this branch, installed side by side with the current store app (com.nextcloud.android.qa, org.nextcloud.qa.*), so both providers are live on one phone and the operational install stays untouched. Same account, same server, same files.

  • Fresh database from this build: PRAGMA user_version = 107 and sqlite_master holds CREATE UNIQUE INDEX idx_filelist_path_owner ON filelist (path, file_owner) β€” read back by pulling the file off the device, not inferred from the code.
  • The listing that was reported in Navigating files/folders through other apps produces double listings or shows no content at all.Β #17051: the new tree added in the file manager (Twig) as an external source, with the folder granted (the folder itself, not the account root). Result: one entry per file, no duplicates. The store app's tree on the same phone still shows them β€” that side-by-side comparison is the interesting part.
  • The repair path, checked separately with this APK: a database made to look like an affected install β€” user_version 106, four rows sharing one path/file_owner, no index β€” was opened by the app and read back as one row, version 107, index present. One provider query was enough; launching the app alone does not open the database when no account is configured.

Caveat, because it is easy to over-read the first two bullets: that device test is a fresh install, so it shows that the duplicates are neither produced nor rendered. The repair of an existing table is the third bullet, and that one ran on an emulator rather than on a phone.

#AI-ASSISTED

@PeterPablo

PeterPablo commented Nov 9, 2026 •

Copy link
Copy Markdown
Author

Second SAF consumer β€” and it is the one from this issue's reproduction steps

Both clients below are unchanged binaries; the only variable is the Nextcloud build, with the patched one installed beside the store app on the same phone, same account, same files.

  • Twig β€” the client whose 1:1 cursor mapping made the doubling visible in this thread: tree added as an external source, folder granted, one entry per file.
  • Material Files β€” named in this issue's own reproduction steps ("Material Files added as External Storage") β€” added as external storage with the account root granted: the tree lists each entry once, the document opens in an editor, one line was added and saved back into the file. With the store app the very same grant doubles every folder and fails to open the folder (an InterruptedException on the first child).

The root grant matters: that is the configuration that used to fail hardest, and Material Files was the evidence this issue rested on. Same app, same phone, only the provider different β€” which is about as clean an A/B as this defect allows. Two independent consumers, one of them the reporting one, and nothing to work around any more.

#AI-ASSISTED

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Navigating files/folders through other apps produces double listings or shows no content at all.

1 participant