Repository navigation
fix(database): one row per file β unique (path, file_owner) index on filelist - #18001
PeterPablo wants to merge 1 commit into
Conversation
`filelist` had no unique index, and the query that feeds the file list (`WHERE parent = ?`) has neither an owner filter nor a DISTINCT. Every write path used an INSERT with some CONFLICT_* clause, which had nothing to conflict with, so a second row for the same path was accepted silently β and then both rows were handed out. A SAF consumer shows that as the same folder twice. Observed with two different SAF clients on the same tree (the file manager Twig and Material Files): duplicate listings in one, a truncated or empty listing in the other, and a trace of the duplicates through the provider cursor, which maps 1:1 onto the rows. A code-side lookup cannot make the invariant hold, and β unlike a code fix β it does not help anybody who already has the duplicates, short of clearing the app data. So make it an invariant of the database: * Migration 106 β 107 collapses the twins (oldest `_id` per path/owner wins, rows without a path are left alone) and then creates the unique index. * The same index is declared on `FileEntity`: Room builds a fresh database from the entity, so the migration alone would leave new installs unprotected. * `upsertSingleFile` now falls back to the stored row when its insert is refused, instead of returning a URI with `-1` in it. Verified on an Android 16 emulator: the migration test seeds three twins, a row of another account and two path-less rows, runs Room's `runMigrationsAndValidate(β¦, 107, β¦)` and asserts the outcome; the existing `FileContentProviderTests` (12 tests) and the JVM unit tests (634) stay green. Assisted-by: GitHub Copilot (Copilot SDK in VS Code) Fixes: nextcloud#17051 Signed-off-by: PeterPablo <4340267+PeterPablo@users.noreply.github.com>
99203dd to
1284892
Compare
|
disclaimer: lot of LLM at play and first time for me touching on this nextcloud/android code base. I will conduct the test that made me first comment on #17051. anyhow, even if you do not want this kind of AI generated PR, it might give ideas on where the issue that multiple people commented on originates. |
Test report from a device β the symptom is goneSetup: the
Caveat, because it is easy to over-read the first two bullets: that device test is a fresh install, so it shows that the duplicates are neither produced nor rendered. The repair of an existing table is the third bullet, and that one ran on an emulator rather than on a phone. #AI-ASSISTED |
Second SAF consumer β and it is the one from this issue's reproduction stepsBoth clients below are unchanged binaries; the only variable is the Nextcloud build, with the patched one installed beside the store app on the same phone, same account, same files.
The root grant matters: that is the configuration that used to fail hardest, and Material Files was the evidence this issue rested on. Same app, same phone, only the provider different β which is about as clean an A/B as this defect allows. Two independent consumers, one of them the reporting one, and nothing to work around any more. #AI-ASSISTED |
What this changes
filelisthas no unique index, and the query behind the file list (SELECT β¦ FROM filelist WHERE parent = ?) has neither an owner filter nor aDISTINCT. A second row for one path is therefore accepted silently, and both rows are then served to whoever asks β which a Storage Access Framework consumer shows as the same folder twice. In #17051 the duplicates were traced through the provider cursor: the client maps that cursor 1:1, so the doubling cannot be created client-side.A lookup in code cannot make that invariant hold β and it does not help anyone whose duplicates are already in the table, where clearing app data was the only cure. So the database enforces it.
Migration 106 β 107
_idper(path, file_owner); rows without a path are left alone (SQLite counts NULLs as distinct, and they have no identity to compare)UNIQUE INDEX idx_filelist_path_owner ON filelist (path, file_owner)FileEntitydeclares the same index. Room builds a fresh database from the entity, so without that declaration every new install would start out unprotected, whatever the migration does.upsertSingleFilefalls back to the stored row when its insert is refused by the index, instead of returning a URI with-1in it. The existing look-up-then-update path is unchanged, and rows with aNULLpath behave exactly as before.Why a constraint and not just a code fix
masteralready looks the row up before writing (upsertSingleFile, from #17941), which narrows the window β but two writers can still pass the lookup, and, more importantly, nothing repairs a table that already holds the duplicates. That gap is the one the reporters are sitting in.Verification
:app:compileGenericDebugKotlinapp/schemas/β¦/107.jsonidx_filelist_path_owner,unique: trueMigration106to107Test(androidTest, API 36 emulator)runMigrationsAndValidate(β¦, 107, β¦), asserts: two rows for the parent, both path-less rows kept, the oldest_idsurvives, and a further twin insert is refusedFileContentProviderTests:app:testGenericDebugUnitTestdetekt,spotlessKotlinCheckWorth a maintainer's eye
FileDao.upsertis a Room@Upsertwhose fallback updates by primary key, so a write that collides on the new index returns-1instead of raising. Harmless β the duplicate state it needs is what the migration removes β but it could look up by(path, file_owner)as well.FileContentProvider.update()usesSQLiteDatabase.CONFLICT_REPLACE; anUPDATEthat moves a row onto a path already stored can now delete the conflicting row instead of raising. In the duplicate state that is the outcome we want, but it is a behaviour change and deserves a look.Fixes #17051
#AI-ASSISTED