Repository navigation
fix: consolidate follow-up application issue repairs - #4776
ScriptedAlchemy wants to merge 84 commits into
Conversation
Refs #4535. Read at most 1 MiB plus one byte from the selected endpoint. Refuse oversized advertised responses and streaming responses without a length before parsing untrusted JSON, as well as malformed Content-Length. Avoid allocating an unbounded remote response. Cover all three refusal paths against a real loopback HTTP server. The isolated transport probe passed; hosted Bazel tests are still needed.
|
Refs #4535. The active /v1/systemone answer contract carries a type tag. Previously an untyped answer with a noul field was accepted as a successful prediction, obscuring protocol drift during model qualification. Refuse missing or mismatched tags before persisting rows or raw samples. Update test server doubles and cover missing-tag denial with partial-log preservation.
Refs #4535. Reject duplicate object keys at every JSON depth before an inference result can enter the resumable evidence log or sample. The loopback regression covers both duplicate answer maps and duplicate Noul values, followed by a successful valid resume.
Refs #4535. Reject duplicate object fields and non-JSON numeric constants in dataset and server manifests, partial resumes, and scored results. Preserve completed summaries and resumable evidence on refusal.
Refuse stale or swapped raw samples on new runs and resumed partials. Record the raw response digest in completed rows; verify sampled experiment, case, request and response before further inference. Keep one capture for the first configuration and reject competing publication without replacing foreign evidence. Refs #4535
Refuse oversized partial headers, rows and raw sample captures before unbounded JSON reads, and preserve resumable evidence unchanged on denial. Add behavioral regressions for oversized partial and sample files followed by successful exact resumption. Refs #4535
Use nearest-rank selection so sparse runs cannot understate their slowest observed decision at p90/p95. Include direct metric and CLI regressions without extrapolating unmeasured latencies. Refs #4535
Preserve resumable evidence and first-response captures when a model replies with non-object timing metadata; reject malformed subset containers before network inference or partial-file creation. Add behavioral recovery tests. Refs #4535
A partial run can already contain the first response when --sample-out is added on retry. Refuse rather than label a later response as the experiment's first raw evidence. Preserve durable partial rows and permit an ordinary resume without a capture. Add a behavioral interruption regression. Refs #4535
Refs #4535 Reject sample destinations under results before inference. Otherwise a raw sample may be misread as a completed run or occupy its final path. Exercise direct, normalized, nested collisions and a valid external sample.
Refuse finite-looking JSON exponents that Python decodes to infinity in untrusted datasets, resumable rows and scored result files. Preserve the last durable evidence/summary instead of admitting nonfinite extra data. Regression coverage includes pre-network dataset rejection, read-only resume rejection and preservation of a previously scored summary. Refs #4535
A removed first-response sample must not be silently recreated from a later case if it disappears while the second HTTP inference is in flight. Preserve the settled first row and allow recovery without a raw sample. Refs #4535
Removing the Plan 38 reference left a target reading as a current fact. `Storage::maintain` does not exist in the code, and an over-budget store only produces a Doctor finding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ge-design-work docs: remove stale code storage design passages
Retain fail-closed Codex executable admission while making scheduler/manual and lock-order tests use an actual configured executable identity. Add a separate regression for the typed unconfigured-backend diagnostic. Resolve three automation_runner_test failures observed on master CI.
Retain first-response evidence during malformed resume tests, supply realistic HTTP headers for the wire fingerprint test, and invoke the correctly named interrupted-capture regression callback. Integrate the already merged lucide-react lockfile update.
Reject resumed HTTP rows and scored experiment files that omit or malform their recorded raw response digest. An incomplete evidence row must not be silently treated as a verified model observation. Cover rejection before reinference, scored-summary preservation and exact recovery once the durable row is restored. Refs #4535
|
Master-merge hold under the owner's latest instruction: “Merge any passing pr after you review and fix them. Don’t merge the decision model ones like kev.” This integration still contains decision-model/HTTP-evaluation changes (including current head 9cf67ec), so it is excluded from merging into master even if CI becomes green. Review and corrective commits remain authorized. Preserve all contributor work; do not force-rewrite, remove excluded changes, reopen superseded source PRs, or create a competing application lane to bypass this hold. Both existing TraceDecay maintenance tasks now carry this exclusion with their schedules unchanged. Ready-for-review status is not merge authorization. No merge or code change was performed by this review pass. |
Count each unlinked SQLite TEMP physical file once when multiple Unix file descriptors refer to the same inode. Preserve unreadable-entry coverage and other platforms' behavior. Regress duplicate aliases that otherwise produce a false over-budget doctor finding. Refs #4152
Ignore nonregular entries if a descriptor is reused between resolving its (deleted) pathname and sampling metadata. Keep existing inode deduplication and unreadable-entry reporting. Regression covers a directory with a matching TEMP basename without changing the valid spill total. Refs #4152
Avoid opening and closing independent Unix file descriptors for verified read snapshots: their close releases unrelated POSIX SQLite record locks. Verify the canonical inode on both sides of SQLite HAS_MOVED and reject unsupported VFS proof without a raw pin. Preserve Windows pinned handles. Exercise rollback and WAL writer locks against an external process while live and immutable readers open and close, then verify committed data. Refs #4513
Require exactly one SQLite database descriptor for a verified Unix reader; keep clone and final-drop assertions while rejecting the obsolete raw-pin contract. Format the fail-closed HAS_MOVED refusal as required by rustfmt. Refs #4513
Join the reviewed #4821 and #4827 prerequisite ancestry without changing the existing #4798 tree. Both prerequisite file blobs already match this head exactly. Preserve all retention source, behavioral tests, and concurrent history while allowing a focused six-file retention review above the prerequisites. Do not merge into the prerequisite branch. Land prerequisites in master, retarget, revalidate, and retain the issue acceptance hold. No changes from #4813, held #4776, or decision-model work are moved. Refs #4787
Refs #4535 Bind a per-call transport timeout to the held-out manifest and HTTP experiment identity. Fail closed on an altered deadline before inference or independent scoring; retain historical defaults. Add falsifiable positive and forged-evidence regression tests. Keep #4797 and #4776 DRAFT; never merge.
Refs #4535 Refuse semantically equivalent but unproducible Rust wire encodings, even when captures and experiment digests are coherently rehashed. Exercise direct validation, zero-network runner refusal and independent scoring rejection for top-level order and whitespace drift. Keep #4797 and #4776 DRAFT; never merge.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 3 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if sample_path and os.path.lexists(sample_path): | ||
| if created: | ||
| raise ValueError(f"sample capture already exists for a new run: {sample_path}") | ||
| verify_existing_sample(sample_path, header, requests, done, questions) |
There was a problem hiding this comment.
🟡 Interrupted sample blocks exact resume
If the process stops after publish_sample but before writing its row, verify_existing_sample rejects the capture on restart. The same experiment cannot resume with --sample-out without manually removing its own capture.
Learn more
The first raw response is published separately from its partial-log row. A process can stop between those writes, leaving a correctly identified sample and a header-only partial log. On resume, verify_existing_sample requires the first case in done, so it rejects the sample before another response can be requested. The former runner could resume after this interruption; the new validation makes its own crash state unrecoverable with the same capture option.
Example: A two-case run writes first-response.json, then exits before appending case first to .partial. Restarting the identical command finds zero completed cases and fails before HTTP, although the capture names this experiment and its first request.
Recommended fix: Make first-sample publication and first-row recovery an explicit transactional state. On resume, validate the sample identity and request when the first row is absent, then either replay that response into a validated row or recover by requesting the first case while verifying the existing sample's response digest. Keep the refusal for samples that identify a different run.
Was this helpful? React with 👍 or 👎 to provide feedback.
| continue; | ||
| } | ||
| let name = entry.file_name(); | ||
| let child = directory.open_dir_nofollow(&name)?; |
There was a problem hiding this comment.
🟡 Seal publication can abort storage census
When a seal renames a listed staging directory, open_dir_nofollow returns NotFound and aborts the census. Doctor loses measurements for the remaining sealed generations during that pass.
Learn more
The census lists child names before opening each directory. A seal publishes its artifact by renaming its staging directory to the generation directory. If that rename happens between the file_type check and this open, this open returns NotFound; the ? propagates it and discards all census results. The Doctor caller treats a failed census as unavailable.
Example: Doctor lists .staging-<digest>, the publisher renames it to <digest>, and Doctor then tries to open .staging-<digest>. The whole census fails despite other sealed generations remaining readable.
Recommended fix: Handle NotFound for a child that vanished between enumeration and open by skipping that entry. Keep other open failures as unavailable, and cover a deterministic rename between listing and opening in a focused test.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| /// Atomically replaces a single file beneath a directory held by its owner. | ||
| pub fn replace_file_in_open_directory(directory: &Dir, name: &str, bytes: &[u8]) -> io::Result<()> { | ||
| use std::io::Write; |
There was a problem hiding this comment.
🟡 Production helper uses a forbidden inline import
The new replace_file_in_open_directory helper imports Write inside its body, violating the repository's top-level import rule.
Learn more
Move Write to the module-level std::io import and remove this function-local import. The repository's AGENTS.md requires top-level explicit imports rather than inline imports.
Was this helpful? React with 👍 or 👎 to provide feedback.
Use the actual published runtime for the admission baseline and retire it through the exact registry before reopening the store. Join physical workers off the async executor and assert closed retirement, preserving every sampled cancellation, rollback, and successful recovery assertion.
|
Closing without merge at the maintainer’s request to stop PRs that implement behavior using LLM/decision models. This mixed integration contains Kev/decision-model evaluation work. Closure does not claim its unrelated storage/runtime issues are fixed; those remain with their dedicated PRs. |
Draft · Owner merge hold
This integration includes decision-model changes. Keep it in draft under the owner’s merge hold. Owner direction
After #4803 merged,
ead18018reconciled the SQLite lock regression with master. The native macOS test verified committed row visibility and independent writer exclusion across the four WAL/rollback and snapshot/immutable reader combinations. The owner’s draft and merge hold remain in place.Consolidated application follow-up
The prior application integration #4772 merged into
masteron November 5, 2026. This is the single new application integration PR for subsequent issue fixes. Add focused commits here rather than opening separate issue PRs/CI lanes; preserve concurrent changes with exact-head, nonforce updates.Included now
Content-Lengthbefore reading, and also reject an oversized body with no length header.Other commits in this draft cover bounded projection rebuilds, sealed graph verification and retirement, cold-store cursors, SQLite reader lifetimes, and the corresponding storage documentation.
Verification and gates
At
78c610e1d1, the branch incorporates masterc231fb7219and preserves both scheduler regressions in the previously conflicting fixture. Native macOS Bazel passed 46 scheduler cases, 72 HTTP runner cases, and 29 scoring cases. The HTTP suite includes real loopback responses with a declared over-limit body and an over-limit stream. Full hosted Linux run.At
6c0cf6f77f, the branch also incorporates masterbe2e1349c6and resolves the overlapping projection test imports. All 502 native macOS global database cases passed, including populated projection cleanup, foreign ownership, staged outputs and rollback. Strict Clippy passed for both library and tests. Full hosted Linux and targeted Windows verification.On the same head all 12 cold-store tests passed, including the production cursor run with an admitted owner, a substituted symlink and a real-directory replacement. The foreign cursor bytes and directory entries survived both refusals.
This is partial progress on #4535: canonical Scout evidence export, independent labels, actual Kev inference, runtime parity and model qualification are still outstanding. Keep #4535 open.
Independent model evidence and review remain outstanding.