Skip to content

feat: harden recursive File provider traversal #104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stone16 merged 6 commits into from
Jul 26, 2026
Merged

feat: harden recursive File provider traversal #104

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c9d0ed6
supply: support recursive File roots (#99)
stone16 Jul 26, 2026
7d579af
test: prove configured File byte ceiling (#99)
stone16 Jul 26, 2026
6742f0d
chore: satisfy recursive File lint (#99)
stone16 Jul 26, 2026
807e75a
fix: preserve recursive File migration fences (#99)
stone16 Jul 26, 2026
5cb0e5e
fix: close recursive File review findings (#99)
stone16 Jul 26, 2026
b859343
fix: revalidate the recursive File snapshot globally (#99)
stone16 Jul 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,11 @@ server-fixed one-second interval when there is no work, and exits on `SIGTERM`
or `SIGINT`.

All dispatch modes read **only** a role-specific scheduler, worker URL,
WorkerLease signing key, and the server-side JSON root registry
(`CONTEXT_ENGINE_WORKER_FILE_ROOTS_JSON`). **A caller may not supply
Organization, Source, job, or token** — that is the point of the boundary.
WorkerLease signing key, the server-side JSON root registry
(`CONTEXT_ENGINE_WORKER_FILE_ROOTS_JSON`), and an optional bounded per-file byte
ceiling (`CONTEXT_ENGINE_WORKER_MAX_FILE_BYTES`, default 1 MiB, accepted only
within 1–64 MiB). Markdown files are discovered recursively. **A caller may not
supply Organization, Source, job, or token** — that is the point of the boundary.
Output is limited to `dispatched` / `no_work` / `refused`.

Lease validation uses the worker's PostgreSQL clock, staying in the same time
Expand Down
13 changes: 13 additions & 0 deletions STATUS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,19 @@ Follow the ADR for its exact evidence boundary.
| [0058](./docs/decisions/0058-schedule-only-upserts-from-mixed-file-pages.md) | Schedule only upserts from mixed File pages |
| [0059](./docs/decisions/0059-dispatch-scheduled-file-imports-through-exact-leases.md) | Dispatch scheduled File imports through exact leases |
| [0060](./docs/decisions/0060-reclaim-expired-file-imports-with-bounded-retries.md) | Reclaim expired File imports with bounded retries |
| [0065](./docs/decisions/0065-recurse-file-discovery-with-anchored-descriptors.md) | Recurse File discovery through anchored descriptors under one bounded byte ceiling |

ADR-0065 extends the active File Provider boundary from a flat root to
deterministic recursive discovery of canonical nested Markdown paths. Each
directory hop and final read stays anchored to the registered root with
no-follow descriptors and stable before/after identity checks; symlinks,
non-regular targets, path escapes, unstable scans, and roots beyond the retained
10,000-path baseline bound fail closed. Worker reads use a server-owned bounded
byte ceiling (1 MiB by default, configurable only within 1–64 MiB), and the real
PostgreSQL evidence covers nested publication plus mixed flat/nested replay.

This does **not** activate provider polling/watchers, a full-resync mechanism,
new delete authority, or any non-Markdown carrier.

### Wire contract, SDK, and trusted delivery

Expand Down
Loading
Loading