Skip to content

feat(sources): #344 LocalDirectorySourceAdapter — capture decisions beyond the IDE#347

Merged
Knapp-Kevin merged 1 commit into
devfrom
feat/344-local-directory-source-adapter
May 15, 2026
Merged

feat(sources): #344 LocalDirectorySourceAdapter — capture decisions beyond the IDE#347
Knapp-Kevin merged 1 commit into
devfrom
feat/344-local-directory-source-adapter

Conversation

@Knapp-Kevin

Copy link
Copy Markdown
Collaborator

Summary

Closes #344 (P2 — planning/brainstorming workflows weren't being auto-captured because SessionEnd hooks only fire during IDE sessions). Partial step on BicameralAI/bicameral-daemon#3 (P3 umbrella — multi-source decision capture pipeline); adds a third adapter to events/sources/ after Granola.

Workflow

Operator drops a planning/brainstorm output into a configured directory (e.g. ~/.bicameral/captured-notes/2026-05-14-auth-design.md); next bicameral-mcp sync-and-brief ingests it. No SessionEnd hook required, no IDE coupling, no remote auth flow.

sources:
  - type: local_directory
    path: ~/.bicameral/captured-notes
    # extensions: [.md, .txt, .json]   # defaults shown
    # source_type_label: planning        # default; override e.g. design-doc
    # max_file_bytes: 1048576            # 1 MiB default

Design (per plan-344 + qor-judge PASS at L1)

  • Implements existing SourceAdapter protocol; reuses existing two-phase-commit plumbing in cli/sync_and_brief_cli.py::_run_source.
  • Non-recursive iterdir(); subdirs ignored; hidden files (.-prefix) ignored; symlinks inside the dir not followed.
  • Extension allow-list defaults to [.md, .txt, .json]; configurable per source.
  • File-size cap mirrors context._DEFAULT_INGEST_MAX_BYTES (1 MiB). Oversized files skipped with stderr warning; mtime not added to watermark candidates so future runs retry after shrinkage.
  • Watermark stores max ISO 8601 mtime; advances only after confirm_watermark() (two-phase commit). Corrupt watermark treated as missing (mirrors Granola precedent).
  • Graceful empty-return on config errors (missing path, not a dir, unreadable); never raises to the CLI caller (matches the _run_source discipline).

Test plan

  • 21 new sociable unit tests against real filesystem via tmp_path — no mocks. Cover registry conformance, protocol conformance, config errors, watermark filtering, extension filter (default + custom), hidden files, subdirs, oversized files, confirm advance, no-items-no-file, source_type_label override, corrupt watermark, stable span_id from path sha256, parametrized content preservation
  • 11 Granola unit tests still pass (regression)
  • tests/test_sync_and_brief_cli.py all pass (CLI integration unchanged)
  • ruff format + ruff check clean
  • No new dependency

Docs

  • docs/policies/sources-config.md gets a new "local_directory source" section with config example, behavior summary, workflow example, and what-this-doesn't-do callouts
  • Source-type table updated to include local_directory
  • Future-source roadmap entry flipped from "P2 follow-up" → "shipped"

Scope honesty

Plan

  • plan-344-local-directory-source-adapter.md (qor-judge PASS at L1 with 3 advisory clarifications, all folded into the implementation)

🤖 Generated with Claude Code

…eyond the IDE

Closes #344 (planning/brainstorming workflows weren't being auto-captured
because SessionEnd hooks only fire during IDE sessions). Partial step
on #337 (multi-source decision capture pipeline) — adds a third adapter
to the events/sources/ registry after Granola.

Workflow: operator drops a planning/brainstorm output into a configured
directory (e.g. ~/.bicameral/captured-notes/); next bicameral-mcp
sync-and-brief ingests it. No SessionEnd hook required, no IDE coupling,
no remote auth flow.

Design (per plan-344 + qor-judge PASS at L1 + audit advisories A1-A3):
- Implements SourceAdapter protocol; reuses existing two-phase-commit
  plumbing in cli/sync_and_brief_cli.py::_run_source.
- Non-recursive iterdir(); subdirs ignored; hidden files (.-prefix)
  ignored; symlinks inside the dir not followed.
- Extension allow-list defaults to [.md, .txt, .json]; configurable.
- File-size cap mirrors context._DEFAULT_INGEST_MAX_BYTES (1 MiB).
  Oversized files skipped with stderr warning; mtime NOT added to
  watermark candidates so future runs retry after shrinkage.
- Watermark stores max ISO 8601 mtime; advances only after confirm()
  (two-phase commit). Corrupt watermark treated as missing.
- Graceful empty-return on config errors (missing path, not a dir,
  unreadable); never raises to the CLI caller.

Tests (21 new sociable unit tests against real filesystem via tmp_path,
no mocks): registry conformance, protocol conformance, config error
handling (missing/nonexistent/file-not-dir), happy-path pull, watermark
filtering, extension filter (default + custom), hidden files, subdirs,
oversized files (with correct watermark behavior), confirm advances to
max mtime, no-new-items-no-watermark-file, source_type_label override
(operator can tag content), corrupt watermark, stable span_id from path
sha256, parametrized content preservation across multi-line/emoji/markdown.

Regression: 11 Granola unit tests still pass; tests/test_sync_and_brief_cli.py
all pass (CLI integration unchanged).

Docs: docs/policies/sources-config.md gets a new "local_directory source"
section + updated source-type table + roadmap entry flipped from
"P2 follow-up" to "shipped".

Scope honesty: closes #344 fully; partial step on #337 (the Slack /
PR review / Linear / git-commit-convention / transcript-dir adapters
are separate sub-cycles).

Plan: plan-344-local-directory-source-adapter.md (qor-judge PASS at L1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4bb3482b-3760-4404-8193-6976dc9078e9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/344-local-directory-source-adapter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Knapp-Kevin Knapp-Kevin added feat Feature work or user-visible capability P2 Medium: next milestone or two; default for new issues post-triage tool MCP tool or handler surface skill Skill instructions or workflow guidance surface labels May 15, 2026
@Knapp-Kevin Knapp-Kevin merged commit 747e3e9 into dev May 15, 2026
6 checks passed
@Knapp-Kevin Knapp-Kevin deleted the feat/344-local-directory-source-adapter branch May 15, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat Feature work or user-visible capability P2 Medium: next milestone or two; default for new issues post-triage skill Skill instructions or workflow guidance surface tool MCP tool or handler surface

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant