Priority C v0 — self-managing team-server, Slack-first ingest#153
Closed
Knapp-Kevin wants to merge 31 commits into
Closed
Priority C v0 — self-managing team-server, Slack-first ingest#153Knapp-Kevin wants to merge 31 commits into
Knapp-Kevin wants to merge 31 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: Turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. 👉 Get your free trial and get 200 agent minutes per Slack user (a $50 value). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Comment on lines
+50
to
+55
| events: list[dict] = resp.json() | ||
| except (httpx.HTTPError, ValueError) as exc: | ||
| logger.warning("team-server pull failed: %s", exc) | ||
| return [] | ||
| if events: | ||
| last_seq = max(int(e.get("sequence", since)) for e in events) |
Contributor
There was a problem hiding this comment.
🔴 Non-200 HTTP responses cause unhandled exception in failure-isolated pull function
pull_team_server_events documents a "never raises" contract (line 41) for failure-isolation, but when the team-server returns a non-200 response (e.g., 500 with {"detail": "Internal Server Error"}), resp.json() at line 50 succeeds and returns a dict. The code exits the try/except block normally, then at line 55 iterates over the dict's string keys. Calling .get("sequence", since) on a string key raises AttributeError, which is not caught. Confirmed by local reproduction: iterating a dict like {"detail": "..."} and calling .get() on each key triggers AttributeError: 'str' object has no attribute 'get'. This violates the explicit failure-isolation invariant that this function never raises, and could cascade into the deterministic retrieval/status path.
Suggested change
| events: list[dict] = resp.json() | |
| except (httpx.HTTPError, ValueError) as exc: | |
| logger.warning("team-server pull failed: %s", exc) | |
| return [] | |
| if events: | |
| last_seq = max(int(e.get("sequence", since)) for e in events) | |
| events: list[dict] = resp.json() | |
| except (httpx.HTTPError, ValueError, TypeError) as exc: | |
| logger.warning("team-server pull failed: %s", exc) | |
| return [] | |
| if not isinstance(events, list): | |
| logger.warning("team-server pull returned non-list: %s", type(events).__name__) | |
| return [] | |
| if events: | |
| last_seq = max(int(e.get("sequence", since)) for e in events) |
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+66
to
+78
| cache_existed_before = await _cache_row_exists( | ||
| db_client, "slack", source_ref, content_hash | ||
| ) | ||
| extraction = await get_or_compute( | ||
| db_client, | ||
| source_type="slack", | ||
| source_ref=source_ref, | ||
| content_hash=content_hash, | ||
| compute_fn=lambda: extractor(text), | ||
| model_version=INTERIM_MODEL_VERSION, | ||
| ) | ||
| if cache_existed_before: | ||
| return # idempotent — already ingested |
Contributor
There was a problem hiding this comment.
🟡 Idempotency check based on extraction cache can permanently lose events if team_event write fails
_ingest_message checks whether the extraction cache row exists before calling get_or_compute, and uses that boolean to decide whether to skip the write_team_event call (line 77-78). If get_or_compute succeeds (creating the cache row at team_server/extraction/canonical_cache.py:39-44) but write_team_event subsequently fails (e.g., transient DB issue), the exception propagates. On the next poll_once invocation for the same Slack message, _cache_row_exists returns True, so the function returns early at line 78 without writing the team_event. The event is permanently lost — there is no retry path since the cache hit signals "already ingested" even though the event was never written.
Prompt for agents
The idempotency logic in _ingest_message uses the extraction cache row as a proxy for whether the full ingest (cache + team_event) completed successfully. But these are two separate, non-atomic DB writes. If get_or_compute succeeds (populating the cache) and write_team_event then fails, subsequent calls see the cache hit and skip the message entirely — permanently losing the event.
Possible fixes:
1. Check for the team_event row's existence instead of (or in addition to) the extraction cache when deciding to skip. E.g., query team_event for a row with matching source_ref.
2. Move the dedup check to after write_team_event using a UNIQUE index on team_event (source_type, source_ref, content_hash) so duplicate writes are rejected at the DB level rather than prevented in application code.
3. Separate the extraction cache from the dedup mechanism entirely — the cache is for computation memoization, the team_event existence (or a separate processed-messages table) is for delivery tracking.
Was this helpful? React with 👍 or 👎 to provide feedback.
Knapp-Kevin
had a problem deploying
to
recording-approval
GitHub Actions
Failure
Priority C v0 — Phase 1 of the team-server vertical-slice for multi-dev decision continuity at organizational scale (per the Sales Enablement & Positioning Playbook + research-brief-priority-c-selective-ingest-2026-05-02.md). The team-server is a self-managing, customer-self-hosted Python service. Per CONCEPT.md anti-goals under literal-keyword parsing (SHADOW_GENOME Failure Entry #6 addendum): "no managed backend" forbids vendor SaaS and human-ops-tax architectures, NOT self-managing customer-deployable backends. Sentry self-hosted, Supabase OSS, and the embedded-SurrealDB philosophy already in repo are precedents. Scaffold delivered: - team_server/app.py (47 LOC): FastAPI app factory; lifespan migrates schema on startup, closes DB on teardown; /health endpoint - team_server/schema.py (80 LOC): v0 schema for workspace, channel_allowlist, extraction_cache, team_event tables. Idempotent ensure_schema(); migration dispatch table for future versions. FLEXIBLE TYPE object on canonical_extraction + payload (per #72 lesson + audit Advisory #3). - team_server/db.py (41 LOC): TeamServerDB factory wrapping ledger.client.LedgerClient with team-server's own ns/db pair. - deploy/team-server.docker-compose.yml + Dockerfile.team-server: single-service compose; volume for persistent data; healthcheck on /health; runs as non-root. Tests (6 functionality tests, all green): - tests/test_team_server_app.py: app starts + serves health, schema migrates from empty + idempotent, lifespan teardown closes DB, /health returns well-formed JSON. - tests/test_team_server_deploy.py: docker-compose config validates. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 of plan-priority-c-team-server-slack-v0.md — adds Slack OAuth v2 flow, workspace persistence with at-rest encrypted tokens, and a schema-validated channel allow-list config loader. Per audit Advisory #2, OAuth routes are factored into team_server/auth/router.py (parallel to Phase 4's events router pattern) rather than left inline in app.py — keeps app.py at 47 lines well under the Section 4 razor cap. Files added: - team_server/auth/slack_oauth.py (58 LOC): pure functions — build_authorize_url, exchange_code; raises SlackOAuthError on ok=false. - team_server/auth/encryption.py: Fernet encrypt/decrypt for OAuth tokens at rest; key loaded from BICAMERAL_TEAM_SERVER_SECRET_KEY env var. - team_server/auth/router.py (73 LOC): /oauth/slack/install + /oauth/slack/callback routes with CSRF state defense; persists workspace row with token encrypted before storage. - team_server/config.py (40 LOC): pydantic-validated YAML loader for channel allow-list; raises ValueError with descriptive message on schema failure. - team_server/app.py (modified): include auth router. Tests (7 functionality tests, all green): - tests/test_team_server_slack_oauth.py: authorize URL contains required params + scopes, exchange_code POSTs correctly, encrypt/decrypt round-trip preserves plaintext while ciphertext differs, callback rejects mismatched state (CSRF defense), callback persists workspace. - tests/test_team_server_channel_allowlist.py: load_channel_allowlist parses valid YAML, raises ValueError on missing team_id. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3 of plan-priority-c-team-server-slack-v0.md — adds the polling worker, the canonical-extraction cache that closes the multi-dev extraction-divergence gap, and the peer-author event writer. Multi-dev convergence mechanism: any dev's session that touches a Slack message produces the SAME canonical extraction across the team because get_or_compute is keyed on (source_type, source_ref, content_hash) and the cache row is shared via the team-server's append-only event log. This is what closes Playbook Pillar #1 (Decision Continuity) at multi-dev/multi-agent scale that the v1 brief incorrectly framed as "build curation only" — see SHADOW_GENOME Failure Entry #6. The interim canonical extraction uses Anthropic Claude with the model_version='interim-claude-v1' tombstone so a future Phase 5 (CocoIndex #136 integration) can identify and rebuild interim entries when memoized deterministic transforms become available. Files added: - team_server/extraction/canonical_cache.py (45 LOC): get_or_compute() returns cached extraction OR invokes compute_fn + persists. - team_server/extraction/llm_extractor.py: interim Anthropic-backed extractor; production-default until CocoIndex lands. - team_server/sync/peer_writer.py (42 LOC): write_team_event() — appends to team_event with author_email='team-server@<team_id>.bicameral' identity (single-bot per workspace; multi-instance HA is v1). - team_server/workers/slack_worker.py (100 LOC): poll_once() — conversations_history per allowlisted channel, content_hash dedup, cache-keyed extraction, peer event write per new message. Tests (6 functionality tests, all green): - tests/test_team_server_canonical_cache.py: cache hit returns existing without compute_fn, cache miss persists + subsequent call returns cached, content_hash variation produces new rows. - tests/test_team_server_slack_worker.py: worker polls only allowlisted channels, writes one team_event per message with peer-author identity, dedups via message ts (idempotent re-poll). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 of plan-priority-c-team-server-slack-v0.md — closes the multi-dev convergence loop by exposing team_event over HTTP and extending EventMaterializer with a failure-isolated team-server pull. The materializer pull is OUTSIDE the deterministic core (per CONCEPT.md literal-keyword parsing of "no network calls in the deterministic core" — SHADOW_GENOME Failure Entry #6 addendum). Failure-isolation contract: team-server outage NEVER cascades into per-dev preflight failures — events/team_server_pull.py swallows transport errors, returns empty events, leaves the watermark unchanged. Files added: - team_server/api/events.py: GET /events?since=N&limit=M endpoint; reads team_event ordered by sequence ascending; pagination via since cursor. - events/team_server_pull.py (57 LOC): pull_team_server_events() queries the team-server's /events endpoint, persists watermark per call, swallows transport errors gracefully. - team_server/app.py (modified): include events router. Tests (6 functionality tests, all green): - tests/test_team_server_events_api.py: /events returns rows in sequence order, paginates via since cursor, returns empty (not error) when no new events. - tests/test_materializer_team_server_pull.py: materializer pulls from team-server URL, watermark advances + persists separately, 503 /transport-error degrades gracefully (no exception, watermark unchanged) — failure-isolation contract verified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
QorLogic SDLC governance trail for the Priority C v0 implementation that landed in commits 1-4 of this PR. Includes: - docs/research-brief-priority-c-selective-ingest-2026-05-02.md (v3) — research substrate. v1 was rejected for INVARIANT_FROM_IMPLEMENTATION (treating v0 agent-fetches-only code state as product principle); v2 added playbook substrate; v3 narrowed to Slack-first + team-server + CocoIndex-conditional after operator dialogue clarified "no managed backend" = "no human-ops-tax architecture," not "no backend." - plan-priority-c-team-server-slack-v0.md (437 LOC) — the L3 plan with five phases (Phase 5 deferred per "if we can manage it" feasibility caveat). - docs/SHADOW_GENOME.md Failure Entry #6 + addendum — captures the framing-error pattern AND the "anti-goals must be parsed by their load-bearing keyword" lesson; symmetric to v0-code-as-principle. - docs/META_LEDGER.md Entries #27 (IMPLEMENT) + #28 (SEAL). Predecessor: efd0304b (#135-triage seal on dev). Implement chain: 211ffb9e. Substantiation seal: 6f4f8f8f1d63ad82b952a3c6aff270d30584e08b0572077ff685e84ce453f6c2 - docs/SYSTEM_STATE.md — Priority C v0 section appended; documents schema additions, architectural properties achieved, audit advisory disposition, Phase 5 deferred state, and the qor-logic-internal steps skipped for downstream-project rationale. - .agent/staging/AUDIT_REPORT.md — PASS verdict, three non-blocking advisories all addressed at implement-time. Verdict: REALITY = PROMISE for Phases 1-4. Phase 5 (CocoIndex #136) explicitly deferred per plan slip-independence design. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ref + schema_version (Phase 0) Schema v1->v2: extraction_cache.UNIQUE keyed on (source_type, source_ref); content_hash becomes a tracked column. canonical_cache.get_or_compute() replaced by upsert_canonical_extraction(...) -> tuple[dict, bool] returning (extraction, changed). Slack worker adapts to the new contract; gates the team_event write on the returned changed flag (idempotent on no-content change). _MIGRATIONS dispatch upgraded to Callable[[LedgerClient], Awaitable[None]]. New schema_version single-row table records the post- migration version as data, not folklore. _migrate_v1_to_v2 dedups duplicate (source_type, source_ref) rows by max(created_at) before redefining the v2 index. Tests: 12 functionality tests across cache_upsert, schema_migration, and slack_worker adaptation; canonical_cache test rewritten under v2 upsert contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ring (Phase 0.5) Establishes the worker-task lifecycle pattern via worker_loop(name, interval_seconds, work_fn) — single source of truth for the asyncio.create_task / per-iteration error isolation / cancel-on-shutdown shape. slack_runner.run_slack_iteration is the canonical reference implementation: iterates the workspace table, decrypts each Fernet token via load_key_from_env() + decrypt_token(ciphertext, key), reads the channel allowlist, constructs an AsyncWebClient, and delegates one polling pass to slack_worker.poll_once. Per-workspace exceptions are caught for failure isolation. app.py lifespan registers the Slack worker unconditionally (no-op when workspace table is empty); the registered task is cancelled and awaited on shutdown. Closes the v0 dormant-Slack-worker gap: v0 plan claimed an active worker but v0 code shipped poll_once with zero production callers. Tests: 7 functionality tests including the round-trip encryption test (encrypt -> store-as-string -> read-as-bytes -> decrypt -> token reaches slack_client) that closes the audit-round-2 blind spot per SHADOW_GENOME #7 addendum. slack_sdk import in slack_runner is lazy (inside run_slack_iteration) so the team_server package is importable in environments where slack_sdk is declared in requirements.txt but not in dev venv. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
team_server/auth/notion_client.py provides internal-integration-token auth (no OAuth router): load_token resolves NOTION_TOKEN env first, falling back to YAML config's notion.token; raises NotionAuthError if neither is set. Pure async functions over httpx.AsyncClient with Notion-Version pinned to 2022-06-28: list_databases (filtered to object=database), query_database (per-database last_edited_time watermark filter, ascending sort, paginated), fetch_page_blocks (paginated children). team_server/extraction/notion_serializer.py serializes a Notion database row deterministically: title line, then sorted-by-key property lines (title/rich_text/select/multi_select/date/checkbox/number/url/ people branches), then a blank line, then body block plain-text. Byte- stable output is the gating invariant for content_hash stability. team_server/config.py: DEFAULT_CONFIG_PATH constant with BICAMERAL_CONFIG_PATH env-var fallback; Path-typed. Tests: 7 client tests (env-vs-config precedence, MockTransport verification of filter shapes + Notion-Version header pinning + block pagination), 3 serializer tests (ordering, all property-type branches, byte-stability across calls). No new package dependencies — httpx and yaml already in v0 deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…se 2)
team_server/workers/notion_worker.py polls allowlist-via-share Notion
databases (the integration sees only databases the operator has shared
with it — derived dynamically from notion_client.list_databases, no
separate allowlist table required). Per-database watermark stored in
the new source_watermark table, advanced monotonically as rows
ingest. Partial-failure recovery: watermark advances only to the last
successfully-ingested row's last_edited_time, so the next poll resumes
correctly. Per-database HTTPError is caught and logged so a single
failing database does not block other databases.
Each row's text input is the deterministic serializer output (title +
sorted properties + body); content_hash is SHA256 over that text.
upsert_canonical_extraction returns (extraction, changed); when
changed=True, a peer-authored team_event is written under
PEER_WORKSPACE_ID="notion" (resulting author_email
"team-server@notion.bicameral" via write_team_event's wrapper).
source_type="notion_database_row"; source_ref="{db_id}/{page_id}".
Tests: 9 functionality tests covering database iteration via
list_databases, first-seen-row event, idempotency on unchanged rows,
new event on edited rows, monotonic watermark advancement, watermark-
to-filter wiring, partial-failure recovery, per-database 404 isolation,
content_hash stability across dict insertion-order changes (the
serializer determinism invariant under the polling layer).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
team_server/workers/notion_runner.py: thin wrapper run_notion_iteration over notion_worker.poll_once for symmetry with slack_runner (both expose a zero-extra-arg work_fn for the lifespan to register via worker_loop). Internal-integration auth means a single token covers a single workspace; v1 ships single-workspace. team_server/app.py lifespan amended: after Slack worker registration (unconditional), attempts notion_client.load_token via DEFAULT_CONFIG_PATH; on success registers a Notion task via the same worker_loop helper. On NotionAuthError logs INFO and continues without Notion ingest. On shutdown, both tasks are cancelled and awaited symmetrically. Tests: 4 functionality tests covering env-gated startup wiring, off-by-default invariant when token unset, cancellation on shutdown, and inner-loop resilience (single-iteration failure does not exit the loop). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three-round audit cycle (VETO -> VETO -> PASS) for Notion ingest + cache contract migration. Plan ships across five phases: - Phase 0 — cache contract migration (schema v1->v2, schema_version table, callable migration dispatch, upsert_canonical_extraction) - Phase 0.5 — worker-task lifecycle pattern + Slack reference wiring (closes the v0 dormant-Slack-worker gap) - Phase 1 — Notion API client + property serializer (internal- integration auth, no OAuth router) - Phase 2 — Notion ingest worker (per-database watermark, peer- authored team_event) - Phase 3 — Notion task registration on lifespan META_LEDGER entries #29-#33 capture: round-1 VETO (4 missing/ undeclared symbols), round-2 VETO (1 wrong-call-shape for decrypt_token), round-3 PASS, IMPLEMENT, and SUBSTANTIATION. SHADOW_GENOME #7 addendum extends the PARALLEL_STRUCTURE_ASSUMED detection heuristic with three new in-sketch checks: signature, type-boundary, helper-symmetry. The two VETOs in this session are the empirical justification. SYSTEM_STATE.md adds the Priority C v1 section: schema state (v2), architectural properties achieved, audit cycle outcomes, implementation deviations from plan. Merkle seal: SHA256(content_hash + previous_hash) = dcb619104e6d88b97a04689093b80b9f03825f9a24bac3c3b9ab3d0107ff24d7 (content_hash 9f003c40..., previous_hash 6f4f8f8f... = Priority C v0 SEAL at Entry #28). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hase 0) Schema v2->v3: extraction_cache gains classifier_version field (option<string> with DEFAULT 'legacy-pre-v3'). upsert_canonical_extraction now requires classifier_version as keyword-only; cache hit requires BOTH content_hash AND classifier_version match. Either differing triggers re-extraction. The option<string> type accommodates pre-v3 rows whose field reads NONE before the migration's UPDATE backfills them — strict TYPE string would reject those reads (surfaced by the v2-to-v3 backfill integration test added per audit advisory L4-B from the QorLogic Fixer's Layer 4 sweep). _migrate_v2_to_v3 callable: defines the field permissively, then unconditionally UPDATE-backfills rows where classifier_version IS NONE. Idempotent. Workers (slack, notion) pass classifier_version="legacy-pre-v3" until pipeline integration (Phase 4) supplies the real heuristic version. Tests: 14 functionality tests across Phase 0 (cache_upsert/schema adaptations + classifier_version axis verification + v2->v3 backfill integration test). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(Phase 1) team_server/extraction/heuristic_classifier.py provides Stage 1 of the extraction pipeline: pure-function classify(message, context, rules) returning ClassificationResult(is_positive, matched_triggers, classifier_version). Deterministic by construction (no LLM, no temperature, no time/uuid/random); rule-set hash drives downstream cache invalidation. Inputs: message dict (text + structural fields), context dict (reactions, thread_position, channel/db_id), TriggerRules (operator- configured + corpus-learned terms). The classifier honors: - keyword positives + keyword negatives (negatives short-circuit) - min_word_count length floor - reaction-count boosters (option d — context-aware) - thread-tail position booster (option d) - learned_keywords merge (option c — populated by Phase 5) derive_classifier_version produces a stable SHA256 hash of the sorted rule-set; changes invalidate the upsert cache via the classifier_version axis added in Phase 0. Tests: 9 functionality tests covering keyword match, negative override, length floor, reaction boost, thread-tail booster, determinism, version-changes-on-rule-change, and unicode/emoji robustness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
team_server/config.py extended with pydantic models for the heuristic trigger rules: HeuristicGlobalRules (workspace-level defaults), HeuristicScopedOverride (per-channel/database additive overrides), SlackHeuristics, NotionHeuristics, NotionConfig, CorpusLearnerConfig. YAML alias 'global:' maps to global_rules field via populate_by_name=True + alias='global' (avoids the Python reserved-word collision). Resolvers resolve_rules_for_slack and resolve_rules_for_notion produce TriggerRules | RulesDisabled, merging global + scoped + learned keywords additively. RulesDisabled is the sentinel for opted-out channels/databases. Backwards compatibility: load_channel_allowlist preserved as an alias for load_rules_from_config so existing v0 OAuth callers continue to work unchanged. Tests: 5 functionality tests covering YAML loading, channel-override merge, database-override merge, disabled-channel sentinel, and ValidationError propagation as ValueError. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
team_server/extraction/llm_extractor.py: full rewrite of the v1.0
paragraph-split placeholder. extract(text, matched_triggers) async
calls the Anthropic Messages API (claude-haiku-4-5 default; selectable
via BICAMERAL_TEAM_SERVER_EXTRACT_MODEL env). Returns structured
{"decisions": [{"summary", "context_snippet"}], "extractor_version",
"matched_triggers"}.
Failure handling:
- ANTHROPIC_API_KEY unset: raises MissingAnthropicKeyError (fail-loud)
- HTTP 429: exponential backoff retry (1s, 2s; max 3 attempts)
- HTTP 5xx / network errors: fail-soft with truncated error string
- Unparseable JSON output: fail-soft with parse-failure message
- Non-text content blocks (ToolUseBlock etc.): fail-soft (closes
Fixer L1-C from the proactive code-quality sweep)
Anthropic SDK imported lazily inside extract() so the module remains
importable when anthropic is in requirements.txt but not in dev venv
(matches the slack_sdk lazy-import pattern from v1.0 Phase 0.5).
extractor_version is a SHA256 prefix of the prompt template + model
name, so changes to either invalidate downstream cache via the
classifier_version cousin axis.
Tests: 7 functionality tests covering structured output parsing,
trigger-grounding in prompt, 429 retry, 500 fail-soft, parse-failure
fail-soft, env-overridden model, and fail-loud-on-missing-key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ge 2 (Phase 4)
team_server/extraction/pipeline.py provides the single entry point
extract_decision_pipeline(*, text, message, context, rules_or_disabled,
llm_extract_fn). Determines the output shape regardless of source:
{decisions, classifier_version, matched_triggers, extractor_version,
skipped}. extractor_version is None when Stage 2 didn't run (chatter,
rules-disabled).
slack_worker._ingest_message: builds context dict (reactions,
thread_position, thread_ts, subtype), resolves rules per channel via
config, routes through pipeline. classifier_version computed cheaply
from rules; the cache check happens BEFORE the LLM call.
notion_worker._ingest_row: builds context dict (last_edited_by,
edit_count), resolves rules per database, routes through pipeline.
Both workers preserve the legacy `extractor(text)` path when config
is None — preserves v1.0 worker tests + provides a clean cutover path
for callers that haven't adopted the rules schema.
Tests: 5 functionality tests covering pipeline short-circuit on
chatter, LLM invocation on positives, rules-disabled passthrough, and
worker-side context handoff for Slack (thread + reactions) and Notion
(edit metadata).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
team_server/extraction/corpus_learner.py reads the team-server's own team_event log (per OQ-1: not the per-repo decision table that doesn't exist server-side), extracts top n-grams from positive-extraction decisions, persists to learned_heuristic_terms with operator-denylist respected. Schema v3->v4 adds learned_heuristic_terms table (UNIQUE on source_type+term). Persistence is upsert-shaped: re-runs update support_count + learned_at without duplicating rows. resolve_rules_for_slack / resolve_rules_for_notion accept a learned=tuple[str, ...] argument that merges into TriggerRules. learned_keywords. The classifier already consumes this via the same match path as operator-configured keywords. app.py lifespan registers a corpus-learner worker via the existing worker_loop helper when config.corpus_learner.enabled is true (default false). Off-by-default; opt-in via YAML. Tests: 7 functionality tests covering n-gram extraction, denylist honor, persistence, determinism, learned-keyword merge, lifespan- on-when-enabled, lifespan-off-when-disabled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First-round PASS audit cycle for the real heuristic+LLM extractor. Plan ships across six phases (Phase 0 cache contract evolution; Phase 1 deterministic Stage 1 classifier; Phase 2 trigger rules schema; Phase 3 real Anthropic SDK Stage 2; Phase 4 pipeline integration; Phase 5 corpus learner option-c). META_LEDGER entries #34-#36 capture: round-1 PASS audit, IMPLEMENT, and SUBSTANTIATION. Three audit advisories (extract() boundary, TeamServerRules typo, corpus learner table-source) all addressed inline during implementation. A proactive QorLogic Fixer code-quality sweep before commit produced 2 MED + 2 LOW findings; both MEDs landed (fail-soft on non-text content blocks; v2->v3 backfill integration test) with one surfacing a real defect (the migration's TYPE string was rejecting reads on pre-v3 rows with NONE classifier_version; corrected to TYPE option<string>). SYSTEM_STATE.md adds the Priority C v1.1 section: schema state (v4), architectural properties achieved (heuristic-first determinism + LLM-only-when-needed + rule-version-driven cache invalidation + all four "dynamic" angles wired), audit cycle outcomes. Merkle seal: SHA256(content_hash + previous_hash) = b37003661820e2ef80591b9d0cfdeac3df092d6d9b4b5d87e3036e7ccf37d95b (content_hash e8b1b6b6..., previous_hash dcb61910... = Priority C v1 SEAL at Entry #33). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
) team_server/auth/allowlist_sync.py reconciles channel_allowlist against the workspace table from config.slack.workspaces[]: per-team_id additive + subtractive sync. Idempotent; picks up operator YAML edits on next restart. Workspaces in YAML without a corresponding workspace- table row (no OAuth completed yet) are logged and skipped — they get picked up on the next sync after OAuth completes. team_server/app.py lifespan: calls sync_channel_allowlist after ensure_schema + config load, before worker registration. The Slack runner's _channel_ids query sees populated rows on first poll cycle. Sync failures log+continue so a partial YAML doesn't block startup. Config load is now done once at the top of the lifespan body and passed through to both the allowlist sync and the corpus learner registration (deduplication of _load_config_or_default calls). Implementation note: SurrealDB v2 strict-types `record<workspace>` on channel_allowlist.workspace_id requires `type::thing()` coercion (the SELECT id from workspace returns a 'workspace:<rid>' string; passing that string back into CREATE/DELETE without coercion fails the field type check). Pattern matches the v1.0 schema migration's existing use of type::thing in _migrate_v1_to_v2. Tests: 7 functionality tests across allowlist_sync (5: insert / idempotent / skip-not-in-yaml / skip-not-in-db / removal-on-yaml-edit) and lifespan integration (2: lifespan invokes sync at startup; lifespan continues when sync raises). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three-round audit cycle (VETO → VETO → PASS) for closing v0 release blockers issues #160 (materializer event_type mismatch) and #161 (channel_allowlist not populated). META_LEDGER entries #37-#41 capture: round-1 VETO (infrastructure- mismatch — pull_team_server_events had zero production callers), round-2 VETO (specification-drift — sketch passed wrapped adapter without unwrap; would echo events O(N²) cross-dev), round-3 PASS, IMPLEMENT, SUBSTANTIATION. SHADOW_GENOME #7 heuristic catalog grew 4→6 across this branch: - Heuristic 5 (upstream-consumer) — Entry #37 - Heuristic 6 (wrapper-side-effect) — Entry #38 The catalog is the productive deposit beyond the code; each heuristic is a durable detection pattern reusable in future audits. SYSTEM_STATE.md adds the v0 release-blockers section: end-to-end ingest pipeline now functional (Slack OAuth → workspace row → YAML allowlist sync → channel_allowlist → Slack worker polls → heuristic+ LLM extraction → team_event → /events HTTP → per-dev consumer pulls → bridges to IngestPayload → per-dev local ledger). Merkle seal: SHA256(content_hash + previous_hash) = 7cc405fc8d39f468d502da669982c88321ce3a84bb571d28e0b14be86ab56bdd (content_hash 14e387b1..., previous_hash b3700366... = Priority C v1.1 SEAL at Entry #36). Closes #160, closes #161. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the user's prompt explicitly contradicts a surfaced decision, the agent now ingests the refinement and wires it via bicameral.resolve_collision(action="supersede"). Closes the v0.9.3 caller-LLM correction-capture loop that died at "render". Mechanical execution; no user-confirmation prompt — PM ratifies in inbox. Canonical action alternatives (keep_both / link_parent) cited from skills/bicameral-resolve-collision/SKILL.md as source-of-truth. Also fixes Section 7's pre-existing feature_group placement bug (top-level kwarg silently dropped by MCP dispatch since v0.x; now correctly placed in decisions[0].feature_group per IngestDecision contract at contracts.py:498). Removes stale .claude/skills/bicameral-preflight/SKILL.md duplicate per CLAUDE.md canonical-source policy (skills/ is canonical). Adds tests/test_e2e_flow_2a_in_default_set.py to gate the e2e Flow 2 contradiction-capture validation surface in CI. Closes #154 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ipt_path Reads Claude Code's SessionEnd hook stdin contract, extracts the parent session's transcript_path, and spawns capture-corrections via `claude -p` with the path propagated through BICAMERAL_PARENT_TRANSCRIPT_PATH env var. Closes the transcript-passing half of #156. Without this bridge, the prior inline shell command spawned `claude -p` with no transcript context, leaving --auto-ingest mode silently no-op. Bridge uses cwd from stdin payload (per Claude Code hook contract), falling back to os.getcwd() for manual invocations. Recursion guard preserved (BICAMERAL_SESSION_END_RUNNING). Defensive: silent no-op on malformed JSON or claude-not-on-PATH; never crashes the parent session. setup_wizard._BICAMERAL_SESSION_END_COMMAND now dispatches via `python3 -m events.session_end_bridge`. skills/bicameral-capture-corrections SKILL.md gains a one-paragraph note documenting the env-var read for --auto-ingest mode. 7 functionality tests cover the stdin → env → subprocess pipeline, including the cwd-from-stdin invariant and the literal-constant guard on the hook-command string. Partially closes #156 (transcript half; design-pivot half deferred to v0.1 per plan boundaries). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan + Merkle-sealed ledger entries for the v0-blocker session that closes #154 (preflight Step 5.6 contradiction-driven refinement capture) and the transcript-passing half of #156 (SessionEnd transcript bridge). Session 2026-05-03T0045-d2a187: 3 audit rounds (rounds 1+2 VETOed for product-taxonomy paraphrase; round 3 PASS after applying the proposed 7th SHADOW_GENOME #7 heuristic — amendment-completeness check via whole-plan grep). Heuristic operationally validated; recommend codifying. Ledger entries #42-#46: - #42: GATE round 1 VETO (infrastructure-mismatch) - #43: GATE round 2 VETO (specification-drift) - #44: GATE round 3 PASS (chain c4fc9944) - #45: IMPLEMENT (chain ceb16cc9) - #46: SEAL (Merkle 61e774e4, content ad6885d6) Closes #154 Partially closes #156 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auto-fixes 71 ruff errors (mostly I001 import-sort + UP045/UP035/UP007 modernization) accumulated across the team-server v0/v1/v1.1 sessions and Priority B v0 final-blockers session. Pure formatting; no behavioral change. Verified by: 131 team-server + plan-scope tests pass post-reformat. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Knapp-Kevin
force-pushed
the
claude/priority-c-selective-ingest
branch
from
d8f9777 to
a03aebe
Compare
Knapp-Kevin
had a problem deploying
to
recording-approval
GitHub Actions
Failure
Three real type errors reported by mypy on PR #153/#159 — none are purely cosmetic; each is fixed by tightening a contract: team_server/extraction/llm_extractor.py: - _one_attempt return type changed from tuple[str, object] to tuple[str, list[Any] | str | None]. The three branches (ok / retry / error) already produce list / None / str respectively; the union documents that explicitly so mypy can narrow at the call site. - After the 'ok' branch check, the call to _success(decisions=...) now has an isinstance(payload, list) assertion. Defensive — and satisfies _success's list parameter type. Asserts the existing invariant; doesn't add new behavior. team_server/app.py: - Replace 'from llm_extractor import extract as _interim_extractor' (2-arg signature) with an adapter function that matches the single-arg Extractor protocol the workers' legacy fallback path expects (Callable[[str], Awaitable[dict]]). - Adapter passes matched_triggers=[] because the legacy fallback path fires when rules_or_disabled is None, which means there's no upstream classifier-rule matching producing triggers. The classifier-rules path goes through extract_decision_pipeline directly and never touches this adapter. Verification: - mypy . — 132 source files, no issues - ruff check . — All checks passed - ruff format --check . — 273 files already formatted - pytest tests/test_team_server_app.py tests/test_team_server_allowlist_lifespan.py tests/test_team_server_allowlist_sync.py — 12 passed Refs PR #153 (the dev-targeting variant of this branch)
Closed
5 tasks
Knapp-Kevin
added a commit
that referenced
this pull request
May 4, 2026
Type fixes (3 mypy errors that blocked PR #153): - llm_extractor.py: _one_attempt return type now tuple[str, list[Any] | str | None]; isinstance assert added on the ok branch before _success(decisions=...) - app.py: _interim_extractor as adapter function (not bare alias) so the worker Extractor protocol Callable[[str], Awaitable[dict]] is honored; the underlying llm_extractor.extract takes 2 args (text + triggers); the adapter passes triggers=[] for the legacy fallback path Plus ruff auto-fix + ruff format pass to bring 44 files inline with current ruff config on dev (the cherry-picked commits pre-dated some style updates). Locally: mypy . (131 files), ruff check ., ruff format --check . all pass; 28/28 team-server tests pass.
5 tasks
jinhongkuan
approved these changes
May 5, 2026
jinhongkuan
left a comment
jinhongkuan
left a comment
Contributor
There was a problem hiding this comment.
shipping this per initiative to support different working envirnments
added 2 commits
May 4, 2026 21:57
Covers the two manual items in the PR description's test plan: - docker-compose stack health - Slack OAuth round-trip in a dev workspace Playwright drives the OAuth consent screen against a real Slack dev workspace via a cloudflared quick tunnel (Slack must hit the team-server's /oauth/slack/callback over HTTPS — localhost won't satisfy that). Records video as evidence; uploads as artifact. Workflow is workflow_dispatch-only and gated by the existing recording-approval environment so it shares the v0-user-flow-e2e recording job's reviewer rule. Three new secrets required — SLACK_CLIENT_ID, SLACK_CLIENT_SECRET, SLACK_STORAGE_STATE_B64 (captured per tests/manual_qa/README.md). The Fernet key for token-at-rest encryption is generated fresh per run; not a secret, since each run gets a clean SurrealKV volume.
Resolution highlights: .claude/settings.json Took dev's refactored hook scripts (post_commit_sync_reminder.py + post_preflight_capture_reminder.py) over the inline python -c form. Kept HEAD's SessionEnd dispatch to events.session_end_bridge — the bridge already encapsulates dev's inline `claude -p /bicameral-capture-corrections` invocation plus transcript-path propagation (#156), so listing both would double-fire. setup_wizard.py Kept HEAD's `python3 -m events.session_end_bridge` form for _build_session_end_command. Updated docstring's slash-command reference from /bicameral:capture-corrections to /bicameral-capture-corrections per dev's #177 rename. events/session_end_bridge.py Updated CHILD_CLAUDE_CMD's slash command from /bicameral:* to /bicameral-* form per dev's #177 rename (the bridge module was added on this branch and never saw the rename pass). skills/bicameral-preflight/SKILL.md Took dev's #175 rewrite of Step 5.6 — contradiction judgment moves from agent-internal to AskUserQuestion. The PR's mechanical-capture version was the bug that #175 was created to fix. tests/test_session_end_hook_drift.py Kept HEAD's bridge-form CANONICAL_COMMAND, matching the resolved setup_wizard behavior. tests/test_setup_wizard.py Adjusted dev's #177 regression guard to check the slash command at events.session_end_bridge.CHILD_CLAUDE_CMD instead of in the SessionEnd hook command string (where it no longer lives after the bridge refactor). Intent of the guard preserved. tests/manual_qa/test_slack_oauth_e2e.py Wrapped playwright/httpx imports in pytest.importorskip so the full suite can collect locally without those deps installed; CI installs them on demand via the manual-QA workflow. Verified: tests/test_session_end_hook_drift.py + tests/test_setup_wizard.py + tests/test_session_end_bridge.py all pass post-resolution (17/17). The pre-existing test_v0417_jargon_hygiene.py failure (BM25 / SurrealDB mentions in skill files) is present on origin/dev unmodified — not introduced by this merge.
jinhongkuan
had a problem deploying
to
recording-approval
GitHub Actions
Failure
Knapp-Kevin
added a commit
that referenced
this pull request
May 5, 2026
…r-fresh feat(team-server): Priority C v0+v1+v1.1 (clean dev-rebased reduplication of #153)
Collaborator
Author
|
Superseded by #181 — same payload, dev-rebased to avoid architectural drift; closing as resolved. |
Knapp-Kevin
added a commit
to Knapp-Kevin/bicameral-mcp
that referenced
this pull request
May 21, 2026
…rift cut Brings commits 1-22 of BicameralAI#153's branch into a fresh dev-based branch: team-server scaffold, OAuth, workers, cache evolution, classifier, LLM extractor, channel_allowlist sync (BicameralAI#161), team-server event consumer + bridge (BicameralAI#160). Skips the post-238c0ce commits that drift architecturally from dev's BicameralAI#175 preflight Step 5.6 rework and the events/session_end_bridge.py vs bicameral-capture-corrections split.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Priority C v0 vertical-slice: a self-managing customer-self-hosted team-server that ingests from Slack, runs canonical extraction once per source-event, and exposes the canonical decision set to per-dev
EventMaterializerinstances via HTTP. Closes the multi-dev extraction-divergence gap that breaks Decision Continuity (Pillar #1) at organizational scale.Five modular commits per the plan's vertical-slice structure:
1504741—feat(team-server): scaffold + self-managing schema (Phase 1)eb5881c—feat(team-server): Slack OAuth + workspace allow-list (Phase 2)8a4bee9—feat(team-server): Slack worker + canonical-extraction cache (Phase 3)1d9c9da—feat(team-server): HTTP /events API + materializer extension (Phase 4)aab80d2—docs(governance): Priority C v0 plan/research/audit/seal artifactsEach commit independently revertable; #149's rebase-merge proposal preserves granularity post-merge.
Architectural framing
Per
docs/SHADOW_GENOME.mdFailure Entry #6 + addendum, CONCEPT.md anti-goals are parsed by their load-bearing keyword: "No managed backend" forbids vendor SaaS and human-ops-tax architectures, NOT self-managing customer-deployable backends. Sentry self-hosted, Supabase OSS, and the embedded-SurrealDB philosophy already in repo are precedents.Multi-dev convergence mechanism: any dev's session that touches a Slack message produces the SAME canonical extraction across the team because
extraction_cacheis keyed on(source_type, source_ref, content_hash)and cache rows + write events flow via the existingevents/team_adapter.pyJSONL-via-git pattern + the new HTTP/eventspull.Phase 5 deferred
CocoIndex (#136) integration deferred per the operator's "if we can manage it" feasibility caveat. v0 ships with
extraction_cache.model_version='interim-claude-v1'tombstone so a follow-up plan can identify and rebuild interim entries when CocoIndex memoization lands.Test plan
pytest tests/test_team_server_*.py tests/test_materializer_team_server_pull.py— 25 / 25 PASS in 5.99spython -m pytest tests/ --co -q— existing 743 tests collect unaffectedpython -m json.tool .claude/settings.json— n/a (no settings change in this PR)workers/slack_worker.py); all functions ≤ 25 lines; depth ≤ 2infrastructure-mismatchpass — every plan claim verified againstupstream/devHEAD pre-implementationTest Functionality Pass— all 25 tests invoke their unit and assert on outputdocker-compose -f deploy/team-server.docker-compose.yml up, confirm/healthreturns{"status":"ok","schema_version":1}workspacerow persists with encrypted tokenQorLogic SDLC artifacts
docs/research-brief-priority-c-selective-ingest-2026-05-02.mdplan-priority-c-team-server-slack-v0.md.agent/staging/AUDIT_REPORT.mdefd0304b(Entry #26, #135-triage seal)6f4f8f8f1d63ad82b952a3c6aff270d30584e08b0572077ff685e84ce453f6c2INVARIANT_FROM_IMPLEMENTATION) + addendum on literal-keyword parsing of anti-goalsRisk grade — L3
EventMaterializerpull from team-server/events. Failure-isolation contract verified by Phase 4 test M1/decision relevance ruler #6 — transport error returns empty events, watermark unchanged, no exception.Audit advisory disposition
team_server/app.pysize monitoring): proactively factored OAuth routes intoteam_server/auth/router.pyand events routes intoteam_server/api/events.py.app.pyends at 47 lines.extraction_cache.canonical_extractionandteam_event.payloadper Schema:provenance ON binds_tosilently strips object metadata (missing FLEXIBLE keyword) #72 lesson.Notes for reviewer
docs/SHADOW_GENOME.mdFailure Entry M1/decision relevance ruler #6 addendum; no DNA edit until/unless an explicit governance pass touches CONCEPT.md.slack_worker.py:poll_onceis deliberate v0 simplicity — Events API would require a public callback URL not all self-host setups have. Polling shape is fine for v0; Events API consideration is a v1 concern.Related
events/team_adapter.py+events/materializer.py(feat: preflight HITL bypass flow (v0.17.1, #112) #118-era multi-dev event-sourcing substrate)