Skip to content

refactor: derivation.derive() pure function for ingest pipeline (#261) - #270

Closed
robotrocketscience wants to merge 4 commits into
mainfrom
refactor/issue-261-derivation
Closed

refactor: derivation.derive() pure function for ingest pipeline (#261)#270
robotrocketscience wants to merge 4 commits into
mainfrom
refactor/issue-261-derivation

Conversation

@robotrocketscience

Copy link
Copy Markdown
Owner

Summary

Closes #261 (partial — see Scope below). Adds src/aelfrice/derivation.py exposing a pure derive(DerivationInput) -> DerivationOutput callable that takes raw text + source metadata and returns a Belief + Edge[] (or a skip with reason). Zero MemoryStore dependency — same input, same output, no I/O.

The derivation half of the pipeline is now testable in isolation and provides the pure callable the v2.x replay_full_equality flip-readiness probe needs. Each call site that goes through derive() is rewritten to: build input → derive → check duplicate → record_ingest + insert.

Changes

  • New src/aelfrice/derivation.py (213 lines): DerivationInput, DerivationOutput, derive(). Two dispatch paths: classify-based (filesystem/git/python_ast/feedback_loop_synthesis/legacy_unknown) and lock-based (mcp_remember/cli_remember).
  • New tests/test_derivation.py (319 lines, 23 tests): purity, each source_kind, lock priors, classifier rejection, unknown-kind ValueError.
  • Refactored src/aelfrice/ingest.py and src/aelfrice/scanner.py (regex path) to call derive().

Scope: 2 of 6 entry points refactored

The issue body asked for all six entry points. Three were left direct-construction; reasons:

  1. mcp_server.tool_lock and cli._cmd_lock — use sha256("lock\x00" + content) as belief id, a deliberately distinct namespace from derive()'s sha256(source_path \x00 text) scheme. Routing through derive() would compute different ids and silently break belief lookup for existing locked beliefs in production stores. Data-identity break, not a refactor — rejected.
  2. triple_extractor._resolve_or_create_belief — uses sha256("triple\x00normalized_phrase") id namespace + fixed (1.0, 1.0) priors for noun-phrase entity beliefs. Same id-namespace incompatibility — rejected.
  3. classification.accept_classifications — applies host-LLM classification results that have already been computed. Calling derive() here would re-run the regex classifier and potentially override the host's typed results. Behavior change, not a refactor — rejected.
  4. Scanner's opt-in llm_router path — returns custom origin (e.g. DOCUMENT_RECENT) not representable in current DerivationInput. The default no-LLM regex path (production default) is fully routed.

These four sites need follow-up design work before they can be unified — id-namespace migration for #1/#2, classifier-replay handling for #3, DerivationInput extension for #4. Each is queen-tier, not mechanical refactor. Recommend filing as follow-up issues if the unification is wanted; otherwise leaving them as their own derivation paths is defensible.

Test plan

Commits

  • f258fa1 — add derivation.derive() pure function
  • a76fb18 — route ingest through derive()
  • 14ac869 — route scanner regex path through derive()

Out of scope

LLM-router path retains direct Belief construction: the router returns
a custom origin (DOCUMENT_RECENT, etc.) that is not expressible through
the current DerivationInput fields. Regex path (production default) is
fully routed through derive().
@robotrocketscience robotrocketscience added review-Kulili PR coordination mutex and removed review-Kulili PR coordination mutex labels Apr 29, 2026
@robotrocketscience

Copy link
Copy Markdown
Owner Author

Superseded by #273 (merged). Same target: extract pure derive() for #261.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v2.x prereq] Factor out deterministic belief derivation function

1 participant