feat(embeddings): route batch-tolerant import embeddings via contextual-orchestrator - #973
Conversation
…rsers Add property-based fuzz targets (Hypothesis, MPL-2.0) for the highest-value untrusted-input surfaces surfaced via CodeGraph: - parse_eml_bytes (raw MIME email from providers) - strip_html_markup / contains_html_markup (hand-rolled HTML sanitiser) - parse_content (content-graph HTML/markdown/plain-text parser) - parse_email_attachment (attacker-controlled filename/content-type/payload) Shared invariants live in fuzz/_invariants.py (no crash, declared-exception-only, structural graph consistency, sanitisation guarantees) and are reused by both the pytest property tests and the optional Atheris harnesses. Seed corpus included. Fuzzing found and this commit fixes two real crashes in email ingest, where _message_to_email_data ran outside parse_eml_bytes' try/except: - non-ASCII addr-spec (IDN domain / encoded-word From) -> UnicodeEncodeError from email.utils.formataddr; now guarded by _safe_formataddr. - multipart-declared message with no parsable parts -> KeyError from Message.get_content() on the single-part path; now guarded like the attachment path. Both are normalised to the declared EmailParseError as a safety net. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
- Atheris (Apache-2.0) libFuzzer harnesses for all four targets, sharing the same invariants as the Hypothesis property tests; seed corpus wired per target. - .github/workflows/fuzz.yml: runs the four Hypothesis targets in parallel on every PR with a wall-clock-capped budget (HYPOTHESIS_PROFILE=ci, ~1000 examples/target, 150s hard cap), plus a larger nightly schedule and an optional best-effort Atheris campaign. - fuzz/conftest.py registers dev/ci Hypothesis profiles (guarded so the primary hash-pinned test job, which has no Hypothesis, still collects cleanly and just skips the property tests). - fuzz/requirements-fuzz.txt (permissive: hypothesis MPL-2.0, atheris Apache-2.0) and fuzz/README.md documenting targets, engines, and the two findings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
Add "The Art, Science, and Engineering of Fuzzing: A Survey" (Manès et al., arXiv:1812.00140) as background for backend/fuzz, which combines property-based (Hypothesis) and coverage-guided (Atheris/libFuzzer) fuzzing of the untrusted-input parsers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
…al-orchestrator Rewire naruon so bulk, latency-tolerant embedding work (email import) goes THROUGH contextual-orchestrator — the routing/cost hub — instead of calling a batch engine directly. The orchestrator owns provider selection, load balancing and cost accounting and forwards to pg-llm-batch; naruon submits a batch and records the reported cost. Primary path (backend/services/batch_embedding_service.py): - Resolve orchestrator base_url + bearer token + endpoint alias + model from the per-tenant Fernet-encrypted tenant_configs row (never os.getenv; token column is EncryptedString). - Submit/retrieve over the existing SSRF-guarded, allowlisted, pinned-address HTTP client (build_llm_provider_http_client) that fronts every outbound LLM call — DNS-rebinding safe, redirects off, trust_env off. - Record a durable llm_batch_jobs / llm_batch_items audit trail (routing mode, orchestrator batch id, reported cost_micro_usd) via alembic 0010. Graceful degradation: - Batch disabled, base URL rejected by the egress guard, or orchestrator unreachable -> return None so the caller uses its per-item embedding path. - The pg-llm-batch submodule stays only as an optional offline-dev fallback, gated behind orchestrator-unavailable + an explicit local DSN, and is import-guarded so naruon runs with it uninitialized. Tests (backend/tests/test_batch_embedding_service.py): mocked orchestrator submit, submit-then-retrieve polling, SSRF/allowlist rejection, unreachable + HTTP-error + incomplete-vector fallback, local-submodule fallback, config resolved from the Fernet DB (secrets encrypted at rest, never env), and the email_import wiring. Backend suite stays green. Docs: docs/papers adds FrugalGPT, RouteLLM, and Hybrid LLM (cost-aware routing / load balancing) with citations; ARCHITECTURE.md documents the routing boundary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
The Hypothesis/Atheris fuzzing suite (.github/workflows/fuzz.yml, backend/fuzz/*, the parser-hardening it bundled in email_parser.py, and the fuzzing survey paper) is unrelated to batch embedding routing and is covered by a separate dedicated fuzzing PR. Remove it so this PR stays scoped to routing batch embeddings via contextual-orchestrator. The LLM cost/routing papers stay. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
Reconcile the batch embedding contract with contextual-orchestrator's new
/v1/batch/embeddings endpoint and complete cost attribution:
- Send the FULL attribution metadata the orchestrator ledger expects — service,
team, group, company (on top of source/organization_id/user_id) — resolved
from new tenant-config columns (batch_attribution_service/team/group/company),
never env. company falls back to organization_id, service to the import name.
- Add a real cross-service contract test: assert the client serializes to exactly
POST /v1/batch/embeddings with the contract payload and parses the exact
{batch_id,status,embeddings,cost_micro_usd,token_counts} response. Both repos
keep a byte-identical tests fixtures/batch_embeddings_contract.json and assert
their side against it, so the contract cannot silently drift again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
PR governance metadata gate is not ready for
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headf0f30e7da5dfdae97d99e0f7facb146a56c84093. -
Head SHA:
f0f30e7da5dfdae97d99e0f7facb146a56c84093 -
Workflow run: 28921722733
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
f0f30e7da5dfdae97d99e0f7facb146a56c84093 - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Python project dependencies (backend/requirements.txt)
Defaulting to user installation because normal site-packages is not writeable
Collecting fastapi==0.138.2 (from -r requirements.txt (line 1))
Downloading fastapi-0.138.2-py3-none-any.whl.metadata (26 kB)
Collecting starlette==1.3.1 (from -r requirements.txt (line 2))
Downloading starlette-1.3.1-py3-none-any.whl.metadata (6.4 kB)
Collecting uvicorn==0.49.0 (from -r requirements.txt (line 3))
Downloading uvicorn-0.49.0-py3-none-any.whl.metadata (6.7 kB)
Requirement already satisfied: pytest==9.1.1 in /home/runner/.local/lib/python3.12/site-packages (from -r requirements.txt (line 4)) (9.1.1)
Collecting httpx==0.28.1 (from -r requirements.txt (line 5))
Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Collecting pydantic-settings==2.14.2 (from -r requirements.txt (line 6))
Downloading pydantic_settings-2.14.2-py3-none-any.whl.metadata (3.4 kB)
Collecting aiosmtplib==5.1.2 (from -r requirements.txt (line 7))
Downloading aiosmtplib-5.1.2-py3-none-any.whl.metadata (3.6 kB)
Collecting aioimaplib==2.0.1 (from -r requirements.txt (line 8))
Downloading aioimaplib-2.0.1-py3-none-any.whl.metadata (9.5 kB)
Collecting sqlalchemy==2.0.51 (from -r requirements.txt (line 9))
Downloading sqlalchemy-2.0.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (9.5 kB)
Collecting alembic==1.18.5 (from -r requirements.txt (line 10))
Downloading alembic-1.18.5-py3-none-any.whl.metadata (7.2 kB)
Collecting greenlet==3.5.3 (from -r requirements.txt (line 11))
Downloading greenlet-3.5.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting asyncpg==0.31.0 (from -r requirements.txt (line 12))
Downloading asyncpg-0.31.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (4.4 kB)
Collecting pgvector==0.4.2 (from -r requirements.txt (line 13))
Downloading pgvector-0.4.2-py3-none-any.whl.metadata (19 kB)
Collecting pytest-asyncio==1.4.0 (from -r requirements.txt (line 14))
Downloading pytest_asyncio-1.4.0-py3-none-any.whl.metadata (4.1 kB)
Collecting openai==2.44.0 (from -r requirements.txt (line 15))
Downloading openai-2.44.0-py3-none-any.whl.metadata (34 kB)
Collecting langchain-text-splitters==1.1.2 (from -r requirements.txt (line 16))
Downloading langchain_text_splitters-1.1.2-py3-none-any.whl.metadata (3.3 kB)
Collecting tiktoken==0.13.0 (from -r requirements.txt (line 17))
Downloading tiktoken-0.13.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (6.7 kB)
Collecting google-api-python-client==2.198.0 (from -r requirements.txt (line 18))
Downloading google_api_python_client-2.198.0-py3-none-any.whl.metadata (7.0 kB)
Collecting google-auth-httplib2==0.4.0 (from -r requirements.txt (line 19))
Downloading google_auth_httplib2-0.4.0-py3-none-any.whl.metadata (3.0 kB)
Collecting google-auth-oauthlib==1.4.0 (from -r requirements.txt (line 20))
Downloading google_auth_oauthlib-1.4.0-py3-none-any.whl.metadata (2.6 kB)
Collecting email-validator==2.3.0 (from -r requirements.txt (line 21))
Downloading email_validator-2.3.0-py3-none-any.whl.metadata (26 kB)
Collecting cryptography==49.0.0 (from -r requirements.txt (line 22))
Downloading cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl.metadata (4.3 kB)
Collecting python-multipart==0.0.32 (from -r requirements.txt (line 23))
Downloading python_multipart-0.0.32-py3-none-any.whl.metadata (2.1 kB)
Collecting prometheus-fastapi-instrumentator==8.0.2 (from -r requirements.txt (line 24))
Downloading prometheus_fastapi_instrumentator-8.0.2-py3-none-any.whl.metadata (13 kB)
Collecting opentelemetry-api==1.43.0 (from -r requirements.txt (line 25))
Downloading opentelemetry_api-1.43.0-py3-none-any.whl.metadata (1.4 kB)
Collecting opentelemetry-sdk==1.43.0 (from -r requirements.txt (line 26))
Downloading opentelemetry_sdk-1.43.0-py3-none-any.whl.metadata (1.7 kB)
Collecting opentelemetry-instrumentation-fastapi==0.64b0 (from -r requirements.txt (line 27))
Downloading opentelemetry_instrumentation_fastapi-0.64b0-py3-none-any.whl.metadata (2.2 kB)
Collecting opentelemetry-exporter-otlp==1.43.0 (from -r requirements.txt (line 28))
Downloading opentelemetry_exporter_otlp-1.43.0-py3-none-any.whl.metadata (2.4 kB)
Collecting protobuf==7.35.1 (from -r requirements.txt (line 29))
Downloading protobuf-7.35.1-cp310-abi3-manylinux2014_x86_64.whl.metadata (595 bytes)
Collecting setuptools==82.0.1 (from -r requirements.txt (line 30))
Downloading setuptools-82.0.1-py3-none-any.whl.metadata (6.5 kB)
Collecting wheel==0.47.0 (from -r requirements.txt (line 31))
Downloading wheel-0.47.0-py3-none-any.whl.metadata (2.3 kB)
Collecting websockets==16.0 (from -r requirements.txt (line 32))
Downloading websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (6.8 kB)
Collecting PyJWT==2.13.0 (from -r requirements.txt (line 33))
Downloading pyjwt-2.13.0-py3-none-any.whl.metadata (3.4 kB)
Collecting icalendar==7.2.0 (from -r requirements.txt (line 34))
Downloading icalendar-7.2.0-py3-none-any.whl.metadata (6.9 kB)
Collecting ruff==0.15.20 (from -r requirements.txt (line 35))
Downloading ruff-0.15.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (26 kB)
Collecting defusedxml==0.7.1 (from -r requirements.txt (line 36))
Downloading defusedxml-0.7.1-py2.py3-none-any.whl.metadata (32 kB)
Collecting pydantic>=2.9.0 (from fastapi==0.138.2->-r requirements.txt (line 1))
Downloading pydantic-2.13.4-py3-none-any.whl.metadata (109 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 109.4/109.4 kB 5.9 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/lib/python3/dist-packages (from fastapi==0.138.2->-r requirements.txt (line 1)) (4.10.0)
Collecting typing-inspection>=0.4.2 (from fastapi==0.138.2->-r requirements.txt (line 1))
Downloading typing_inspection-0.4.2-py3-none-any.whl.metadata (2.6 kB)
Collecting annotated-doc>=0.0.2 (from fastapi==0.138.2->-r requirements.txt (line 1))
Downloading annotated_doc-0.0.4-py3-none-any.whl.metadata (6.6 kB)
Collecting anyio<5,>=3.6.2 (from starlette==1.3.1->-r requirements.txt (line 2))
Downloading anyio-4.14.1-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: click>=7.0 in /usr/lib/python3/dist-packages (from uvicorn==0.49.0->-r requirements.txt (line 3)) (8.1.6)
Collecting h11>=0.8 (from uvicorn==0.49.0->-r requirements.txt (line 3))
Downloading h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
Requirement already satisfied: iniconfig>=1.0.1 in /home/runner/.local/lib/python3.12/site-packages (from pytest==9.1.1->-r requirements.txt (line 4)) (2.3.0)
Requirement already satisfied: packaging>=22 in /usr/lib/python3/dist-packages (from pytest==9.1.1->-r requirements.txt (line 4)) (24.0)
Requirement already satisfied: pluggy<2,>=1.5 in /home/runner/.local/lib/python3.12/site-packages (from pytest==9.1.1->-r requirements.txt (line 4)) (1.6.0)
Requirement already satisfied: pygments>=2.7.2 in /usr/lib/python3/dist-packages (from pytest==9.1.1->-r requirements.txt (line 4)) (2.17.2)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from httpx==0.28.1->-r requirements.txt (line 5)) (2023.11.17)
Collecting httpcore==1.* (from httpx==0.28.1->-r requirements.txt (line 5))
Downloading httpcore-1.0.9-py3-none-any.whl.metadata (21 kB)
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from httpx==0.28.1->-r requirements.txt (line 5)) (3.6)
Collecting python-dotenv>=0.21.0 (from pydantic-settings==2.14.2->-r requirements.txt (line 6))
Downloading python_dotenv-1.2.2-py3-none-any.whl.metadata (27 kB)
Collecting Mako (from alembic==1.18.5->-r requirements.txt (line 10))
Downloading mako-1.3.12-py3-none-any.whl.metadata (2.9 kB)
Collecting typing-extensions>=4.8.0 (from fastapi==0.138.2->-r requirements.txt (line 1))
Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
Collecting numpy (from pgvector==0.4.2->-r requirements.txt (line 13))
Downloading numpy-2.5.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (6.6 kB)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai==2.44.0->-r requirements.txt (line 15)) (1.9.0)
Collecting jiter<1,>=0.10.0 (from openai==2.44.0->-r requirements.txt (line 15))
Downloading jiter-0.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.2 kB)
Collecting sniffio (from openai==2.44.0->-r requirements.txt (line 15))
Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting tqdm>4 (from openai==2.44.0->-r requirements.txt (line 15))
Downloading tqdm-4.68.4-py3-none-any.whl.metadata (57 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.4/57.4 kB 16.7 MB/s eta 0:00:00
Collecting langchain-core<2.0.0,>=1.2.31 (from langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading langchain_core-1.4.8-py3-none-any.whl.metadata (4.7 kB)
Collecting regex (from tiktoken==0.13.0->-r requirements.txt (line 17))
Downloading regex-2026.6.28-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.5/40.5 kB 13.3 MB/s eta 0:00:00
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from tiktoken==0.13.0->-r requirements.txt (line 17)) (2.31.0)
Requirement already satisfied: httplib2<1.0.0,>=0.19.0 in /usr/lib/python3/dist-packages (from google-api-python-client==2.198.0->-r requirements.txt (line 18)) (0.20.4)
Collecting google-auth!=2.24.0,!=2.25.0,<3.0.0,>=1.32.0 (from google-api-python-client==2.198.0->-r requirements.txt (line 18))
Downloading google_auth-2.55.2-py3-none-any.whl.metadata (5.2 kB)
Collecting google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0,>=1.31.5 (from google-api-python-client==2.198.0->-r requirements.txt (line 18))
Downloading google_api_core-2.31.0-py3-none-any.whl.metadata (3.2 kB)
Collecting uritemplate<5,>=3.0.1 (from google-api-python-client==2.198.0->-r requirements.txt (line 18))
Downloading uritemplate-4.2.0-py3-none-any.whl.metadata (2.6 kB)
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib==1.4.0->-r requirements.txt (line 20))
Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB)
Collecting dnspython>=2.0.0 (from email-validator==2.3.0->-r requirements.txt (line 21))
Downloading dnspython-2.8.0-py3-none-any.whl.metadata (5.7 kB)
Collecting cffi>=2.0.0 (from cryptography==49.0.0->-r requirements.txt (line 22))
Downloading cffi-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.5 kB)
Collecting prometheus-client<1.0.0,>=0.8.0 (from prometheus-fastapi-instrumentator==8.0.2->-r requirements.txt (line 24))
Downloading prometheus_client-0.25.0-py3-none-any.whl.metadata (2.1 kB)
Collecting opentelemetry-semantic-conventions==0.64b0 (from opentelemetry-sdk==1.43.0->-r requirements.txt (line 26))
Downloading opentelemetry_semantic_conventions-0.64b0-py3-none-any.whl.metadata (2.4 kB)
Collecting opentelemetry-instrumentation-asgi==0.64b0 (from opentelemetry-instrumentation-fastapi==0.64b0->-r requirements.txt (line 27))
Downloading opentelemetry_instrumentation_asgi-0.64b0-py3-none-any.whl.metadata (2.0 kB)
Collecting opentelemetry-instrumentation==0.64b0 (from opentelemetry-instrumentation-fastapi==0.64b0->-r requirements.txt (line 27))
Downloading opentelemetry_instrumentation-0.64b0-py3-none-any.whl.metadata (7.2 kB)
Collecting opentelemetry-util-http==0.64b0 (from opentelemetry-instrumentation-fastapi==0.64b0->-r requirements.txt (line 27))
Downloading opentelemetry_util_http-0.64b0-py3-none-any.whl.metadata (2.6 kB)
Collecting opentelemetry-exporter-otlp-proto-grpc==1.43.0 (from opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading opentelemetry_exporter_otlp_proto_grpc-1.43.0-py3-none-any.whl.metadata (2.6 kB)
Collecting opentelemetry-exporter-otlp-proto-http==1.43.0 (from opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading opentelemetry_exporter_otlp_proto_http-1.43.0-py3-none-any.whl.metadata (2.4 kB)
Requirement already satisfied: python-dateutil in /usr/lib/python3/dist-packages (from icalendar==7.2.0->-r requirements.txt (line 34)) (2.8.2)
Collecting tzdata>=2025.3 (from icalendar==7.2.0->-r requirements.txt (line 34))
Downloading tzdata-2026.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting googleapis-common-protos~=1.57 (from opentelemetry-exporter-otlp-proto-grpc==1.43.0->opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading googleapis_common_protos-1.75.0-py3-none-any.whl.metadata (8.6 kB)
Collecting grpcio<2.0.0,>=1.63.2 (from opentelemetry-exporter-otlp-proto-grpc==1.43.0->opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading grpcio-1.82.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (3.7 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.43.0 (from opentelemetry-exporter-otlp-proto-grpc==1.43.0->opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading opentelemetry_exporter_otlp_proto_common-1.43.0-py3-none-any.whl.metadata (1.8 kB)
Collecting opentelemetry-proto==1.43.0 (from opentelemetry-exporter-otlp-proto-grpc==1.43.0->opentelemetry-exporter-otlp==1.43.0->-r requirements.txt (line 28))
Downloading opentelemetry_proto-1.43.0-py3-none-any.whl.metadata (2.3 kB)
Collecting wrapt<3.0.0,>=1.0.0 (from opentelemetry-instrumentation==0.64b0->opentelemetry-instrumentation-fastapi==0.64b0->-r requirements.txt (line 27))
Downloading wrapt-2.2.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (7.4 kB)
Collecting asgiref~=3.0 (from opentelemetry-instrumentation-asgi==0.64b0->opentelemetry-instrumentation-fastapi==0.64b0->-r requirements.txt (line 27))
Downloading asgiref-3.11.1-py3-none-any.whl.metadata (9.3 kB)
Collecting pycparser (from cffi>=2.0.0->cryptography==49.0.0->-r requirements.txt (line 22))
Downloading pycparser-3.0-py3-none-any.whl.metadata (8.2 kB)
Collecting proto-plus<2.0.0,>=1.24.0 (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0,>=1.31.5->google-api-python-client==2.198.0->-r requirements.txt (line 18))
Downloading proto_plus-1.28.0-py3-none-any.whl.metadata (2.2 kB)
Collecting requests (from tiktoken==0.13.0->-r requirements.txt (line 17))
Downloading requests-2.34.2-py3-none-any.whl.metadata (4.8 kB)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth!=2.24.0,!=2.25.0,<3.0.0,>=1.32.0->google-api-python-client==2.198.0->-r requirements.txt (line 18)) (0.2.8)
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /usr/lib/python3/dist-packages (from httplib2<1.0.0,>=0.19.0->google-api-python-client==2.198.0->-r requirements.txt (line 18)) (3.1.1)
Collecting jsonpatch<2.0.0,>=1.33.0 (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)
Collecting langchain-protocol>=0.0.17 (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading langchain_protocol-0.0.18-py3-none-any.whl.metadata (2.4 kB)
Collecting langsmith<1.0.0,>=0.3.45 (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading langsmith-0.9.8-py3-none-any.whl.metadata (22 kB)
Requirement already satisfied: pyyaml<7.0.0,>=5.3.0 in /usr/lib/python3/dist-packages (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16)) (6.0.1)
Collecting tenacity!=8.4.0,<10.0.0,>=8.1.0 (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading tenacity-9.1.4-py3-none-any.whl.metadata (1.2 kB)
Collecting uuid-utils<1.0,>=0.12.0 (from langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading uuid_utils-0.16.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
Collecting annotated-types>=0.6.0 (from pydantic>=2.9.0->fastapi==0.138.2->-r requirements.txt (line 1))
Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.46.4 (from pydantic>=2.9.0->fastapi==0.138.2->-r requirements.txt (line 1))
Downloading pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting charset_normalizer<4,>=2 (from requests->tiktoken==0.13.0->-r requirements.txt (line 17))
Downloading charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.7/41.7 kB 11.7 MB/s eta 0:00:00
Requirement already satisfied: urllib3<3,>=1.26 in /usr/lib/python3/dist-packages (from requests->tiktoken==0.13.0->-r requirements.txt (line 17)) (2.0.7)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib==1.4.0->-r requirements.txt (line 20)) (3.2.2)
Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/lib/python3/dist-packages (from Mako->alembic==1.18.5->-r requirements.txt (line 10)) (2.1.5)
Requirement already satisfied: jsonpointer>=1.9 in /usr/lib/python3/dist-packages (from jsonpatch<2.0.0,>=1.33.0->langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16)) (2.0)
Collecting orjson>=3.9.14 (from langsmith<1.0.0,>=0.3.45->langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading orjson-3.11.9-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.0/42.0 kB 10.5 MB/s eta 0:00:00
Collecting requests-toolbelt>=1.0.0 (from langsmith<1.0.0,>=0.3.45->langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB)
Collecting xxhash>=3.0.0 (from langsmith<1.0.0,>=0.3.45->langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading xxhash-3.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (15 kB)
Collecting zstandard>=0.23.0 (from langsmith<1.0.0,>=0.3.45->langchain-core<2.0.0,>=1.2.31->langchain-text-splitters==1.1.2->-r requirements.txt (line 16))
Downloading zstandard-0.25.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (3.3 kB)
Downloading fastapi-0.138.2-py3-none-any.whl (129 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.3/129.3 kB 22.8 MB/s eta 0:00:00
Downloading starlette-1.3.1-py3-none-any.whl (73 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.6/73.6 kB 27.7 MB/s eta 0:00:00
Downloading uvicorn-0.49.0-py3-none-any.whl (71 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.4/71.4 kB 25.6 MB/s eta 0:00:00
Downloading httpx-0.28.1-py3-none-any.whl (73 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.5/73.5 kB 29.8 MB/s eta 0:00:00
Downloading pydantic_settings-2.14.2-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.7/61.7 kB 24.3 MB/s eta 0:00:00
Downloading aiosmtplib-5.1.2-py3-none-any.whl (28 kB)
Downloading aioimaplib-2.0.1-py3-none-any.whl (34 kB)
Downloading sqlalchemy-2.0.51-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (3.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 39.2 MB/s eta 0:00:00
Downloading alembic-1.18.5-py3-none-any.whl (264 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.7/264.7 kB 85.9 MB/s eta 0:00:00
Downloading greenlet-3.5.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (614 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 614.2/614.2 kB 158.9 MB/s eta 0:00:00
Downloading asyncpg-0.31.0-cp312-cp312-manylinux_2_28_x86_64.whl (3.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 147.5 MB/s eta 0:00:00
Downloading pgvector-0.4.2-py3-none-any.whl (27 kB)
Downloading pytest_asyncio-1.4.0-py3-none-any.whl (16 kB)
Downloading openai-2.44.0-py3-none-any.whl (1.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 204.7 MB/s eta 0:00:00
- Result: PASS
Python coverage with missing-line report (backend)
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/naruon/naruon/pr-head/backend
configfile: pytest.ini
## Changed-File Evidence Map
```mermaid
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (4 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (4 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (8 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (8 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (4 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (4 files)"]
R3 --> V3["docs review"]
OpenCode Review Overview
Pull request overviewOpenCode reviewed the current-head bounded evidence and found no blocking issues. FindingsNo blocking findings. SummaryApproval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (2 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (2 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (8 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (8 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
|
Drop F401 dead imports EmailImportEmbeddingProvider and the batch_embedding_service module alias; neither is referenced. Unblocks the backend ruff gate on Python 3.14.
Add token/char-budgeted map-reduce for /v1/batch/embeddings so Azure/LiteLLM embedding size limits are handled before provider calls. Oversized inputs are mapped to safe parts, part vectors are reduced with a token-weighted average, and the usage ledger still records one row per original vector.\n\nRefs ContextualWisdomLab/naruon#973.
…46) * feat: LLM cost-review ledger + sync/batch routing hub (pg-llm-batch) Extend the OpenAI-compatible orchestrator into the LLM cost-review and routing control point. Cost review: - llm_usage_records ledger written on every completion (sync + batch) with token counts and cost computed from a configurable llm_price_entries price table; seven attribution dimensions catalogued in cost_attribution_dimensions (account, service, upstream_api/provider, model_name, team, group, company). - Rollup/report by any dimension over any time window. - Token counts reuse pg-llm-batch pg_tiktoken when a DSN is set; deterministic heuristic otherwise. In-memory + PEP-249 SQL (stdlib sqlite3 / psycopg) stores. Routing: - RoutingPolicy decides sync vs batch from request hints + KV thresholds. - Batch path submits to pg-llm-batch (added as a git submodule + a client) over its OpenAI-compatible Batch API; local in-process backend preserves the mock/standalone path. Cost-optimising upstream selection (cheapest_upstream). Server: - /healthz liveness; /api/v1/cost_reports/rollup, /api/v1/llm_usage_records, /api/v1/cost_attribution_dimensions, /api/v1/batch_routing_jobs(+/{id}[/results]). - /v1/chat/completions records usage and honours attribution + routing hints; real token usage now reported. OpenAPI contract updated. Config/secrets come from a KV store, never runtime os.getenv. DB objects are 2+ word snake_case. Papers (FrugalGPT, RouteLLM, Hybrid LLM) added under docs/papers with citations. CodeGraph index built on clone and used to locate the complete/route_once/_invoke seam before editing. Tests: cost-ledger writes + multi-dim rollup + price computation, sync-vs-batch decision, and batch submit/retrieve to pg-llm-batch (mocked); 174 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P * feat(batch): add batch embeddings endpoint for naruon contract Add POST /v1/batch/embeddings + GET /v1/batch/embeddings/{batch_id} so naruon's batch-tolerant import embeddings route through the orchestrator (the routing / cost hub) instead of 404-ing and silently falling back to per-item calls. - batch_routing.py: EmbeddingBatchRequest/ResultItem, LocalEmbeddingBatchBackend (deterministic in-process embedder for the mock/standalone path) and PgLlmBatchEmbeddingBackend (embeddings JSONL -> pg-llm-batch). - cost_router.py: submit/poll/complete embeddings batch; records one usage-ledger row per vector with full attribution, returns naruon's exact shape {batch_id, status, embeddings:[{index,embedding}], cost_micro_usd, token_counts, total_tokens, part_count}. Idempotent poll (cost recorded once). - server.py: inference-scoped routes; accepts inputs (naruon) or OpenAI input; maps attribution dims from metadata into the ledger. - api_contract.py OpenAPI + README/rest_api_design docs. - Real end-to-end contract test over a live socket via LocalEmbeddingBatchBackend, asserting the shared tests/fixtures/batch_embeddings_contract.json shape. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P * feat: split oversized embedding batches Add token/char-budgeted map-reduce for /v1/batch/embeddings so Azure/LiteLLM embedding size limits are handled before provider calls. Oversized inputs are mapped to safe parts, part vectors are reduced with a token-weighted average, and the usage ledger still records one row per original vector.\n\nRefs ContextualWisdomLab/naruon#973. * fix(ci): raise review tooling minimums Raise the opencode review CI uv and pytest minimums past known vulnerable ranges so the central OSV scan does not fail on the review-tooling manifest. Mirrors the focused fix already validated by #50. * fix(ci): remove pg batch submodule gitlink * feat: make usage telemetry prompt-safe and non-blocking * chore: retrigger required OpenCode workflow --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
Heads-up for the next rebase: develop now has migration id Generated by Claude Code |
|
Rebase heads-up #2: develop now also has Generated by Claude Code |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including ARCHITECTURE.md, backend/alembic/versions/0011_llm_batch_orchestrator.py, backend/db/models.py, backend/services/batch_embedding_service.py, backend/services/email_import_service.py, and 6 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects ARCHITECTURE.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: PR adds batch embedding routing via orchestrator with tests and proper security
- Head SHA:
9c9c37f192f9c3a4593415245dc6f066dd477355 - Workflow run: 29146048887
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (2 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (2 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (8 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (8 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
Superseded by current-head OpenCode approval on 9c9c37f and passing coverage-evidence/opencode-review checks.
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including ARCHITECTURE.md, backend/alembic/versions/0012_llm_batch_orchestrator.py, backend/db/models.py, backend/services/batch_embedding_service.py, backend/services/email_import_service.py, and 6 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects ARCHITECTURE.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: PR adds batch embedding routing via orchestrator with tests and migration; no blocking issues found.
- Head SHA:
876bc13be2c52cae046d25ca211f8b1df10809f9 - Workflow run: 29146697148
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (2 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (2 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (8 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (8 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
github-project-automation
Bot
moved this from In Progress
to Done
in naruon Platform Roadmap
2 participants
Why
naruon#969 wired the import-embedding hotspot directly to the
pg-llm-batchengine. This PR rewires batch LLM work to go through contextual-orchestrator
-- the routing/cost hub -- so the orchestrator controls provider selection,
load balancing, and cost, and can forward to
pg-llm-batch. naruon no longerowns the batch engine on the primary path.
What changed
Primary path --
backend/services/batch_embedding_service.py(new).try_batch_import_embeddings(...)submits bulk import embeddings to theorchestrator batch API (
POST /v1/batch/embeddings, poll retrieve untilterminal) and maps vectors back to input order.
resolved from the per-tenant Fernet-encrypted
tenant_configsrow viaget_scoped_tenant_config-- neveros.getenv. The token and local DSNare
EncryptedStringcolumns (Fernet at rest).build_llm_provider_http_client-- the SSRF-guarded,allowlisted, pinned-address client (HTTPS-only, no redirects,
trust_envoff) that fronts every other outbound LLM call.
cost_micro_usd+ batch id are recorded on the job row.Tracking --
db.models+ alembic0012_llm_batch_orchestrator.llm_batch_jobs/llm_batch_items(2+ word snake_case) audit trail withrouting_mode(orchestrator|local_engine),orchestrator_batch_uid,cost_micro_usd. TenantConfig gainsbatch_embedding_enabled,batch_orchestrator_base_url,batch_orchestrator_token,batch_orchestrator_endpoint,batch_embedding_model,batch_local_dsn.develop(58f957a0), this migration follows0011_email_model_reconciliationas0012_llm_batch_orchestrator, keeping asingle Alembic head.
Graceful degradation. Batch disabled, base URL rejected by the egress guard,
or orchestrator unreachable -> return
Nonesoemail_import_servicetransparently falls back to the per-item embedding path.
Local engine gated for offline dev only. The opt-in
docker-compose.pg-llm-batch.ymloverlay can point at an externalpg-llm-batchcheckout viaPG_LLM_BATCH_CHECKOUT, but naruon does not vendora gitlink for it in this PR.
load_batch_engineis import-guarded, so naruonruns without the package installed and degrades to the per-item path unless an
explicit local DSN is configured after orchestrator-unavailable.
CodeGraph
Built a CodeGraph index on the fresh clone and explored before editing. Blast
radius confirmed
build_llm_provider_http_client(backend/services/llm_provider_urls.py:441,8 callers) as the shared pinned-client boundary and
generate_embeddings(backend/services/embedding.py:37, 13 callers incl.
email_import_service) as thehotspot; the batch adapter reuses both.
Tests
backend/tests/test_batch_embedding_service.py(fully mocked): orchestratorsubmit, submit-then-retrieve polling, SSRF/allowlist rejection, unreachable +
HTTP-error + incomplete-vector fallback, local-engine fallback, config from the
Fernet DB (secrets encrypted at rest, never env), email_import wiring.
Current maintainer verification after merging latest
develop(2026-07-11,head
876bc13b):python3 -m ruff check services/batch_embedding_service.py tests/test_batch_embedding_service.py tests/test_batch_embeddings_contract.py tests/test_alembic_migrations.py tests/test_email_model_reconciliation.py db/models.pypython3 -m pytest -q tests/test_batch_embedding_service.py tests/test_batch_embeddings_contract.py tests/test_alembic_migrations.py tests/test_email_model_reconciliation.py(35 passed)python3 -m bandit -q -r services/batch_embedding_service.pygit diff --check.gitmodules/vendor/pg-llm-batch; the offlinefallback is external-checkout only, so Strix scans regular PR-head files.
Docs
docs/papers/preserves source PDFs without Git LFS for FrugalGPT(arXiv:2305.05176), RouteLLM (arXiv:2406.18665), Hybrid LLM (arXiv:2404.14618),
and the current batch-specific 2026 paper Robust Batch-Level Query Routing
(arXiv:2603.26796).
git check-attr filter -- docs/papers/*.pdfreportsfilter: unspecifiedfor each PDF.ARCHITECTURE.mddocuments the batchembedding routing boundary.
Permissive-only: no GPL/AGPL deps added; the optional local batch engine remains
an external first-party checkout, not vendored into this PR.
Contract reconciliation (with contextual-orchestrator PR #46)
This PR's client POSTed to
/v1/batch/embeddings, but the orchestrator branchexposed only
/api/v1/batch_routing_jobs(chat JSONL) -- so real batch calls404'd and silently fell back to per-item embedding; only the mocks passed. The
two PRs are now reconciled on one contract:
POST /v1/batch/embeddings+GET /v1/batch/embeddings/{batch_id}, routed through pg-llm-batch and recordedin its cost ledger, returning the exact
{batch_id, status, embeddings, cost_micro_usd, token_counts}shape parsed here.service, team, group, company (on top of source/organization_id/user_id) --
resolved from new tenant-config columns
(
batch_attribution_service/team/group/company), never env;companyfallsback to
organization_id.backend/tests/test_batch_embeddings_contract.py):asserts the client serializes to exactly the path + payload and parses the exact
response, against a shared
backend/tests/fixtures/batch_embeddings_contract.jsonthat the orchestrator repo keeps a byte-identical copy of and asserts its
server against -- so the contract cannot silently drift again.
Scope: fuzzing suite removed
The bundled Hypothesis/Atheris fuzzing suite (
.github/workflows/fuzz.yml,backend/fuzz/*, itsemail_parser.pyhardening, and the fuzzing survey paper)is unrelated to batch routing and is covered by a separate dedicated fuzzing PR;
it has been removed so this PR stays scoped to batch embedding routing.