-
Notifications
You must be signed in to change notification settings - Fork 1
fix(ai): rebase adaptive defaults onto current main #106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
3b14387
fix(ai): rebase adaptive defaults onto current main
cursoragent 75d6fd8
test(ai): lock auto and verify orchestrator transport contracts
cursoragent 7b6eeb8
test(ai): require payload literals for auto and verify scans
cursoragent dc0c62f
docs: record that orchestrator verify is still undeployed
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| """LineageWeave delegates product-default LLM execution to auto policy.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from pathlib import Path | ||
|
|
||
| from lineageweave import adjudication_client, post_chat, post_evaluation | ||
| from lineageweave.post_chat import ChatSourceDocument, ContextualOrchestratorPostChatClient | ||
|
|
||
|
|
||
| def test_post_evaluation_adapter_defaults_to_auto(monkeypatch) -> None: | ||
| observed: dict[str, object] = {} | ||
|
|
||
| def fake_post_json(url, payload, *, headers, timeout): | ||
| observed.update( | ||
| url=url, | ||
| payload=payload, | ||
| headers=headers, | ||
| timeout=timeout, | ||
| ) | ||
| return {"choices": [{"message": {"content": "{}"}}]} | ||
|
|
||
| monkeypatch.setattr(post_evaluation, "post_json", fake_post_json) | ||
| adapter = post_evaluation._OrchestratorCompleteAdapter( | ||
| "https://orchestrator.example.test", "inference_token" | ||
| ) | ||
| adapter.complete([{"role": "user", "content": "Evaluate this evidence."}]) | ||
|
|
||
| assert observed["payload"]["mode"] == "auto" | ||
|
|
||
|
|
||
| def test_post_evaluation_judge_uses_auto_by_default() -> None: | ||
| client = post_evaluation.ContextualOrchestratorPostEvaluationClient( | ||
| "https://orchestrator.example.test", "inference_token" | ||
| ) | ||
| assert client._judge.mode == "auto" | ||
|
|
||
|
|
||
| def test_post_chat_requests_verify_mode(monkeypatch) -> None: | ||
| """Citation chat must send verify on the wire, not a docstring mention of auto.""" | ||
|
|
||
| observed: dict[str, object] = {} | ||
|
|
||
| def fake_post_json(url, payload, *, headers, timeout): | ||
| observed["payload"] = payload | ||
| return { | ||
| "choices": [ | ||
| { | ||
| "message": { | ||
| "content": ( | ||
| '{"answer_text": "The follow-up names the same bid.",' | ||
| ' "cited_source_numbers": [1]}' | ||
| ) | ||
| } | ||
| } | ||
| ] | ||
| } | ||
|
|
||
| monkeypatch.setattr(post_chat, "post_json", fake_post_json) | ||
| client = ContextualOrchestratorPostChatClient( | ||
| "https://orchestrator.example.test", "inference_token" | ||
| ) | ||
| answer = client.answer( | ||
| "What happened between these events?", | ||
| [ | ||
| ChatSourceDocument( | ||
| post_id="post-bid-follow-up", | ||
| post_title="Bid follow-up", | ||
| post_body="Northridge asked to confirm the bid date.", | ||
| ) | ||
| ], | ||
| ) | ||
|
|
||
| assert answer.cited_post_ids == ("post-bid-follow-up",) | ||
| assert observed["payload"]["mode"] == "verify" | ||
|
|
||
|
|
||
| def test_adjudication_requests_verify_mode(monkeypatch) -> None: | ||
| """Lineage adjudication must send verify on the wire, not a source substring.""" | ||
|
|
||
| observed: dict[str, object] = {} | ||
|
|
||
| def fake_post_json(url, payload, *, headers, timeout): | ||
| observed["payload"] = payload | ||
| return {"choices": [{"message": {"content": "0.91"}}]} | ||
|
|
||
| monkeypatch.setattr(adjudication_client, "post_json", fake_post_json) | ||
| client = adjudication_client.ContextualOrchestratorAdjudicationClient( | ||
| "https://orchestrator.example.test", "inference_token" | ||
| ) | ||
| confidence = client.judge( | ||
| "Quarterly budget review meeting notes", | ||
| "Budget review follow-up: revised quarterly numbers", | ||
| ) | ||
|
|
||
| assert confidence == 0.91 | ||
| assert observed["payload"]["mode"] == "verify" | ||
|
|
||
|
|
||
| def test_runtime_clients_do_not_force_single_model_route() -> None: | ||
| package_root = Path(__file__).resolve().parents[1] / "lineageweave" | ||
| violations: list[str] = [] | ||
| for path in sorted(package_root.glob("*.py")): | ||
| text = path.read_text(encoding="utf-8") | ||
| if '"mode": "route"' in text or "'mode': 'route'" in text: | ||
| violations.append(f"{path.name}: request payload") | ||
| if 'mode="route"' in text or "mode='route'" in text: | ||
| violations.append(f"{path.name}: constructor/call default") | ||
| if 'mode: str = "route"' in text or "mode: str = 'route'" in text: | ||
| violations.append(f"{path.name}: typed default") | ||
| assert violations == [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| """Contract tests for adaptive contextual-orchestrator consumer defaults.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from pathlib import Path | ||
| import unittest | ||
|
|
||
| ROOT = Path(__file__).resolve().parents[1] | ||
| AUTO_CLIENTS = ( | ||
| "lineageweave/post_summary.py", | ||
| "lineageweave/post_evaluation.py", | ||
| "lineageweave/keyman_extraction.py", | ||
| "lineageweave/commitment_extraction.py", | ||
| "lineageweave/entity_relationship_classification.py", | ||
| ) | ||
| VERIFY_CLIENTS = ( | ||
| "lineageweave/post_chat.py", | ||
| "lineageweave/adjudication_client.py", | ||
| ) | ||
| _ROUTE_MARKERS = ( | ||
| '"mode": "route"', | ||
| 'mode="route"', | ||
| 'mode: str = "route"', | ||
| ) | ||
| _PAYLOAD_AUTO = '"mode": "auto"' | ||
| _PAYLOAD_VERIFY = '"mode": "verify"' | ||
| _TYPED_AUTO_DEFAULT = 'mode: str = "auto"' | ||
| _FORWARDED_MODE = '"mode": mode' | ||
|
|
||
|
|
||
| def _source(relative: str) -> str: | ||
| return (ROOT / relative).read_text(encoding="utf-8") | ||
|
|
||
|
|
||
| class AdaptiveOrchestratorDefaultTest(unittest.TestCase): | ||
| """Protect production clients from regressing to forced one-model routing.""" | ||
|
|
||
| def test_auto_clients_request_auto_and_never_force_route(self) -> None: | ||
| for relative in AUTO_CLIENTS: | ||
| source = _source(relative) | ||
| with self.subTest(path=relative): | ||
| for marker in _ROUTE_MARKERS: | ||
| self.assertNotIn(marker, source) | ||
| if relative.endswith("post_evaluation.py"): | ||
| self.assertIn(_FORWARDED_MODE, source) | ||
| self.assertIn(_TYPED_AUTO_DEFAULT, source) | ||
| continue | ||
| self.assertIn( | ||
| _PAYLOAD_AUTO, | ||
| source, | ||
| f"{relative} must send a payload-level mode=auto literal", | ||
| ) | ||
|
|
||
| def test_verify_clients_keep_checked_judgment_and_never_force_route(self) -> None: | ||
| for relative in VERIFY_CLIENTS: | ||
| source = _source(relative) | ||
| with self.subTest(path=relative): | ||
| for marker in _ROUTE_MARKERS: | ||
| self.assertNotIn(marker, source) | ||
| self.assertIn( | ||
| _PAYLOAD_VERIFY, | ||
| source, | ||
| f"{relative} must send a payload-level mode=verify literal", | ||
| ) | ||
| self.assertNotIn( | ||
| _PAYLOAD_AUTO, | ||
| source, | ||
| f"{relative} must send verify, not a payload-level auto default", | ||
| ) | ||
|
|
||
| def test_docstring_mode_mention_is_not_a_payload_literal(self) -> None: | ||
| """A class docstring contrast must not satisfy the auto/verify scan.""" | ||
|
|
||
| source = ( | ||
| '"""Calls the orchestrator with mode="auto", not mode="verify"""\n' | ||
| 'body = {"messages": [], "mode": "route"}\n' | ||
| ) | ||
| self.assertNotIn(_PAYLOAD_AUTO, source) | ||
| self.assertNotIn(_PAYLOAD_VERIFY, source) | ||
| self.assertIn('mode="auto"', source) | ||
| self.assertIn('mode="verify"', source) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
Repository: ContextualWisdomLab/LineageWeave
Length of output: 1846
🏁 Script executed:
Repository: ContextualWisdomLab/LineageWeave
Length of output: 3932
verify지원 배포를 먼저 제공하십시오.upstream PR
ContextualWisdomLab/contextual-orchestrator#149는 병합되지 않고 종료되었습니다. 따라서 현재route_and_verify를 포함하는 upstream 커밋을 고정할 수 없습니다. 두 클라이언트는mode="verify"를 전송하므로 현재 배포에서 citation chat과 lineage adjudication이invalid_mode로 실패합니다.verify를 지원하는 upstream 버전을 배포하고 버전을 고정한 후 이 계약을 활성화하십시오.autofallback은 사용하지 마십시오.🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in
dc0c62fby correcting the stale “#149 is still open” note.This repo cannot deploy or pin contextual-orchestrator. It is an HTTP service (
ORCHESTRATOR_BASE_URL), not a package. #149 closed unmerged; verify-mode work continues on contextual-orchestrator#622 and is not on deployed orchestratormain(invalid_modestill accepts onlyauto/route/conduct).LineageWeave keeps the explicit ADR-0013
verifycontract for citation chat and lineage adjudication. Noautofallback.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.