Repository navigation
fix(orchestrator): require a real production service #458
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
seonghobae
merged 27 commits into
develop
from
fix/orchestrator-production-fail-closed-20260809
Aug 16, 2026
Merged
Changes from 14 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
c3def46
fix(orchestrator): fail closed outside explicit development mode
seonghobae 6525970
test(orchestrator): prove production never returns deterministic fake…
seonghobae 02d1bf6
test(coverage): include orchestrator production boundary
seonghobae 2b02f23
docs(orchestrator): define fail-closed learned-coordination boundary
seonghobae eb9435e
ci(orchestrator): verify fail-closed production behavior once
seonghobae cc60a85
docs(changelog): record orchestrator production boundary
seonghobae bdc81fa
ci: remove completed orchestrator one-shot writer
seonghobae af69b5f
test(orchestrator): reject oversized provider responses
seonghobae 6281964
fix(orchestrator): bound provider response bodies
seonghobae 75f73e4
merge develop: preserve security hardening in orchestrator PR
seonghobae cc76f59
test(orchestrator): enforce streaming response byte budget
seonghobae 7ea77f0
fix(orchestrator): bound provider response while streaming
seonghobae c5c890e
docs(orchestrator): record streaming response bound
seonghobae 1e6916a
test(orchestrator): expect explicit development adapter
seonghobae 4a806b5
test(orchestrator): cover residual production boundary branches
seonghobae d648c45
test(orchestrator): register residual branch coverage suite
seonghobae 766d577
test(orchestrator): cover IPv6 loopback transport
seonghobae 2a81fb4
fix(orchestrator): recognize IPv6 loopback hostname
seonghobae bd9bfe3
test(orchestrator): reject ambiguous provider endpoint configuration
seonghobae 268536f
fix(orchestrator): require a canonical provider origin
seonghobae 8c150a7
docs(orchestrator): document canonical provider origin
seonghobae 46723a0
test(orchestrator): classify provider rejection before response body
seonghobae 1578bf3
fix(orchestrator): classify provider rejection before body parsing
seonghobae b2f9639
test(api): isolate orchestrator mock from inherited provider config
seonghobae e482891
test(orchestrator): cover non-JSON response branch
seonghobae d57572a
test(orchestrator): require cancellation of rejected bodies
seonghobae dd85ee0
fix(orchestrator): cancel rejected provider response bodies
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # contextual-orchestrator Production Contract | ||
|
|
||
| ScopeWeave delegates AI briefing work to `contextual-orchestrator`; it does not | ||
| silently replace unavailable production inference with deterministic text. | ||
|
|
||
| ## Required environment | ||
|
|
||
| ```text | ||
| ORCHESTRATOR_URL=https://orchestrator.example | ||
| ORCHESTRATOR_TOKEN=<secret> | ||
| ORCHESTRATOR_MODEL=contextual-orchestrator | ||
| ``` | ||
|
|
||
| Production requests fail closed when the endpoint or bearer token is absent. | ||
| Non-loopback HTTP endpoints are rejected, requests are bounded to 120 seconds, | ||
| message count and content size are validated, provider payloads are not exposed | ||
| in errors, and an empty or malformed assistant response is never reported as a | ||
| successful briefing. | ||
|
|
||
| Provider response bodies have a hard 1 MiB caller-side byte budget **while they | ||
| are being read**. An oversized numeric `Content-Length` is rejected before body | ||
| allocation; when the header is absent or inaccurate, the stream reader counts | ||
| bytes incrementally, cancels the body as soon as the budget is exceeded, and | ||
| never buffers an unbounded provider payload before applying the limit. Empty, | ||
| non-stream-readable, malformed-length, non-JSON, oversized, or structurally | ||
| invalid responses fail with stable operator-safe errors rather than exposing | ||
| provider payload details. | ||
|
|
||
| The deterministic adapter is available only when `SCOPEWEAVE_DEV=1` and the | ||
| endpoint is absent. That variable must never be set in staging or production. | ||
|
|
||
| ## Orchestration responsibility | ||
|
|
||
| ScopeWeave intentionally sends only a versioned OpenAI-compatible request to | ||
| the orchestration service. Model selection, single-model versus multi-agent | ||
| allocation, task decomposition, role-specific reasoning effort, recursion | ||
| limits, access lists, synthesis, and verification belong to | ||
| `contextual-orchestrator`, where they can be evaluated and evolved centrally. | ||
| This separation is consistent with learned coordination research: Conductor | ||
| learns task decompositions, worker assignments, communication topologies, and | ||
| recursive test-time scaling; TRINITY adaptively assigns Thinker, Worker, and | ||
| Verifier roles over multiple turns; Fugu operationalizes learned orchestration | ||
| behind one model-compatible API. | ||
|
|
||
| ScopeWeave therefore does not hard-code a local fake solver, fixed topology, or | ||
| provider-specific bypass. Changes to orchestration policy require benchmarked | ||
| ablation evidence in `contextual-orchestrator`, including single-model, | ||
| parallel, sequential, hierarchical, recursive, and verifier-assisted paths. | ||
|
|
||
| ## APA 7th references | ||
|
|
||
| Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2025). | ||
| *Learning to orchestrate agents in natural language with the Conductor*. | ||
| arXiv. https://doi.org/10.48550/arXiv.2512.04388 | ||
|
|
||
| Sakana AI. (2026). *Sakana Fugu: Multi-agent system as a model*. | ||
| https://sakana.ai/fugu/ | ||
|
|
||
| Xu, J., Sun, Q., Schwendeman, P., Nielsen, S., Cetin, E., & Tang, Y. (2025). | ||
| *TRINITY: An evolved LLM coordinator*. arXiv. | ||
| https://doi.org/10.48550/arXiv.2512.04695 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,274 @@ | ||
| // contextual-orchestrator(LLM 오케스트레이션) 클라이언트. | ||
| // 실서버: ORCHESTRATOR_URL + ORCHESTRATOR_TOKEN 설정 시 OpenAI 호환 | ||
| // /v1/chat/completions 호출. 미설정 시 결정적 MOCK으로 전 플로우 테스트 가능. | ||
| // contextual-orchestrator client. Production requires an authenticated endpoint; | ||
| // deterministic responses exist only under the explicit SCOPEWEAVE_DEV=1 boundary. | ||
| const OC_URL = (process.env.ORCHESTRATOR_URL || '').replace(/\/$/, ''); | ||
| const OC_TOKEN = process.env.ORCHESTRATOR_TOKEN || ''; | ||
| const OC_MODEL = process.env.ORCHESTRATOR_MODEL || 'contextual-orchestrator'; | ||
| const ORCHESTRATOR_TIMEOUT_MS = 120_000; | ||
| const MAX_MESSAGE_COUNT = 256; | ||
| const MAX_CONTENT_LENGTH = 100_000; | ||
| const MAX_PROVIDER_RESPONSE_BYTES = 1024 * 1024; | ||
|
|
||
| export const orchestratorMock = !OC_URL; | ||
| export const orchestratorMock = process.env.SCOPEWEAVE_DEV === '1' && !OC_URL; | ||
|
|
||
| /** Stable provider-boundary failure for AI briefing requests. */ | ||
| export class OrchestratorConfigurationError extends Error { | ||
| /** | ||
| * Create one operator-safe orchestrator error. | ||
| * @param {string} code machine-readable failure code | ||
| * @param {string} message operator-safe detail | ||
| */ | ||
| constructor(code, message) { | ||
| super(message); | ||
| this.name = 'OrchestratorConfigurationError'; | ||
| this.code = code; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Resolve explicit development mode or a complete authenticated production endpoint. | ||
| * @returns {{mock: true} | {mock: false, baseUrl: string, token: string}} | ||
| */ | ||
| function orchestratorConfiguration() { | ||
| if (orchestratorMock) return { mock: true }; | ||
| if (!OC_URL) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_not_configured', | ||
| 'contextual-orchestrator is unavailable because ORCHESTRATOR_URL is not configured.', | ||
| ); | ||
| } | ||
| let url; | ||
| try { | ||
| url = new URL(OC_URL); | ||
| } catch { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_url_invalid', | ||
| 'ORCHESTRATOR_URL must be a valid absolute URL.', | ||
| ); | ||
| } | ||
| if (!['https:', 'http:'].includes(url.protocol)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_url_invalid', | ||
| 'ORCHESTRATOR_URL must use HTTP or HTTPS.', | ||
| ); | ||
| } | ||
| if (url.protocol !== 'https:' && !['localhost', '127.0.0.1', '::1'].includes(url.hostname)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_transport_insecure', | ||
| 'contextual-orchestrator production traffic requires HTTPS.', | ||
| ); | ||
| } | ||
| if (!OC_TOKEN.trim()) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_token_missing', | ||
| 'ORCHESTRATOR_TOKEN is required for production requests.', | ||
| ); | ||
| } | ||
| return { mock: false, baseUrl: url.toString().replace(/\/$/, ''), token: OC_TOKEN }; | ||
| } | ||
|
|
||
| /** | ||
| * Validate and copy OpenAI-compatible messages without accepting unbounded content. | ||
| * @param {unknown} messages candidate conversation | ||
| * @returns {{role: string, content: string}[]} | ||
| */ | ||
| function validatedMessages(messages) { | ||
| if (!Array.isArray(messages) || messages.length === 0 || messages.length > MAX_MESSAGE_COUNT) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_messages_invalid', | ||
| 'Orchestrator messages must be a non-empty bounded array.', | ||
| ); | ||
| } | ||
| return messages.map((message) => { | ||
| if (!message || typeof message !== 'object' || Array.isArray(message)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_message_invalid', | ||
| 'Each orchestrator message must be an object.', | ||
| ); | ||
| } | ||
| if (!['system', 'developer', 'user', 'assistant'].includes(message.role)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_message_role_invalid', | ||
| 'Orchestrator message role is unsupported.', | ||
| ); | ||
| } | ||
| if ( | ||
| typeof message.content !== 'string' | ||
| || message.content.length === 0 | ||
| || message.content.length > MAX_CONTENT_LENGTH | ||
| ) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_message_content_invalid', | ||
| 'Orchestrator message content is outside the accepted boundary.', | ||
| ); | ||
| } | ||
| return { role: message.role, content: message.content }; | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * Build the stable response-size failure used by declared and streamed limits. | ||
| * @returns {OrchestratorConfigurationError} Operator-safe size error. | ||
| */ | ||
| function responseSizeError() { | ||
| return new OrchestratorConfigurationError( | ||
| 'orchestrator_response_size_invalid', | ||
| 'contextual-orchestrator response size is outside the accepted boundary.', | ||
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Read one provider body without ever buffering more than the configured limit. | ||
| * | ||
| * A trustworthy numeric Content-Length can reject an oversized response before | ||
| * body allocation. The stream reader remains authoritative because providers | ||
| * may omit or misstate that header. The reader is cancelled as soon as the | ||
| * accumulated byte count exceeds the limit. | ||
| * | ||
| * @param {Response} response provider response | ||
| * @returns {Promise<Buffer>} Non-empty bounded response bytes. | ||
| */ | ||
| async function boundedResponseBytes(response) { | ||
| const declaredLength = response.headers?.get?.('content-length'); | ||
| if (declaredLength !== null && declaredLength !== undefined && declaredLength !== '') { | ||
| const normalizedLength = String(declaredLength).trim(); | ||
| if (!/^\d+$/.test(normalizedLength)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator returned an invalid response length.', | ||
| ); | ||
| } | ||
| const length = Number(normalizedLength); | ||
| if (!Number.isSafeInteger(length)) throw responseSizeError(); | ||
| if (length === 0 || length > MAX_PROVIDER_RESPONSE_BYTES) throw responseSizeError(); | ||
| } | ||
|
|
||
| const reader = response.body?.getReader?.(); | ||
| if (!reader || typeof reader.read !== 'function') { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator response body is not stream-readable.', | ||
| ); | ||
| } | ||
|
|
||
| const chunks = []; | ||
| let totalBytes = 0; | ||
| try { | ||
| for (;;) { | ||
| const { done, value } = await reader.read(); | ||
| if (done) break; | ||
| if (!(value instanceof Uint8Array)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator returned an invalid response chunk.', | ||
| ); | ||
| } | ||
| totalBytes += value.byteLength; | ||
| if (totalBytes > MAX_PROVIDER_RESPONSE_BYTES) { | ||
| try { | ||
| await reader.cancel(); | ||
| } catch { | ||
| // Cancellation is best effort after the byte budget has already failed closed. | ||
| } | ||
| throw responseSizeError(); | ||
| } | ||
| chunks.push(Buffer.from(value)); | ||
| } | ||
| } catch (error) { | ||
| if (error instanceof OrchestratorConfigurationError) throw error; | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator response could not be read.', | ||
| ); | ||
| } finally { | ||
| try { | ||
| reader.releaseLock?.(); | ||
| } catch { | ||
| // Releasing a consumed/cancelled reader is cleanup only and cannot alter the result. | ||
| } | ||
| } | ||
|
|
||
| if (totalBytes === 0) throw responseSizeError(); | ||
| return Buffer.concat(chunks, totalBytes); | ||
| } | ||
|
|
||
| /** | ||
| * Parse one bounded provider response without returning raw provider payloads in failures. | ||
| * @param {Response} response provider response | ||
| * @returns {Promise<Record<string, unknown>>} | ||
| */ | ||
| async function responseJson(response) { | ||
| const bytes = await boundedResponseBytes(response); | ||
| let data; | ||
| try { | ||
| data = JSON.parse(bytes.toString('utf8')); | ||
| } catch { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator returned a non-JSON response.', | ||
| ); | ||
| } | ||
| if (!data || typeof data !== 'object' || Array.isArray(data)) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator returned an invalid response object.', | ||
| ); | ||
| } | ||
| return data; | ||
| } | ||
|
|
||
| /** | ||
| * Generate one AI briefing through contextual-orchestrator. | ||
| * @param {unknown} messages OpenAI-compatible messages | ||
| * @returns {Promise<string>} | ||
| */ | ||
| export async function chat(messages) { | ||
| if (orchestratorMock) { | ||
| const user = messages.filter((m) => m.role === 'user').map((m) => m.content).join('\n'); | ||
| return `[mock-orchestrator] 분석 요약: ${user.slice(0, 120)}…에 대한 모의 응답입니다. ` | ||
| const configuration = orchestratorConfiguration(); | ||
| const safeMessages = validatedMessages(messages); | ||
| if (configuration.mock) { | ||
| const user = safeMessages | ||
| .filter((message) => message.role === 'user') | ||
| .map((message) => message.content) | ||
| .join('\n'); | ||
| return `[dev-orchestrator] 분석 요약: ${user.slice(0, 120)}…에 대한 개발 응답입니다. ` | ||
| + '리스크: 지연 작업을 우선 점검하세요. 권고: 임계경로 작업의 담당자 부하를 재배분하세요.'; | ||
| } | ||
| const ctrl = new AbortController(); | ||
| const to = setTimeout(() => ctrl.abort(), 60000); | ||
| if (typeof globalThis.fetch !== 'function') { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_transport_unavailable', | ||
| 'Orchestrator HTTP transport is unavailable.', | ||
| ); | ||
| } | ||
|
|
||
| let response; | ||
| try { | ||
| const res = await fetch(`${OC_URL}/v1/chat/completions`, { | ||
| response = await globalThis.fetch(`${configuration.baseUrl}/v1/chat/completions`, { | ||
| method: 'POST', | ||
| headers: { | ||
| 'content-type': 'application/json', | ||
| ...(OC_TOKEN ? { authorization: `Bearer ${OC_TOKEN}` } : {}), | ||
| authorization: `Bearer ${configuration.token}`, | ||
| }, | ||
| // orchestrator는 알 수 없는 필드를 거부(strict validation) — model+messages만 전송. | ||
| body: JSON.stringify({ model: 'contextual-orchestrator', messages }), | ||
| signal: ctrl.signal, | ||
| body: JSON.stringify({ model: OC_MODEL, messages: safeMessages }), | ||
| signal: AbortSignal.timeout(ORCHESTRATOR_TIMEOUT_MS), | ||
| }); | ||
| const data = await res.json().catch(() => ({})); | ||
| const content = data?.choices?.[0]?.message?.content; | ||
| if (!res.ok || !content) throw new Error(data?.error?.message || `orchestrator failed (${res.status})`); | ||
| return content; | ||
| } finally { | ||
| clearTimeout(to); | ||
| } catch { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_provider_unavailable', | ||
| 'contextual-orchestrator could not be reached.', | ||
| ); | ||
| } | ||
| const data = await responseJson(response); | ||
| const content = data?.choices?.[0]?.message?.content; | ||
| if (!response.ok) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_provider_rejected', | ||
| `contextual-orchestrator rejected the request with HTTP ${response.status}.`, | ||
| ); | ||
| } | ||
| if (typeof content !== 'string' || !content.trim()) { | ||
| throw new OrchestratorConfigurationError( | ||
| 'orchestrator_response_invalid', | ||
| 'contextual-orchestrator returned no assistant content.', | ||
| ); | ||
| } | ||
| return content; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.