Skip to content

docs(#482): add OpenAI Symphony to landscape analysis - #7173

Merged
ralphbean merged 3 commits into
mainfrom
agent/482-symphony-landscape
Sep 9, 2026
Merged

docs(#482): add OpenAI Symphony to landscape analysis#7173
ralphbean merged 3 commits into
mainfrom
agent/482-symphony-landscape

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Adds OpenAI Symphony to the landscape analysis as a production agent orchestration system, with reciprocal pointers from the agent-architecture and agent-infrastructure problem docs.

Symphony is treated as a study reference, not an adoption candidate: a central daemon with implementation-defined trust, cwd-discipline isolation, and single-agent self-review, opposite fullsend's repo-as-coordinator, sandbox-as-containment, and decomposed independent review.

Related Issue

#482 — Research: Evaluate OpenAI Symphony against fullsend's architecture

Changes

  • New OpenAI Symphony entry in docs/landscape.md (production agent orchestration systems), covering coordination, trust/security, isolation vs per-repo autonomy, proof of work, governance, and implementation reusability
  • Comparison table of Symphony vs fullsend direction
  • New architectural pattern: central scheduler daemon with a language-agnostic spec
  • Reciprocal backlink in docs/problems/agent-architecture.md (coordinator contrast, next to Forge and Gas City)
  • Reciprocal backlink in docs/problems/agent-infrastructure.md (SSH worker extension as a remote-execution data point)

The write-up is pinned to openai/symphony@8001b52 (2026-08-12) and reflects current tracker adapters (Linear, GitHub Issues, Jira Cloud, Asana, GitLab), host-side tracker credential stripping, and collision-resistant workspace keys — not the Linear-only April 2026 research snapshot.

Borrowable pieces called out (non-blocking follow-ups, not implemented here): workspace safety invariants, continuation-turn semantics, SSH worker scheduling, and token-accounting rules.

Testing

  • lychee --offline --include-fragments passes on the full markdown corpus (0 errors)
  • hack/lint-docs-links passes on changed files
  • gitleaks / scan-secrets pass on changed files
  • make lint (pre-commit) could not fetch remote hook repos in this sandbox (HTTP 403); equivalent local markdown hooks were run instead
  • No production code changes; no new tests required

Checklist

  • PR title follows Conventional Commits (docs(#482): ...)
  • Commits are signed off (DCO) — human and human-directed agent sessions only (autonomous agent commits are exempt)
  • I wrote this contribution myself and can explain all changes in it

Closes #482

Post-script verification

  • Branch is not main/master (agent/482-symphony-landscape)
  • Secret scan passed (gitleaks — d5f36921ac754705619f38c637ef692873809fbc..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

Add OpenAI Symphony to the production agent orchestration section of
landscape.md, with reciprocal pointers from agent-architecture.md and
agent-infrastructure.md.

Symphony is a study reference, not an adoption candidate: a central
daemon with implementation-defined trust, cwd-discipline isolation, and
single-agent self-review, opposite fullsend's repo-as-coordinator,
sandbox-as-containment, and decomposed independent review.

The entry is pinned to symphony@8001b52 (2026-08-12) and reflects
current tracker adapters (Linear, GitHub Issues, Jira, Asana, GitLab),
host-side tracker credential stripping, and collision-resistant
workspace keys — not the Linear-only April 2026 snapshot.

Borrowable pieces called out: workspace safety invariants,
continuation-turn semantics, SSH worker scheduling, and token-accounting
rules.

Note: pre-commit could not fetch remote hook repos (HTTP 403). Equivalent
local checks were run: gitleaks, lychee --offline --include-fragments,
hack/lint-docs-links, EOF/trailing-whitespace/private-key scans.

Closes #482
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner September 9, 2026 17:17
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Sep 9, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:19 PM UTC · Completed 5:36 PM UTC

Commit: ebb68ec · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $4.09

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review fullsend-ai-review Bot added the risk/low PR risk: low label Sep 9, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

Risk Assessment: low (1/5)

Details

Re-review anchoring: Tier 1 signals unchanged from prior (docs-only, 3 files, ~78 lines, no protected/security/CI/dependency paths, same bot author), and this commit is again a minor correction-only delta (retargeted cross-reference, reworded sentence) fixing two more review findings on an already-reviewed PR. Tier 2 shows landscape.md as an active, expected churn hotspot for a living catalog document, already priced into the prior score. Tier 3 remains clean. Composite preserves the prior score of 1 (low).

Previous run

Risk Assessment: low (1/5)

Details

Re-review anchoring: Tier 1 signals are unchanged from the prior assessment (docs-only, no protected/security/CI/dependency paths, same non-first-time bot author, small diff), and the current diff is a minor correction-only delta to docs/landscape.md (citation fixes, a count fix, one added bullet). Tier 2 shows landscape.md as an active churn hotspot, expected for a living catalog document and already reflected in the prior low score. Tier 3 shows the PR appropriately closes issue #482 with no unresolved risk labels or discussion. Weighted composite rounds to 1 (low), preserving the prior score.

Previous run (2)

Risk Assessment: low (1/5)

Details

Docs-only PR (no protected paths, no security-sensitive files, no CI/dependency changes) by a non-first-time bot author, adding a small 75-line research entry; the only elevating signal is routine churn on the shared landscape.md catalog file, which is expected for a living document and does not offset otherwise uniformly low signals.

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

Looks good to me

Previous run

Review

Findings

Medium

  • [design-doc-misalignment] docs/landscape.md:276 — Symphony's continuation-turn semantics are mapped to the wrong fullsend open question. The bullet describes intra-run thread reuse (first turn = full prompt; continuation turns = continuation guidance on the same thread; single subprocess across turns) and maps it to agent-architecture.md's open question "Should agents be stateless or stateful?" That open question (agent-architecture.md:214) is specifically "stateless (fresh context per task) or stateful (accumulated knowledge of the codebase)" — cross-task/codebase memory, not multi-turn execution within a single issue run. Treating continuation-turn semantics as bearing on that question is a category error.
    Remediation: Retarget the mapping — describe continuation-turns as intra-run session/thread reuse (fresh subprocess per turn vs. one thread for the issue) as a distinct axis, not as resolving or informing the accumulated-codebase-knowledge open question. (Note: cross-run-memory.md would also be the wrong retarget, since that doc concerns lessons persisting across separate runs, not intra-run thread continuity.)

Low

  • [internal-inconsistency] docs/landscape.md:619 — Pattern 7's summary ("Forge, Gas City, and Kiro Crew all have a controller process; ... The trade-off is portability versus a single attack/failure target, and implementations that put scheduling judgment in the controller rather than in repo-visible rules.") places the "scheduling judgment in the controller" trade-off immediately after naming Gas City alongside Forge and Kiro Crew. This reads as attributing controller-side judgment to all three, which contradicts the Symphony entry's own distinction a few sections earlier: Gas City enforces Zero Framework Cognition so its controller contains zero judgment, while Symphony's orchestrator visibly contains judgment (priority sort, blocker rules, reconciliation).
    Remediation: Reword so the judgment-in-controller trade-off is attributed only to Symphony (and any other cognitive controllers), e.g.: "Forge, Gas City, and Kiro Crew also have a controller process; Gas City's still forbids judgment in it (Zero Framework Cognition). Symphony's addition is shipping the architecture as a conformance spec, at the cost of a cognitively active scheduler."

Other notes

  • All three prior-review findings (medium incorrect-citation at docs/landscape.md:248/262, low internal-inconsistency "Five distinct approaches" at :586, low internal-inconsistency missing DoS bullet at :250) were verified fixed: citations now correctly point to ADR 0025 (extending ADR 0017), the count reads "Seven distinct approaches," and a Denial of Service / Resource Exhaustion bullet was added. No re-raise.
  • A sub-agent flagged that this PR's new Symphony entry substantively engages docs/problems/security-threat-model.md and docs/problems/autonomy-spectrum.md at a depth comparable to agent-architecture.md and agent-infrastructure.md (which did get reciprocal backlinks), but does not add backlinks to the former two. On review, AGENTS.md's reciprocal-backlink rule is scoped to new files added under docs/problems/ — this PR adds a section to the existing docs/landscape.md, so the rule does not require it, and existing landscape convention doesn't backlink every cited problem doc either. Noted for awareness only, not a blocking finding (consistent with the prior review's assessment of the same point).
  • Risk assessment (re-review anchored): low (1/5) — unchanged from the prior review. Docs-only correction diff (citation/count/bullet fixes) to an already-low-risk PR; no protected, security-sensitive, CI, or dependency paths touched.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (2)

Review

Findings

Medium

  • [incorrect-citation] docs/landscape.md:262 (also line 248) — The new OpenAI Symphony entry attributes a "four-tier credential delivery model" to ADR 0017 in two places. ADR 0017 establishes a two-tier model (prefetch/post-process default, host-side REST fallback); it is ADR 0025 ("Provider-based credential delivery for sandboxed agents") that explicitly adopts the four-tier model as an extension of ADR 0017. The "preferred tiers" framing on line 262 is also ADR 0025 language.
    Remediation: Cite ADR 0025 as the source of the four-tier model at both line 248 and line 262, e.g. "the four-tier credential delivery in ADR 0025 (extending ADR 0017)".

Low

  • [internal-inconsistency] docs/landscape.md:586 — The "Architectural patterns in the field" intro still reads "Five distinct approaches:" although the section already had 6 numbered patterns before this PR; adding pattern 7 here widens the stale count to off-by-two.
    Remediation: Change "Five distinct approaches:" to "Seven distinct approaches:" (or drop the numeral).

  • [internal-inconsistency] docs/landscape.md:250 — The stated threat priority "external injection > insider > DoS > drift > supply chain" correctly matches the current 5-item ranking in docs/problems/security-threat-model.md (DoS is Threat 6, priority docs: Add codebase context problem document and trim CLAUDE.md #3), but the per-threat bullet analysis right below it (lines 252-256) only covers Threats 5, 1, 2, 3, and 4 — there's no Denial of Service / Resource Exhaustion bullet even though DoS is named in the ranking one line above.
    Remediation: Add a bullet analyzing Symphony against Threat 6 (DoS / Resource Exhaustion), or drop "DoS" from the line-250 parenthetical so it matches what's actually analyzed.

Other notes

  • This sandbox has no outbound network access, so factual claims about the external openai/symphony repository (pinned commit, SPEC.md section contents, star/fork counts, WORKFLOW.md defaults, etc.) could not be independently verified against the live source. Everything checkable against the fullsend repository itself (internal links/anchors, ADR content, threat-model content, doc structure/style, scope vs. the linked issue) was verified and is otherwise consistent — no broken links or anchors were found, style matches existing landscape.md entries, and the write-up's scope matches issue Research: Evaluate OpenAI Symphony against fullsend's architecture #482.
  • A sub-agent also flagged that this PR adds reciprocal backlinks to two of the several docs/problems/ files the new entry substantively engages (agent-architecture.md, agent-infrastructure.md) but not others (e.g. security-threat-model.md, autonomy-spectrum.md). On review this isn't required by AGENTS.md's cross-referencing rule, which targets new problem docs, not landscape entries — noted for awareness only, not included as a blocking finding.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 5:37 PM UTC · Completed 5:44 PM UTC

Commit: ebb68ec · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $0.82

…cape entry

Addresses review feedback on PR #7173:
- Attribute the four-tier credential delivery model to ADR 0025 (which
  introduces it) rather than ADR 0017 (which defines the two-tier model
  ADR 0025 extends), at both citation sites in the OpenAI Symphony entry
- Update "Five distinct approaches" to "Seven" now that this PR's new
  pattern #7 brings the numbered list to seven entries
- Add a Denial of Service / Resource Exhaustion bullet to the Symphony
  threat-model mapping so the per-threat analysis covers every threat
  named in the priority ranking one line above it

Addresses #7173
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (bot-triggered)

Fixed all three review findings in the OpenAI Symphony landscape entry: corrected the ADR citation for the four-tier credential delivery model (ADR 0025, not ADR 0017) at both occurrences, updated the stale 'Five distinct approaches' count to 'Seven' to match the newly added seventh pattern, and added the missing Denial of Service / Resource Exhaustion bullet to the Symphony threat-model mapping. No disagreements — all three findings were verified correct against the repository's ADRs and threat-model doc. Verified with scan-secrets, pre-commit-equivalent checks (end-of-file/whitespace/merge-conflict/private-key/symlinks), hack/lint-docs-links, and a full-corpus lychee link check (0 errors).

Fixed (3):

  1. incorrect-citation: four-tier credential delivery model attributed to ADR 0017 instead of ADR 0025 (docs/landscape.md): At both line 248 and line 262, changed the citation of the four-tier credential delivery model from ADR 0017 (which actually defines a two-tier model) to ADR 0025 (which introduces the four-tier model as an extension of ADR 0017), phrased as "the four-tier credential delivery in ADR 0025 (extending ADR 0017)". Verified against both ADR files: ADR 0017's Status line says its tiers are 'extended by ADR 0025', and ADR 0025's Decision section states 'Adopt a four-tier credential delivery model, extending ADR 0017's two-tier credential delivery model.'
  2. internal-inconsistency: 'Five distinct approaches' stale count in 'Architectural patterns in the field' (docs/landscape.md): Changed 'Five distinct approaches:' to 'Seven distinct approaches:'. Verified by counting the numbered '### N.' headers in that section: there are 7 (the new OpenAI Symphony entry added as docs: Add contributor guidance problem domain #7).
  3. internal-inconsistency: DoS named in threat-priority parenthetical but not analyzed in the per-threat bullet list (docs/landscape.md): Added a new bullet 'Denial of Service / Resource Exhaustion.' analyzing Symphony against Threat 6, placed after the existing Supply chain bullet, so all five threats named in the priority ranking (external injection > insider > DoS > drift > supply chain) are covered. Cited the same already-referenced SPEC Section 7 (orchestrator concurrency limits) and linked to security-threat-model.md's Threat 6 anchor, verified to exist.

Tests: passed

Decision points
  • For the DoS finding, added the missing analysis bullet rather than dropping 'DoS' from the threat-priority parenthetical (alternatives: Add a bullet analyzing Symphony against Threat 6, Remove 'DoS' from the priority-ranking parenthetical on line 250; rationale: Adding the bullet preserves the accurate threat-priority statement (DoS genuinely is priority docs: Add codebase context problem document and trim CLAUDE.md #3 per security-threat-model.md) and keeps the per-threat analysis complete, which is more informative than shortening the ranking to avoid the mismatch.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:45 PM UTC · Completed 6:00 PM UTC

Commit: 565d0f1 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $4.20

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 6:02 PM UTC · Completed 6:08 PM UTC

Commit: 565d0f1 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $0.82

…ttribution

Retargets the continuation-turn semantics bullet away from the
stateless-vs-stateful open question (that question is about cross-task/
codebase memory, not intra-run multi-turn execution) and describes it as
its own intra-run session-reuse axis instead. Rewords the Pattern 7 summary
so the "judgment in the controller" trade-off is attributed to Symphony
specifically rather than implied for Forge, Gas City, and Kiro Crew
collectively, consistent with the Symphony entry's own Gas City contrast.

Addresses #7173

Note: pre-commit could not fetch remote hook repos in this sandbox
(HTTP 403); ran the equivalent local hooks directly instead
(lint-broken-symlinks, lint-docs-links, lychee offline/include-fragments)
plus manual trailing-whitespace/EOF checks - all passed.
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 2 (bot-triggered)

Addressed both current-iteration review findings: fixed a category-error mapping of Symphony's continuation-turn semantics to the wrong fullsend open question, and reworded the Pattern 7 summary so the controller-judgment trade-off is attributed only to Symphony, not implied for Forge/Gas City/Kiro Crew collectively. Both prior-iteration findings (credential-tier citation, stale pattern count, missing DoS bullet) were already fixed in a previous commit and verified by the reviewer as resolved, so no further action was needed on those. The non-blocking backlink observation was reiterated by the reviewer as informational only and required no change.

Fixed (2):

  1. design-doc-misalignment: continuation-turn semantics mismapped to the stateless-vs-stateful open question (docs/landscape.md:276) (docs/landscape.md): Reworded the continuation-turn semantics bullet to describe it as its own intra-run session-reuse axis (fresh subprocess per turn vs. one thread for the life of an issue run) instead of mapping it to agent-architecture.md's stateless-vs-stateful open question, which is about cross-task/codebase memory, not intra-run multi-turn execution. Did not retarget to cross-run-memory.md per the reviewer's explicit note that this would also be a category error.
  2. internal-inconsistency: Pattern 7 summary implies the controller-judgment trade-off applies to Forge, Gas City, and Kiro Crew collectively (docs/landscape.md:619) (docs/landscape.md): Reworded the Pattern 7 summary so the judgment-in-controller trade-off is attributed specifically to Symphony, and explicitly notes Gas City's Zero Framework Cognition still forbids judgment in its controller, consistent with the earlier Symphony-vs-Gas-City contrast in the same document.

Tests: passed

Decision points
  • Did not add a reciprocal backlink retarget to cross-run-memory.md for the continuation-turn semantics bullet (alternatives: Map continuation-turn semantics to cross-run-memory.md instead of agent-architecture.md's open question, Describe continuation-turn semantics as a standalone axis with no cross-reference to either doc's memory framing; rationale: The reviewer explicitly flagged that cross-run-memory.md would also be a wrong retarget, since it concerns lessons persisting across separate runs, not intra-run thread continuity. Chose to describe it as its own distinct axis without a forced mapping.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 6:09 PM UTC · Completed 6:27 PM UTC

Commit: f243475 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $3.14

@fullsend-ai-review
fullsend-ai-review Bot dismissed stale reviews from themself September 9, 2026 18:27

Superseded by updated review

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Sep 9, 2026
@ralphbean
ralphbean enabled auto-merge September 9, 2026 18:47
@ralphbean
ralphbean added this pull request to the merge queue Sep 9, 2026
Merged via the queue into main with commit f0eb1a2 Sep 9, 2026
53 of 54 checks passed
@ralphbean
ralphbean deleted the agent/482-symphony-landscape branch September 9, 2026 18:58
@fullsend-ai-retro

fullsend-ai-retro Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:00 PM UTC · Completed 7:09 PM UTC

Commit: f243475 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.40

@fullsend-ai-retro

Copy link
Copy Markdown

PR #7173 (docs(#482): add OpenAI Symphony to landscape analysis) closed issue #482 cleanly via triage → code → 3 review/fix cycles → human approval, merged in ~2 hours. Review quality was good: across two review passes the bot caught five real, substantive-to-nit findings (a wrong ADR citation attributing the four-tier credential model to ADR 0017 instead of 0025, a threat listed in the priority list but not analyzed, a stale 'five approaches' count, a claim mismapped to the wrong open-question category, and an internal contradiction about Gas City's controller having 'judgment'). The fix agent resolved all five with no disagreements, and round 3 correctly self-scoped by declining to add backlinks outside AGENTS.md's stated rule. The one human action beyond re-triggering triage was a same-day approval after the bot had already approved a low-risk (1/5) docs-only change — an appropriate confirmation, not a rubber stamp devoid of value. Two systemic patterns surfaced here are already well tracked elsewhere, so I'm recording them as evidence rather than new proposals: (1) a transient GitHub 422 when posting a file-level comment whose line fell outside the diff hunk — the designed fallback (fold into review body) worked correctly, and this is the same root cause tracked by fullsend-ai/agents#193 and the open #6039; this run is one more data point that the underlying diff-hunk validation gap (agents#430/#699/#760, blocked on fullsend#2569) is still open. (2) Total workflow cost was ~$20 (triage $0.44, code $6.68, three review passes at $4.09+$4.20+$3.14, two fix passes at $0.82 each) for a pure-markdown, non-security, RICE-0.4 change — consistent with the still-open fullsend-ai/agents#373 ('skip irrelevant review dimensions for docs-only PRs') and #5298 (docs sub-agent budget prioritization); no new issue needed. I also checked for and found no gap in risk-assessment anchoring (score stayed correctly pinned at low/1 across all three re-reviews, unlike the anomaly agents#1037 recently addressed). Given the two observed friction points are already tracked by open issues, and fullsend#5817 flags an existing glut of narrow duplicate 'skip retro' issues to avoid compounding, I'm filing no new proposals for this run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch risk/low PR risk: low

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Research: Evaluate OpenAI Symphony against fullsend's architecture

1 participant