Skip to content

docs(#55): add OpenHands entry to landscape analysis - #7108

Merged
ralphbean merged 2 commits into
mainfrom
agent/55-add-openhands-landscape
Sep 8, 2026
Merged

docs(#55): add OpenHands entry to landscape analysis#7108
ralphbean merged 2 commits into
mainfrom
agent/55-add-openhands-landscape

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Adds an OpenHands section to docs/landscape.md under "Production agent orchestration systems," documenting the platform's architecture, licensing constraints, security history, and relevance to fullsend.

Related Issue

Closes #55

Changes

  • Added OpenHands entry to docs/landscape.md following the format of existing entries in the production agent orchestration systems section
  • Documented the PolyForm licensing constraint on the enterprise directory (only the MIT-licensed resolver and CLI are viable for open-source use)
  • Documented known prompt injection vulnerabilities (Johann Rehberger's 2025 disclosure of zero-click token exfiltration and RCE)
  • Analyzed relevance to fullsend across three dimensions: sandboxing model, injection surface, and event stream as audit trail
  • Cross-referenced issue Experiment: evaluate OpenHands resolver against fullsend problem space #260 for planned concrete experiments against the resolver

Testing

  • make lint passes (stage changes first, then run)
  • No tests needed — documentation-only change
  • Markdown link linting (lychee) passes with 0 errors
  • lint-docs-links passes

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits use docs type — documentation-only change does not appear in release notes

Closes #55

Post-script verification

  • Branch is not main/master (agent/55-add-openhands-landscape)
  • Secret scan passed (gitleaks — 695da0ac8c3f989dcf587e381ec7c5d83f541d23..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

Add an OpenHands section to docs/landscape.md in the production
agent orchestration systems category. Documents the platform's
architecture, the PolyForm licensing constraint on the enterprise
directory (only the MIT-licensed resolver and CLI are viable for
open-source use), known prompt injection vulnerabilities disclosed
by Johann Rehberger in 2025, and relevance to fullsend's problem
areas including sandboxing, injection defense, and observability.

Cross-references issue #260 which tracks concrete experiments
against the resolver.

Closes #55
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner September 8, 2026 17:24
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Sep 8, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Ended 5:26 PM UTC

Commit: 74a4d29 · View workflow run →

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Site preview

Preview: https://2de7ebee-site.fullsend-ai.workers.dev

Commit: 61ca2000ac3796248576c6176087859acb6e632d

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 8, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:27 PM UTC · Completed 5:43 PM UTC

Commit: 74a4d29 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.68

@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review fullsend-ai-review Bot added the risk/low PR risk: low label Sep 8, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 8, 2026

Copy link
Copy Markdown

Risk Assessment: low (1/5)

Details

Routine documentation-only PR by a bot adding 23 lines to a landscape analysis file, with no protected paths, no security or CI concerns, and an issue scope that directly matches the change; all signals unchanged from prior assessment, anchoring to prior score of 1.

Previous run

Risk Assessment: low (1/5)

Details

Routine documentation-only PR by a bot adding 22 lines to a landscape analysis file, with no protected paths, no security or CI concerns, and an issue scope that directly matches the change.

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 8, 2026

Copy link
Copy Markdown

Looks good to me

Previous run

Review

Findings

High

  • [factual error / stale claim] docs/landscape.md:177 — The Licensing section states the enterprise/ directory is licensed under PolyForm Free Trial and that only the MIT-licensed resolver and local CLI are viable without a commercial agreement. According to the updated triage evaluation in issue Explore OpenHands and evaluate relevance to fullsend #55 (September 8), licensing has been resolved — the entire project is now MIT-licensed and the enterprise/ directory no longer exists. This section contains a factual error that would mislead organizations evaluating OpenHands' licensing posture. The stale framing also propagates to line 189, where "MIT-licensed resolver" implies a licensing distinction that no longer exists.
    Remediation: Rewrite the Licensing paragraph to reflect the current MIT-only state. Remove the PolyForm description. Acknowledge that PolyForm was previously a constraint but the project has since relicensed fully to MIT. Remove the "MIT-licensed" qualifier from "MIT-licensed resolver" on line 189.

  • [missing material information] docs/landscape.md:175 — The Architecture section describes OpenHands as running agents inside Docker containers with its own event-driven loop but does not mention the Agent Canvas pivot or Agent Client Protocol (ACP) support. The updated triage evaluation identified this as the most significant architectural change: OpenHands now functions as a self-hosted developer control center that can run external coding agents (Claude Code, Codex, Gemini) via ACP, making it agent-agnostic at the coding-agent layer while still depending on its orchestration layer.
    Remediation: Update the Architecture section to document the ACP pivot and address the orchestration-vs-agent-runtime trade-off — ACP makes the platform agent-agnostic at the coding-agent layer, but adoption still requires the OpenHands orchestration model.

Medium

  • [incomplete analysis] docs/landscape.md:183 — The Relevance section and its three observations are based on the pre-ACP architecture and omit the orchestration-vs-agent-runtime trade-off identified by both the triage evaluation and a maintainer question in issue Explore OpenHands and evaluate relevance to fullsend #55. The existing observations about sandboxing, injection surface, and event stream remain valid but are incomplete — they miss the most strategically relevant question for fullsend: whether OpenHands' ACP support changes the over-specialization concern raised by a maintainer.
    Remediation: Add a fourth observation addressing the orchestration-vs-agent-runtime trade-off: ACP support means OpenHands is no longer locked to its own coding agent, but the dependency on the OpenHands Agent Server for orchestration remains.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 5:45 PM UTC · Completed 5:51 PM UTC

Commit: 74a4d29 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $1.02

Update the OpenHands landscape entry to reflect changes since the
original evaluation:

- Licensing: rewrite to reflect the project's current MIT-only license,
  noting the historical PolyForm constraint has been resolved and the
  enterprise/ directory removed.
- Architecture: document the Agent Canvas pivot and Agent Client
  Protocol (ACP) support, which makes the platform agent-agnostic at
  the coding-agent layer while still requiring the Agent Server for
  orchestration.
- Relevance: add a fourth observation covering the orchestration-vs-
  agent-runtime trade-off identified in issue #55's updated triage.
  Remove the now-unnecessary "MIT-licensed" qualifier from the resolver
  reference.

Addresses #7108
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (bot-triggered)

Updated the OpenHands landscape entry to reflect three significant changes identified in the review: (1) rewrote the Licensing section to reflect the project's current MIT-only license, (2) documented the Agent Canvas / ACP architectural pivot in the Architecture section, and (3) added a fourth relevance observation covering the orchestration-vs-agent-runtime trade-off. All changes are sourced from the updated triage evaluation in issue #55.

Fixed (3):

  1. Factual error / stale claim — Licensing section states enterprise/ directory is PolyForm Free Trial, but the project is now fully MIT-licensed (docs/landscape.md): Rewrote the Licensing paragraph to reflect the current MIT-only license state. Acknowledged the historical PolyForm constraint while noting it has been resolved and the enterprise/ directory removed.
  2. Missing material information — Architecture section omits Agent Canvas pivot and ACP support (docs/landscape.md): Added documentation of the Agent Canvas pivot and Agent Client Protocol (ACP) support to the Architecture section, noting the platform is now agent-agnostic at the coding-agent layer while still requiring the OpenHands Agent Server for orchestration.
  3. Incomplete analysis — Relevance section missing orchestration-vs-agent-runtime trade-off (docs/landscape.md): Added a fourth observation covering the orchestration-vs-agent-runtime trade-off: ACP makes OpenHands agent-agnostic at the coding layer but creates coupling at the orchestration layer. Updated count from 'Three' to 'Four' observations. Removed 'MIT-licensed' qualifier from resolver reference on the closing line.

Tests: passed

Decision points
  • Keep historical PolyForm context in Licensing section rather than removing it entirely (alternatives: Remove all PolyForm references, Keep historical context for evaluators; rationale: The historical context is useful for organizations that may have previously dismissed OpenHands due to the PolyForm constraint — knowing it was resolved helps them re-evaluate.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 8, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:53 PM UTC · Completed 6:06 PM UTC

Commit: 61ca200 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.09

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review September 8, 2026 18:05

Superseded by updated review

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Sep 8, 2026
@ralphbean
ralphbean added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit c590865 Sep 8, 2026
34 checks passed
@ralphbean
ralphbean deleted the agent/55-add-openhands-landscape branch September 8, 2026 19:44
@fullsend-ai-retro

fullsend-ai-retro Bot commented Sep 8, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:46 PM UTC · Completed 7:59 PM UTC

Commit: 61ca200 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.34

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #7108 — docs(#55): add OpenHands entry to landscape analysis

Timeline

  1. 17:08 — ralphbean triggered /fs-triage on issue #55 asking if it was still relevant
  2. 17:15 — Triage agent posted updated evaluation explicitly noting OpenHands is now MIT-only (PolyForm removed) and has pivoted to Agent Canvas / ACP
  3. 17:17 — Code agent started (run 34255993304)
  4. 17:22 — Code agent committed with stale PolyForm licensing info and no ACP mention — contradicting the triage findings posted 7 minutes earlier
  5. 17:24 — PR docs(#55): add OpenHands entry to landscape analysis #7108 opened; two review dispatches triggered (one cancelled, one succeeded)
  6. 17:43 — Review agent posted 3 findings (2 high, 1 medium): stale licensing, missing ACP architecture, incomplete analysis
  7. 17:44 — ralphbean approved the PR
  8. 17:45–17:49 — Fix agent (run 34258841800, $1.02) addressed all 3 findings
  9. 17:53 — Re-review (run 34259597440, $3.09) confirmed fixes
  10. 18:06requires-manual-review label added
  11. 19:44 — Merged by ralphbean

Workflow quality assessment

Review quality: Excellent. The review agent correctly identified all three factual errors the code agent introduced — stale PolyForm licensing, missing ACP/Agent Canvas architecture pivot, and an incomplete relevance analysis. These were genuine errors in the content, not false positives.

Fix agent: Efficient. All 3 findings addressed in a single commit at $1.02 cost. Re-review confirmed everything was resolved.

Code agent: Missed available context. The code-implementation skill Step 2 instructs the agent to read all issue comments and understand triage context. The triage evaluation was available 2 minutes before the code agent started, yet the code agent produced content with stale factual claims. This triggered an avoidable review→fix→re-review cycle costing $4.11.

Redundant dispatches: A second triage ($0.74) and second code run ($0.68, no PR created) were triggered by ralphbean's follow-up comment. Total redundant cost: ~$1.42.

Total pipeline cost: ~$12.02 (Triage $1.43, Code $2.80, Review $6.77, Fix $1.02)

Evidence for existing issues

  • #2131 (Code agent should receive triage output as context): This PR provides a new example — the triage output was available in issue comments but the code agent did not incorporate its factual findings about OpenHands' licensing and architecture changes.
  • #2185 (Code agent should follow triage strategy): The triage explicitly recommended documenting the ACP pivot, resolved MIT licensing, and orchestration trade-off, but the code agent produced content based on older information instead.
  • #7107 (Debounce review dispatch on push): The cancelled first review dispatch (run 34256911012) is consistent with the pattern described in Add configurable debounce delay before review agent dispatch on push events #7107.
  • #5289, #4751 (Autonomy readiness for docs review): The review agent's factual verification outperformed the code agent's output and caught errors the human approver did not flag before the fix landed. This is further evidence of strong review quality on docs PRs.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment risk/low PR risk: low

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore OpenHands and evaluate relevance to fullsend

1 participant