Skip to content

feat(terminal): remote path via aws ec2-instance-connect + pty - #1533

Merged
HongmingWang-Rabbit merged 4 commits into
mainfrom
feat/terminal-eic-remote
Apr 22, 2026
Merged

HongmingWang-Rabbit merged 4 commits into
mainfrom
feat/terminal-eic-remote

Conversation

@HongmingWang-Rabbit

Copy link
Copy Markdown
Contributor

Summary

Terminal tab now works for CP-provisioned workspaces. Closes #1528.

How it works

```
HandleConnect(ws)
├── SELECT instance_id FROM workspaces WHERE id = ws
├── instance_id empty → existing local Docker path (unchanged)
└── instance_id set → handleRemoteConnect
└── spawn: aws ec2-instance-connect ssh
--connection-type eice
--instance-id i-xxx
--os-user ec2-user
-- docker exec -it ws-yyy /bin/bash
└── wrap in pty (creack/pty) for TTY behavior
└── bridge pty ↔ canvas WebSocket
```

Design decisions

  • Subprocess out to aws-cli rather than native AWS SDK. EIC Endpoint tunnel uses a signed WebSocket with specific framing; aws-cli v2 implements it correctly. Reimplementing in Go is ~500 lines of crypto + WS protocol work for zero user-visible benefit. Tenant image gains ~1MB for aws-cli + openssh-client via apk.
  • sshCommandFactory is a var so tests can stub the command without spawning real aws-cli processes.
  • Context cancellation is bidirectional: WS close kills ssh; ssh exit closes WS.
  • Error message points at the design doc when EIC wiring is incomplete (common bootstrap failure mode for new tenants).

Blocked by

  • feat(workspace): persist CP-returned EC2 instance_id on provision #1531 (instance_id persistence) — needs to land + redeploy first so new CP workspaces carry the id
  • Your one-time infra: (a) IAM policy on `molecule-cp` with `ec2-instance-connect:SendSSHPublicKey` + `OpenTunnel` (condition `aws:ResourceTag/Role=workspace`); (b) one EIC Endpoint in the workspace VPC. See `docs/infra/workspace-terminal.md`.

Tests

  • `TestHandleConnect_RoutesToRemote` — instance_id in DB → CP branch (verified via sqlmock expectations)
  • `TestHandleConnect_RoutesToLocal` — empty instance_id → local Docker branch
  • `TestSshCommandFactory_BuildsEICCommand` — argv shape regression guard

All three pass. Pre-existing Platform Go failures (SSRF test regression from #1469) tracked in #1525; not caused by this change.

Post-merge verification (your environment)

  1. Pull new tenant image (auto ~5min after merge)
  2. Provision a fresh CP workspace → `SELECT instance_id FROM workspaces` non-null
  3. Open Terminal tab on that workspace → bash prompt
  4. Kill EIC Endpoint temporarily → Terminal shows the "check tenant aws CLI + IAM" hint

Hongming Wang and others added 4 commits April 21, 2026 17:56
Foundation for the EIC-based terminal handler (#1528). The tenant's
workspace-server needs to map workspace_id → EC2 instance_id to open
an SSH session, but CPProvisioner.Start returned the instance id only
for logging — it was never written anywhere. This PR adds the column
and writes it at provision time.

Scope kept intentionally small: no terminal code yet. The follow-up
PR will consume this column from the terminal handler.

What's here:
- migrations/038_workspace_instance_id — nullable TEXT column on
  workspaces, partial index on non-null for fast lookup
- workspace_provision.go — UPDATE after CPProvisioner.Start; failure
  logs but doesn't fail provisioning (row just lacks instance_id and
  terminal falls back to the existing not-reachable error)
- docs/infra/workspace-terminal.md — full design for the terminal
  flow: EIC vs SSM comparison, IAM policy JSON, SG rules, key
  lifetime, failure modes, rollout checklist

Refs: #1528
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Researched the actual molecule-controlplane repo rather than guessing:
- Workspaces launch in a shared CP workspace VPC (p.VPCID), not per
  tenant
- CP already tags instances with Role=workspace at ec2.go:1126 — my
  prior IAM policy used molecule:role which doesn't match anything
- workspaceIngressRules() currently opens only 8000/tcp — no port 22

Corrected:
- IAM policy Condition now matches existing Role tag (no CP change
  needed for the scope to work fleet-wide)
- Added OpenTunnel action so EIC Endpoint path works
- Dropped the \"open 22 in SG\" recommendation. Cross-VPC topology
  makes SG CIDR rules awkward (would need peering + tenant-CIDR
  bookkeeping). EIC Endpoint is one VPC resource + no SG changes.
- Simplified rollout to two items: add IAM policy, create EIC Endpoint

Kept direct-SG path as an explicit not-recommended alternative.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last CP-provisioned-workspace gap: Terminal tab now works
for workspaces running on separate EC2 instances. Follow-up to
#1531 which added instance_id persistence.

How it works:
- HandleConnect checks workspaces.instance_id
- Empty → existing local Docker path (unchanged)
- Set   → spawn `aws ec2-instance-connect ssh --connection-type eice
          --instance-id X --os-user ec2-user -- docker exec -it ws-Y
          /bin/bash` under creack/pty, bridge pty ↔ canvas WebSocket

Why subprocess AWS CLI instead of native AWS SDK:
- EIC Endpoint tunnel needs a signed WebSocket with specific framing
- aws-cli v2 implements it correctly; reimplementing in Go is ~500
  lines of crypto + WS protocol work for zero user-visible benefit
- Tenant image picks up 1MB of aws-cli + openssh-client via apk

Handler design:
- sshCommandFactory is a var so tests can stub it (no real aws calls)
- Context cancellation propagates both ways (WS close → kill ssh;
  ssh exit → close WS)
- User-visible error points at docs/infra/workspace-terminal.md when
  EIC wiring is incomplete (common bootstrap failure)

Tests:
- TestHandleConnect_RoutesToRemote — instance_id in DB → CP branch
- TestHandleConnect_RoutesToLocal — empty instance_id → local branch
- TestSshCommandFactory_BuildsEICCommand — argv shape regression guard

Dockerfile.tenant: + openssh-client + aws-cli (Alpine main repo)

Refs: #1528, #1531

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Proven by end-to-end testing against a live Hermes workspace EC2:
CP-provisioned workspaces run the agent as a NATIVE process under
the ubuntu user, not inside a Docker container. The earlier
\`aws ec2-instance-connect ssh -- docker exec -it ws-X bash\` was
doubly wrong:
- aws-cli's \`ssh\` subcommand doesn't accept a trailing command
- Even if it did, there's no container to exec into

Replaced with a three-step pipeline that matches what actually
works when run by hand:
1. ssh-keygen  — ephemeral ed25519 per session
2. aws ec2-instance-connect send-ssh-public-key --instance-os-user ubuntu
3. aws ec2-instance-connect open-tunnel --local-port N  (runs in background)
4. ssh -p N -i <key> ubuntu@127.0.0.1

Infra prerequisites (verified in docs/infra/workspace-terminal.md):
- EIC service-linked role created
- EIC Endpoint in the workspace VPC (we created eice-08b035ec8789202f9)
- Workspace SG allows 22/tcp from the EIC Endpoint's SG
- molecule-cp IAM: ec2:DescribeInstances + ec2-instance-connect:*

Changes in this commit:
- eicSSHOptions struct carries session inputs between factories
- openTunnelCmd + sshCommandCmd + sendSSHPublicKey are package vars
  so tests can stub them individually
- Default OS user is \"ubuntu\" (Ubuntu 24.04 CP AMI). Override via
  WORKSPACE_EC2_OS_USER env var if the AMI changes
- AWS_REGION env var respected; default us-east-2 matches current CP
- pickFreePort + waitForPort helpers — no hardcoded ports, tolerates
  multiple concurrent sessions
- Tests updated: two argv-shape regressions for open-tunnel + ssh
  (SSH shape was the silent-drift case that caused the first failure)

Refs: #1528, #1531
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@HongmingWang-Rabbit
HongmingWang-Rabbit merged commit 3820a0c into main Apr 22, 2026
9 of 11 checks passed
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1549. PR #1533 added remote terminal support for CP-provisioned workspaces using aws ec2-instance-connect — tutorial explains data flow, runnable snippet, and IAM verification checklist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1545. Runnable Python demo + README walkthrough for PR #1533 (EC2 Instance Connect Endpoint terminal support).
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1545. Runnable Python demo + README walkthrough for PR #1533 (EC2 Instance Connect Endpoint terminal support).
molecule-ai Bot pushed a commit that referenced this pull request Apr 22, 2026
Blog post covering the EIC Endpoint SSH integration shipped in PR #1533.
Covers: what it is, why it matters for DevOps teams, how it works (data flow
+ code examples), IAM configuration checklist, and getting-started steps for
hosted SaaS and self-hosted EC2 deployments. Closes #1546.
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Blog post covering the EIC Endpoint SSH integration shipped in PR #1533.
Covers: what it is, why it matters for DevOps teams, how it works (data flow
+ code examples), IAM configuration checklist, and getting-started steps for
hosted SaaS and self-hosted EC2 deployments. Closes #1546.

Co-authored-by: Molecule AI DevRel Engineer <devrel-engineer@agents.moleculesai.app>
molecule-ai Bot pushed a commit that referenced this pull request Apr 22, 2026
… Endpoint

Runnable tutorial for PR #1533:
- How EIC SSH bridges PTY to Canvas Terminal tab
- Prerequisites: IAM policy, EIC Endpoint, aws-cli in tenant image
- 6-step runnable snippet (workspace create → poll → Terminal verify → CloudWatch audit)
- Design notes: subprocess aws-cli pattern, bidirectional context cancel
- Teardown, links to social copy and infra runbook

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1549. PR #1533 added remote terminal support for CP-provisioned workspaces using aws ec2-instance-connect — tutorial explains data flow, runnable snippet, and IAM verification checklist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1545. Runnable Python demo + README walkthrough for PR #1533 (EC2 Instance Connect Endpoint terminal support).
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
Closes #1545. Runnable Python demo + README walkthrough for PR #1533 (EC2 Instance Connect Endpoint terminal support).
molecule-ai Bot added a commit that referenced this pull request Apr 22, 2026
…public IP (#1612)

* docs(social): EC2 Instance Connect SSH launch copy + terminal demo visual

PR #1533 (feat/terminal: remote path via aws ec2-instance-connect + pty)
Issue #1547 (social: launch thread for EC2 Instance Connect SSH)

Content:
- docs/marketing/social/2026-04-22-ec2-instance-connect-ssh/social-copy.md
  5-post X thread + LinkedIn single post, dark theme brand voice
- docs/assets/blog/2026-04-22-ec2-instance-connect-ssh/ec2-terminal-demo.png (1200x800)
  Canvas Terminal tab mockup showing EC2 bash prompt via EIC

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(blog): Phase 33 direct-connect migration — Cloudflare Tunnel to public IP

Migrate from Cloudflare Tunnel (outbound WebSocket) to direct-connect
agent workspaces with per-workspace public IPs. Covers operator actions,
developer notes, security model, and Phase 33 rollout timeline.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Molecule AI Social Media Brand <social-media-brand@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Molecule AI DevRel Engineer <devrel-engineer@agents.moleculesai.app>
molecule-ai Bot added a commit that referenced this pull request Apr 23, 2026
… Endpoint (#1617)

* docs(social): EC2 Instance Connect SSH launch copy + terminal demo visual

PR #1533 (feat/terminal: remote path via aws ec2-instance-connect + pty)
Issue #1547 (social: launch thread for EC2 Instance Connect SSH)

Content:
- docs/marketing/social/2026-04-22-ec2-instance-connect-ssh/social-copy.md
  5-post X thread + LinkedIn single post, dark theme brand voice
- docs/assets/blog/2026-04-22-ec2-instance-connect-ssh/ec2-terminal-demo.png (1200x800)
  Canvas Terminal tab mockup showing EC2 bash prompt via EIC

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(tutorial): EC2 Instance Connect SSH — workspace terminal via EIC Endpoint

Runnable tutorial for PR #1533:
- How EIC SSH bridges PTY to Canvas Terminal tab
- Prerequisites: IAM policy, EIC Endpoint, aws-cli in tenant image
- 6-step runnable snippet (workspace create → poll → Terminal verify → CloudWatch audit)
- Design notes: subprocess aws-cli pattern, bidirectional context cancel
- Teardown, links to social copy and infra runbook

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Molecule AI Social Media Brand <social-media-brand@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Molecule AI DevRel Engineer <devrel-engineer@agents.moleculesai.app>
molecule-ai Bot added a commit that referenced this pull request Apr 23, 2026
)

* docs(social): EC2 Instance Connect SSH launch copy + terminal demo visual

PR #1533 (feat/terminal: remote path via aws ec2-instance-connect + pty)
Issue #1547 (social: launch thread for EC2 Instance Connect SSH)

Content:
- docs/marketing/social/2026-04-22-ec2-instance-connect-ssh/social-copy.md
  5-post X thread + LinkedIn single post, dark theme brand voice
- docs/assets/blog/2026-04-22-ec2-instance-connect-ssh/ec2-terminal-demo.png (1200x800)
  Canvas Terminal tab mockup showing EC2 bash prompt via EIC

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(blog): AI agent credential model — one key, named, monitored

Companion post to the enterprise-key-management launch post.
Focuses on the agent-specific angle: dynamic tool interfaces,
emergent behavior containment, delegation chains, and the
security properties that survive agent compromise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Molecule AI Social Media Brand <social-media-brand@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Molecule AI DevRel Engineer <devrel-engineer@agents.moleculesai.app>
@HongmingWang-Rabbit
HongmingWang-Rabbit deleted the feat/terminal-eic-remote branch April 24, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant