Skip to content

feat(workers-dev): persistent engine connection + TUI overhaul - #368

Merged
andersonleal merged 9 commits into
mainfrom
feat/workers-dev-tui
Jun 29, 2026
Merged

feat(workers-dev): persistent engine connection + TUI overhaul#368
andersonleal merged 9 commits into
mainfrom
feat/workers-dev-tui

Conversation

@andersonleal

@andersonleal andersonleal commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Builds on the dev TUI introduced in #311. All changes are isolated to workers-dev/ — no other crate is touched.

Engine connection & process lifecycle

  • Replace the per-poll iii trigger subprocess with one persistent IIIClient (lazy OnceCell, reused for every poll). This stops the engine register/unregister storm that fired every tick.
  • Spawn workers in their own process group and group-kill on stop, so killing cargo run no longer orphans the worker binary it forked.

TUI overhaul

  • Two-column master/detail layout on wide terminals: content-fit worker list on the left, the selected worker's logs flexing to fill the right. Falls back to a vertical stack below ~100 cols. +/- drags the divider (or resizes log height when stacked).
  • Scrollable per-worker logs through the full ring buffer, with ▶ live / ⏸ scrolled follow state; name filter (/); at-a-glance health summary (●◐✗○) and ⚠ unreachable engine banner.
  • Event-driven redraw loop (off-thread engine polling via watch, action errors surfaced in a footer banner).

Polish

  • Honest process states: external (installed via iii worker add) and elsewhere (engine-connected but started outside this tool) instead of a contradictory stopped + connected.
  • Theme-agnostic muted text via Modifier::DIM on the default foreground (readable on both light and dark terminals).
  • Restart confirmation shows the dependent blast radius.

SDK

  • Import WorkerMetadata / IIIConnectionState from the canonical iii_sdk::runtime (matching all sibling workers), not the legacy iii_sdk::iii re-export.

Builds clean (cargo build, clippy — no issues, cargo fmt --check), cargo test — 14 passed. Design-critique trend across iterations: 29 → 31 → 32 → 33 / 40.

Summary by CodeRabbit

  • New Features

    • Added a more responsive dashboard with improved log following, scrolling, resizing, filtering, and clearer keybindings.
    • Worker status now shows clearer process states and crash exit codes.
  • Bug Fixes

    • Improved engine connection handling so status and logs surface reachability issues more clearly.
    • Stopping workers now better terminates related child processes to avoid orphans.
  • Documentation

    • Updated the README with revised worker status meanings and expanded TUI usage notes.

…rkers

The TUI polled engine status by spawning `iii trigger engine::workers::list`
every poll tick; each spawn opened a one-shot WebSocket the engine logged as a
worker register/unregister pair, flooding engine logs every ~2s. Replace it
with one persistent iii-sdk connection (labeled `workers-dev`), lazily opened
and reused for every poll, so the engine sees one register at connect and one
unregister at clean exit.

Also fixes several issues found while reviewing the worker:

- Run each worker in its own process group and group-kill (SIGTERM then
  SIGKILL) on stop. `cargo run` forks the worker binary as a child, so killing
  only cargo orphaned the worker (it kept running, still connected). This also
  repairs --stop-on-exit, which was silently broken.
- Poll the engine off the UI thread via a watch channel so a slow or
  unreachable engine can't freeze keyboard input; shorten the query timeout to
  3s.
- Surface "engine unreachable" in the TUI header and `status` output instead
  of silently rendering every worker as disconnected.
- Serialize TUI start/stop/restart via an in-flight counter so fast keypresses
  can't interleave actions on the same worker.
- `workers-dev up` now boots `iii -c harness/engine.config.yaml` (detached)
  when the engine isn't reachable.
…olish

Reworks the dashboard TUI and fixes DX gaps found dogfooding it:

- Scrollable in-place log pane with a follow toggle (PgUp/PgDn, f); no longer
  ejects the dashboard to tail logs.
- At-a-glance health summary in the header (connected/compiling/crashed/stopped)
  and a `?` overlay with the full key reference.
- Filter workers by name with `/` (group headers kept only for matching groups).
- Inline crash exit codes in the worker table.
- Event-driven rendering: redraw only on input, a new snapshot, a spinner tick,
  or a live-log refresh — idle no longer repaints continuously.
- Animated spinner for compiling workers; resizable log/table split (+/-).
- Restart confirm dialog lists the dependents that will also restart.
- Worker-action failures now surface as a footer banner instead of eprintln,
  which was lost or garbled under the alt-screen.
- Repaint on terminal resize (the event-driven loop had ignored resize events).

Drops the per-worker log tail from the poll snapshot (logs are fetched on
demand for the selected worker), making each status poll cheaper.
- Cap the log pane so the worker list (primary content) stays visible: on
  short terminals it no longer starves the list to ~2 rows while the log pane
  sits empty; the table flexes to fill on tall terminals.
- Footer adapts to width in three tiers, always keeping `? keys · q quit`,
  which previously truncated off at narrow widths.
- Content-fit column widths so Process/Engine/PID/Uptime align in a tight
  cluster instead of floating across the row behind a wide Uptime column.
- "No workers match …" placeholder when a filter yields nothing.
- Status-glyph legend (connected/compiling/crashed/stopped) in the ? overlay.
- Drop the duplicated "workers-dev" header box title.
The Worker column was 38 wide only to hold two things stuffed into the name
cell, which pushed every worker's status far to the right. Move both into the
Process column where management/process state belongs:

- Non-startable workers show "external" (was a repeated "(iii worker add)"
  suffix on the name). This also resolves the "Process: stopped / Engine:
  connected" contradiction — a connected unmanaged worker now reads
  "external / connected".
- A crash shows "exit N" in the Process column (was trailing the name).

The name cell is now just glyph + name, so the Worker column shrinks to 24 and
the status block sits right next to the name. At narrow widths all five
columns now fit. The ? overlay gains an "external" legend line.
Rework the dashboard into a two-column layout on wide terminals: the
worker list (content-fit at 66 cols) on the left, the selected worker's
logs flexing to fill the rest on the right. Below ~103 cols the two panes
stack vertically as before, so neither is crushed. Header and footer span
full width; modals re-anchor to the body so they keep their width in
either layout.

Critique-driven polish on top of the rework:
- Process column shows "elsewhere" when a worker is engine-connected but
  not spawned here, instead of the contradictory "stopped" + "connected".
- +/- drags the column divider in two-column mode (50..66) and still
  resizes the log height when stacked.
- 1-col gutter between panes so their borders no longer fuse into a seam.
- Muted text uses ANSI gray (7) instead of bright-black (8), which themes
  define with readable contrast.
Muted styles now apply Modifier::DIM to the default foreground instead of
a fixed ANSI gray, so they de-emphasize correctly on both light- and
dark-background terminals. A single canonical muted_cell_style() backs the
rest (footer, hints, "external"/"elsewhere", "—", stopped, Other header).
…k::runtime

WorkerMetadata and IIIConnectionState were imported via the legacy
iii_sdk::iii re-export; 0.20 canonicalizes them under iii_sdk::runtime
(where all sibling workers already import them). No behavior change.
@vercel

vercel Bot commented Jun 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
workers Ready Ready Preview, Comment Jun 29, 2026 3:50pm
workers-tech-spec Ready Ready Preview, Comment Jun 29, 2026 3:50pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

skill-check — worker

0 verified, 29 skipped (no docs/).

Layer Result
structure
vale
ai
render

Four for four. Nicely done.

@coderabbitai

coderabbitai Bot commented Jun 29, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@andersonleal, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 39691af8-6b61-4ce9-a39c-eefa26450ef9

📥 Commits

Reviewing files that changed from the base of the PR and between 574ca62 and b3ab1cb.

📒 Files selected for processing (5)
  • workers-dev/README.md
  • workers-dev/src/config.rs
  • workers-dev/src/logs.rs
  • workers-dev/src/orchestrator.rs
  • workers-dev/src/tui/mod.rs
📝 Walkthrough

Walkthrough

Introduces a persistent IIIClient connection in Orchestrator replacing per-query subprocess spawning, switches Unix worker processes to their own process groups with SIGTERM/SIGKILL termination, removes engine_host/engine_port from Config, adds exit_code to WorkerView, and rewrites the TUI into a watch-channel-driven, responsive master/detail dashboard.

Changes

workers-dev: persistent engine client, process-group lifecycle, and TUI overhaul

Layer / File(s) Summary
Config and WorkerView data shape changes
workers-dev/Cargo.toml, workers-dev/src/config.rs, workers-dev/src/status.rs
Adds iii-sdk/libc dependencies; removes engine_host/engine_port from Config, adds ENGINE_QUERY_TIMEOUT_MS, canonicalizes engine_url; adds exit_code: Option<i32> to WorkerView and removes engine_pid/last_logs.
Persistent IIIClient and engine worker querying
workers-dev/src/orchestrator.rs, workers-dev/src/status.rs, workers-dev/src/main.rs
Orchestrator::new accepts progress: bool and stores a OnceCell<IIIClient>; engine_client() lazily registers/connects; shutdown() closes it; fetch_engine_workers switches from spawning iii trigger subprocess to an in-process client.trigger call; main passes progress and calls orchestrator.shutdown().
Unix process-group start/stop and wait_connected
workers-dev/src/orchestrator.rs, workers-dev/src/runtime.rs
start_one() calls process_group(0) on Unix; stop_one() uses new terminate_process_group (SIGTERM→SIGKILL with polling on Unix, kill fallback otherwise); wait_connected() polls via shared client and emits detailed exit-code error messages; runtime adds PID semantics docs.
dashboard_snapshot, build_view, and command wiring
workers-dev/src/orchestrator.rs, workers-dev/src/commands/mod.rs, workers-dev/src/status.rs
dashboard_snapshot returns (Vec<WorkerView>, Option<String>) tuple; build_view sets exit_code/uptime instead of engine_pid/last_logs; commands updated for ensure_engine, deduplicated log tail printing, and explicit shutdown.
Worker discovery and log rendering refactors
workers-dev/src/discover.rs, workers-dev/src/logs.rs
WorkerGroup derives PartialOrd/Ord; name-mismatch warns+skips instead of bailing; sorting uses derived order; log_line_to_ratatui uses truncate_chars; print_colored_line inlines classification (removes write_crossterm_colored_line).
TUI state model, run() event loop, and key handlers
workers-dev/src/tui/mod.rs
Adds DashboardState, UiCtx, Actions (atomic in-flight counter + mpsc errors), UiMode variants; run() refactored to background watch-channel poller + non-blocking keyboard loop; handle_dashboard_key, handle_filter_key, handle_confirm_key, spawn_action helpers, and build_display_rows with name filtering.
TUI rendering: layout, table, logs, footer, overlays, and theme
workers-dev/src/tui/mod.rs, workers-dev/src/tui/theme.rs, workers-dev/README.md
draw_ui implements responsive two-column/stacked layout; draw_header shows health summary; status_icon uses cycling spinner glyph; draw_log_pane handles follow/scroll; draw_footer shows transient error banner or width-tiered help; overlays rewritten as centered modals; muted styles switch to Modifier::DIM; README documents new keys and process status labels.
Formatting-only changes
workers-dev/src/graph.rs
Reformatting of error messages, return expressions, and test fixture in graph.rs with no behavioral changes.

Sequence Diagram(s)

sequenceDiagram
  participant main
  participant Orchestrator
  participant IIIClient
  participant Engine
  participant TUI

  main->>Orchestrator: new(config, progress=true)
  Orchestrator->>IIIClient: OnceCell init (lazy)
  main->>Orchestrator: ensure_engine()
  Orchestrator->>IIIClient: engine_client() → register + wait Connected
  IIIClient->>Engine: WebSocket connect
  Engine-->>IIIClient: Connected
  main->>TUI: run()
  loop background poller
    Orchestrator->>IIIClient: trigger engine::workers::list
    IIIClient->>Engine: TriggerRequest
    Engine-->>IIIClient: EngineWorkersResponse
    Orchestrator-->>TUI: watch channel DashboardState{views, engine_error}
  end
  TUI->>Orchestrator: spawn_start / spawn_stop / spawn_restart
  Orchestrator->>IIIClient: start_one() process_group(0)
  Orchestrator->>IIIClient: terminate_process_group() SIGTERM→SIGKILL
  main->>Orchestrator: shutdown()
  Orchestrator->>IIIClient: close connection
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • iii-hq/workers#311: Introduced the initial workers-dev TUI, orchestrator, status, and log rendering that this PR substantially refactors across src/tui/mod.rs, src/orchestrator.rs, src/status.rs, src/logs.rs, and src/tui/theme.rs.

Suggested reviewers

  • sergiofilhowz
  • ytallo

Poem

🐇 Hoppity-hop through the process tree,
No orphaned workers shall roam wild and free!
One client persists where subprocesses sprawled,
The TUI spins with each frame it's called.
SIGTERM, then SIGKILL — a tidy goodbye,
The dashboard now dances, responsive and spry!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 64.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the two main changes: a persistent engine connection and a major TUI overhaul.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/workers-dev-tui

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
workers-dev/src/status.rs (1)

71-87: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

exit_code is still invisible in workers-dev status.

build_view() now fills WorkerView.exit_code, but this formatter still prints only process/engine/PID/uptime. On the CLI status path, crashed workers therefore still lose the exit-code detail this field was added to expose inline.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@workers-dev/src/status.rs` around lines 71 - 87, The workers-dev status
formatter in build_view()/the status printing block is still omitting the new
exit_code field, so crashed worker details are not shown inline. Update the
println! output for WorkerView to include exit_code alongside the existing
WORKER, PROCESS, ENGINE, PID, and UPTIME columns, and adjust the header/format
string in the same status rendering path so the value from WorkerView.exit_code
is visible when present.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@workers-dev/README.md`:
- Around line 38-39: The README text in the `workers-dev` section is
inconsistent with the current TUI status labels from `process_cell()`. Update
the sentence describing workers connected to the engine so it uses the same
`Process: external` wording as the rest of the section, and make sure the
description for non-Rust workers installed via `iii worker add` matches the new
`external` label everywhere.

In `@workers-dev/src/config.rs`:
- Around line 71-78: In Config::load, the engine URL canonicalization is always
rebuilding the URL as ws:// even when parse_engine_url() parsed a secure wss://
endpoint. Preserve the original scheme when reconstructing engine_url by
carrying the parsed scheme through parse_engine_url() (or equivalent helper) and
using it in the final format so CLI/config TLS endpoints remain wss:// after
load.

In `@workers-dev/src/discover.rs`:
- Around line 77-82: The skip-on-mismatch behavior in discover logic is
incomplete because explicit worker loading still hard-fails later. Update the
worker loading path in WorkerGraph::load() and/or Config::load() so a stale
workers entry whose iii.worker.yaml name does not match the folder is handled
consistently with discover.rs: emit the mismatch warning and skip that worker
instead of aborting startup. Keep the check aligned with the existing
name-versus-folder comparison used in the discovery code so both auto-discovered
and explicitly listed workers follow the same “skip and continue” behavior.

In `@workers-dev/src/logs.rs`:
- Around line 75-86: The timestamp rendering in
`split_tracing_timestamp`/`Line::from` is allowing one extra visible column
because the separator `" "` is always added even when `truncate_chars(ts,
max_width)` already fills the width. Update this branch so the separator and
remaining text are only appended when there is space left, and compute the
remaining width from the actual rendered timestamp length to keep the total
within `max_width`.

In `@workers-dev/src/orchestrator.rs`:
- Around line 324-365: The wait_connected method is masking engine polling
failures by turning engine_workers() errors into an empty list, which makes
start/restart wait until timeout and misreport the worker as the problem. Update
orchestrator::wait_connected to preserve and propagate the last engine query
error instead of defaulting to no workers, and make the timeout/bail path
mention that engine polling failed if that is what happened. Keep the existing
connected and crashed/stopped checks intact, but ensure the shared client/query
failure from engine_workers() is surfaced clearly.

In `@workers-dev/src/tui/mod.rs`:
- Around line 446-450: The KeyCode::Char('x') handler in
workers-dev/src/tui/mod.rs currently allows stopping any row with a worker_name,
including external and elsewhere entries that have no local process to
terminate. Update the x-action handling in the TUI event logic to gate stop
behavior on rows that are actually owned and spawned by workers-dev, and keep
the action disabled or ignored for external/elsewhere rows. Use the existing
UiMode, worker_name, and spawn_stop flow to locate the check and make the stop
path conditional on the row’s ownership state.
- Around line 906-910: The empty-state text in the `total == 0` branch of
`mod.rs` is using the wrong message when `selected_name` is `None`. Update the
logic around this `lines: Vec<Line>` construction to distinguish between
“selected worker with no output” and “no worker selected because the filter
removed everything,” and render a different hint for the no-selection case. Keep
the existing selected-worker message only when `selected_name` is present.

---

Outside diff comments:
In `@workers-dev/src/status.rs`:
- Around line 71-87: The workers-dev status formatter in build_view()/the status
printing block is still omitting the new exit_code field, so crashed worker
details are not shown inline. Update the println! output for WorkerView to
include exit_code alongside the existing WORKER, PROCESS, ENGINE, PID, and
UPTIME columns, and adjust the header/format string in the same status rendering
path so the value from WorkerView.exit_code is visible when present.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8822cc69-1734-4b89-ba6c-68db4c7e213b

📥 Commits

Reviewing files that changed from the base of the PR and between cc9d4d2 and 574ca62.

⛔ Files ignored due to path filters (1)
  • workers-dev/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (13)
  • workers-dev/Cargo.toml
  • workers-dev/README.md
  • workers-dev/src/commands/mod.rs
  • workers-dev/src/config.rs
  • workers-dev/src/discover.rs
  • workers-dev/src/graph.rs
  • workers-dev/src/logs.rs
  • workers-dev/src/main.rs
  • workers-dev/src/orchestrator.rs
  • workers-dev/src/runtime.rs
  • workers-dev/src/status.rs
  • workers-dev/src/tui/mod.rs
  • workers-dev/src/tui/theme.rs

Comment thread workers-dev/README.md
Comment thread workers-dev/src/config.rs Outdated
Comment thread workers-dev/src/discover.rs
Comment thread workers-dev/src/logs.rs
Comment thread workers-dev/src/orchestrator.rs
Comment thread workers-dev/src/tui/mod.rs
Comment thread workers-dev/src/tui/mod.rs Outdated
- config: preserve wss:// scheme when canonicalizing engine_url
- config: skip (warn) configured workers missing/mismatched in discovery
  instead of aborting startup later in WorkerGraph::load
- logs: stop a max-width timestamp overflowing the log pane by one column
- orchestrator: surface engine-query failures in wait_connected timeout
- tui: only allow stop on workers with a live local process
- tui: distinct empty-log hint when no worker is selected
- README: use the "Process: external" label the TUI actually renders
@andersonleal
andersonleal merged commit 28fa6f1 into main Jun 29, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants