Skip to content

perf(client): reduce remote request and message sync overhead - #11029

Merged
juliusmarminge merged 3 commits into
pingdotgg:mainfrom
Bil0000:t3code/remote-sync-performance
Sep 11, 2026
Merged

perf(client): reduce remote request and message sync overhead#11029
juliusmarminge merged 3 commits into
pingdotgg:mainfrom
Bil0000:t3code/remote-sync-performance

Conversation

@Bil0000

@Bil0000 Bil0000 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Remote connection setup and snapshot requests rebuild handlers for the entire HTTP API, even when they need one group. Each streamed message update also scans the message list twice. Both costs run in the shared client runtime used by web, desktop, and mobile.

This PR makes two small changes:

  • Use the existing Effect HttpApiClient.group to build only the handlers needed by authentication, environment metadata, and snapshot requests. Keep request validation, transport context, authorization refresh, and error mapping in place.
  • Apply message updates in one native array pass. Preserve message order, unchanged object references, streamed/final text handling, and attachment updates.

Three focused commits separate HTTP processing, message updates, and the benchmark. No new dependency, response cache, wire format, or connection policy.

Benchmarks

Actual before/after client functions; milliseconds, lower is better. The HTTP transport returns fixture responses in memory, so these measure client processing, including response validation, rather than server or network latency. The message cases apply 200 deltas to the last message in the loaded history through the production reducer.

Workload Chrome before Chrome after Less time Node before → after
Read connection descriptor 6.186 0.570 90.8% 9.489 → 0.325
Issue WebSocket ticket 5.738 1.992 65.3% 6.490 → 1.830
Load a snapshot with 100 messages 11.634 4.642 60.1% 7.701 → 1.871
Apply 200 deltas, 100 loaded messages 0.982 0.604 38.5% 0.363 → 0.244
Apply 200 deltas, 1,000 loaded messages 3.522 1.868 47.0% 3.772 → 2.122

Chrome values are medians of nine alternating before/after batches, 50 operations per batch, after 100 warm-up operations per version. Node values are Vitest sample medians with a 1-second warm-up and 1.5-second measurement per case. Timing varied substantially between runs; these results do not establish end-to-end latency. Network bytes, GPU load, memory use, and battery drain were not measured.

The benchmark at packages/client-runtime/src/remotePerformance.bench.ts runs the five Node cases. To reproduce, copy that file to the baseline (b7b3ef1e6fcb5c22a9790d2578fe8af7ce396835) and run from packages/client-runtime:

# Baseline checkout, with the benchmark copied in
vp test bench src/remotePerformance.bench.ts --run --outputJson /tmp/t3-remote-before.json

# PR checkout
vp test bench src/remotePerformance.bench.ts --run --compare /tmp/t3-remote-before.json --outputJson /tmp/t3-remote-after.json

Verification

  • 192 focused tests passed across authorization, connection resolution, RPC sessions, HTTP error handling, message updates, shell sync, thread sync, and pagination. Added checks for malformed responses and updates at each message position without input mutation.
  • 10,000 deterministic comparisons of the old and new message reducer produced identical outputs, including duplicate IDs, streaming/final text, attachments, and unchanged message references.
  • Client-runtime and web typechecks, targeted lint/format checks, and the production web build passed.
  • Verified remote pairing, saved message loading, switching away and back without another snapshot request, and reconnecting after a remote server restart. The client reconnected, received the synchronization marker, preserved the messages, and resumed without another snapshot request. No browser page errors.

Native desktop/mobile sessions and a physical WAN were not exercised. Their shared runtime is covered by the tests above; app-wide regression freedom is not claimed.

Model: GPT-6. Harness: Codex.

Summary by CodeRabbit

  • Refactor

    • Updated remote environment request handling across authorization, session, pull request, shell snapshot, and thread operations.
    • Preserved existing endpoints, payloads, headers, timeouts, and response behavior.
    • Streamlined thread message updates while maintaining unchanged messages and correctly appending new ones.
  • Tests

    • Expanded coverage for invalid authentication responses and thread message updates.
  • Performance

    • Added benchmarks for remote environment operations and thread event processing.
  • User Impact

    • No user-facing behavior changes are expected.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Sep 10, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This performance refactor changes shared remote HTTP and authentication code, including a file under the authorization directory. Despite its narrow scope and added tests, authentication-directory changes require human review.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 195d0dc1-22c4-4692-b0ee-c6557c6e6a54

📥 Commits

Reviewing files that changed from the base of the PR and between addfb13 and 4d0bd4e.

📒 Files selected for processing (12)
  • packages/client-runtime/src/authorization/remote.ts
  • packages/client-runtime/src/environment/descriptor.ts
  • packages/client-runtime/src/remotePerformance.bench.ts
  • packages/client-runtime/src/rpc/http.ts
  • packages/client-runtime/src/state/environmentHttpAuth.test.ts
  • packages/client-runtime/src/state/environmentHttpAuth.ts
  • packages/client-runtime/src/state/pullRequestDiffHttp.ts
  • packages/client-runtime/src/state/session.ts
  • packages/client-runtime/src/state/shellSnapshotHttp.ts
  • packages/client-runtime/src/state/threadReducer.test.ts
  • packages/client-runtime/src/state/threadReducer.ts
  • packages/client-runtime/src/state/threadSnapshotHttp.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The client runtime now uses grouped environment HTTP API clients across remote authorization, metadata, session, orchestration, and pull request requests. The thread reducer uses a single-pass message upsert. Tests and benchmarks cover the updated request handling and reducer behavior.

Changes

Environment HTTP group migration

Layer / File(s) Summary
Group client factory
packages/client-runtime/src/rpc/http.ts
Adds makeEnvironmentHttpApiGroupClient for constructing clients for specific EnvironmentHttpApi groups.
Authenticated request flow
packages/client-runtime/src/state/environmentHttpAuth.ts, packages/client-runtime/src/authorization/remote.ts, packages/client-runtime/src/state/environmentHttpAuth.test.ts
Makes authenticated requests generic over the selected group. Remote token, session, and WebSocket-ticket operations use the grouped auth client. Invalid JSON responses are tested for all authenticated loaders.
Environment request callers
packages/client-runtime/src/environment/descriptor.ts, packages/client-runtime/src/state/session.ts, packages/client-runtime/src/state/shellSnapshotHttp.ts, packages/client-runtime/src/state/threadSnapshotHttp.ts, packages/client-runtime/src/state/pullRequestDiffHttp.ts
Selects metadata, auth, orchestration, or pullRequests groups and invokes API methods directly on the grouped client.

Thread reducer and performance validation

Layer / File(s) Summary
Thread message upsert
packages/client-runtime/src/state/threadReducer.ts, packages/client-runtime/src/state/threadReducer.test.ts
Replaces the previous find-and-map upsert with one map pass and verifies targeted updates, appends, and reference identity of unchanged messages.
Runtime benchmarks
packages/client-runtime/src/remotePerformance.bench.ts
Adds benchmarks for remote HTTP operations and text-delta application on threads with 100 and 1,000 messages.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 4d0bd

This change reduces remote request and message-update processing overhead while preserving existing request and message-update behavior. No current merge-blocking risk remains.

Suggested reviewers: juliusmarminge, t3dotgg

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 12 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the primary changes: reduced remote request and message synchronization overhead.
Description check ✅ Passed The description provides detailed change rationale, benchmark results, verification steps, scope, and limitations. It omits the template headings and checklist, but the required information is otherwi…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@juliusmarminge
juliusmarminge merged commit 6e8931d into pingdotgg:main Sep 11, 2026
22 checks passed
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Sep 11, 2026
## What's Changed
* fix(mobile): prevent Hermes crashes when opening threads by @jakeleventhal in pingdotgg/t3code#11233
* feat(web): open Usage on the Limits tab by default by @juliusmarminge in pingdotgg/t3code#11261
* perf(web): avoid scanning chat history for sidebar backgrounds by @juliusmarminge in pingdotgg/t3code#11206
* perf(mobile): reuse completed code lines while streaming by @juliusmarminge in pingdotgg/t3code#11211
* perf(client): reduce remote request and message sync overhead by @Bil0000 in pingdotgg/t3code#11029
* fix(web): refresh usage limit countdowns without switching tabs by @t3-code[bot] in pingdotgg/t3code#11187
* fix(client-runtime): typecheck device hub ticket request on main by @juliusmarminge in pingdotgg/t3code#11304
* feat(settings): add per-project overrides for scopable server settings by @juliusmarminge in pingdotgg/t3code#11176
* feat(web): pick settings environment and project as two selects by @juliusmarminge in pingdotgg/t3code#10636
* feat(settings): edit any scopable setting as a project override by @juliusmarminge in pingdotgg/t3code#10639
* feat(web): float device streams over chat by @juliusmarminge in pingdotgg/t3code#11285
* fix(web): floating preview can use the margins beside the composer by @juliusmarminge in pingdotgg/t3code#11290
* perf(client-runtime): speed up message sync on desktop and mobile by @Bil0000 in pingdotgg/t3code#11302
* fix(web): use the configured panel shortcut on the PR page by @Bil0000 in pingdotgg/t3code#11292
* feat(web): add PR page selections to new draft threads by @Bil0000 in pingdotgg/t3code#11296


**Full Changelog**: pingdotgg/t3code@v0.0.41-nightly.20260911.1551...v0.0.41-nightly.20260911.1564

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.41-nightly.20260911.1564
aorwall added a commit to aorwall/t3code that referenced this pull request Sep 12, 2026
Merges `upstream/main` at `e81606494` into the fork, from merge base
`02297e3db` — 47 upstream commits.

The theme of this range is scopable settings: upstream made every server
setting addressable at a scope (global / environment / project) with
per-project overrides, which is why 11 of the 15 conflicts are settings
files. The rest is conversation rewind, floating device streams, and a
large batch of message-sync and markdown-streaming perf work.

## Merge stats

- Landed (`HEAD^1..HEAD`): 277 files, 17243+/4783−
- Upstream range (base..`HEAD^2`): 275 files, 17011+/4749−
- Fork delta (`HEAD^2..HEAD`): 756 files, 76559+/2096−

The two file lists reconcile: the 3 extra landed files are
`docs/fork/inventory.json`, `docs/fork/upstream-merge-log.md` and
`docs/fork/gaps.md`; the 1 file in the range that did not land is
`apps/web/src/routes/settings.integrations.tsx`, resolved `ours` per the
`moatless-admin-integrations-route` inventory entry (that route is a
Moatless admin page here, and upstream's embedded-surface settings live
at `/settings/browser`).

All 15 conflicts were resolved by the verdict `preflight.mjs` printed.
No `decide` conflict was left unresolved. Details, including the
owned-concern sweep (no keyword hits) and the unsupported-method
reconciliation (0 ADD, 0 DROP, 2 KEEP, 4 known exceptions), are in the
dated entry in `docs/fork/upstream-merge-log.md`.

Two findings worth naming here:

- **A silent auto-merge failure.** pingdotgg#11285 changed the mini-player target
from a tab id to a source union. Git updated upstream's own assertion in
`PreviewView.test.tsx` and left the fork-only "under the frame
capability" case next to it still asserting the old string. No conflict
marker, no `resolution-check.mjs` finding — only the fork's own test
suite caught it.
- **Stale inventory anchors.** Upstream moved the project Actions
section out of `ProjectSettingsPanel.tsx` into a new
`ProjectActionsSettings.tsx`, which is where `scriptsEditable` is now
derived and where upstream's new writing Reset button is gated. Four
inventory entries were re-pointed in this merge rather than silently
dropping their deltas.

## Usable as-is

Client work the fork can expose with no Moatless backend change:

- Scoped settings UI and the two-select scope picker (pingdotgg#10639, pingdotgg#10636) —
`SettingsScopeContext`, `ScopedSwitch`, `settingKeys`, the `mixed`
state. The reading half works against Moatless today.
- Float device streams over chat, as a source union rather than a tab id
(pingdotgg#11285); recording status on floating previews (pingdotgg#11312); floating
preview using composer margins (pingdotgg#11290).
- PR-page selections into new drafts (pingdotgg#11296);
projects-on-another-machine badge (pingdotgg#11323); Usage opening on Limits
(pingdotgg#11261).
- macOS permission onboarding (pingdotgg#11289); hold-to-quit fix (pingdotgg#11016);
preview keystrokes kept out of the composer (pingdotgg#11354).
- Message-sync and markdown-streaming perf: pingdotgg#11302, pingdotgg#11029, pingdotgg#11211,
pingdotgg#11198, pingdotgg#11196, pingdotgg#11193, pingdotgg#11181, pingdotgg#11206.
- Assorted web/mobile fixes: pingdotgg#11361, pingdotgg#10757, pingdotgg#11357, pingdotgg#10571, pingdotgg#11348,
pingdotgg#11349, pingdotgg#11281, pingdotgg#11188, pingdotgg#11283, pingdotgg#11292, pingdotgg#11187, pingdotgg#11228, pingdotgg#11103, pingdotgg#10612,
pingdotgg#11032, pingdotgg#11233, pingdotgg#11234, pingdotgg#11304, pingdotgg#11240.

## Unsupported in Moatless / needs implementation

- **Conversation rewind** — `thread.conversation.revert` (pingdotgg#11358). A new
member of `DispatchableClientOrchestrationCommand` in
`packages/contracts/src/orchestration.ts`, bringing the fork to 30
command types (28 upstream's, 2 fork-only). Moatless does not dispatch
it, and a client command cannot be refused per-type, so "Edit from here"
on `RevertUserMessageButton` is reachable whenever the turn is idle and
does nothing. Needs backend dispatch.
- **Per-project setting overrides** — the `projectSettingsOverrides`
capability and the 17-key `ProjectSettingsOverrides` record (pingdotgg#11176).
Two pieces are needed: the capability reported by
`/.well-known/t3/environment`, and `server.updateSettings` served at
project scope. Until both land, the capability filter in
`scopedSettings.ts:170` and `ProjectActionsSettings.tsx:72` drops the
write on the client — the control renders, the user toggles it, and
**the write never leaves the browser**. A silent no-op is worse than a
hidden control or an honest refusal; recorded in `docs/fork/gaps.md`.
- **Default thread permissions** — `defaultRuntimeMode` (pingdotgg#11346). Reads
fine, cannot be saved. Same `server.updateSettings` write path as above,
one level deeper, not a separate gap.

## Backend behavior to consider reproducing in Moatless

Upstream server-side work the fork cannot use directly, but that
Moatless would benefit from:

- **Queue messages during context compaction** (pingdotgg#11107,
`ProviderCommandReactor.ts`) — a message sent while compaction is in
flight is currently dropped rather than held.
- **Restore provider history and prompts when rewinding** (pingdotgg#11338,
`CheckpointReactor.ts`) — the counterpart to
`thread.conversation.revert` above; rewinding the thread without
rewinding provider state leaves the two out of sync.
- **Detect file renames in review diffs** (pingdotgg#8086,
`apps/server/src/vcs/GitVcsDriverCore.ts`) — a rename currently reads as
a whole-file delete plus a whole-file add.
- **Preserve qualified Codex model ids** (pingdotgg#9921, `ModelManifest.ts` +
`CodexTextGeneration.ts`).
- **Model defaults** astra-medium / fable-5.1-medium (pingdotgg#11347).

All five are recorded under the runtime-fixes entry in
`docs/fork/gaps.md`.

## Verification

`verify.mjs` (full pass): 7 of 8 checks green — `duplicate-adds`,
`tripwires`, `resolution-check`, `unsupported-methods`, `fmt:check`,
`lint`, `typecheck`.

`test` is red on **`@t3tools/desktop` only**, at
`scripts/browser-secret-native.test.mjs > bundled libsecret helper`:
`Command failed: pkg-config --cflags --libs libsecret-1`. This is the
standing sandbox gap, not a merge regression — the test file's last
commit is `498ab9c39` (pingdotgg#7261, before the merge base), `git diff
--name-only` against both merge parents is empty for it, and `pkg-config
--exists libsecret-1` fails in this environment. It is already an entry
in `docs/fork/gaps.md`. Every other package passes, including
`@t3tools/web` (5079 tests) after the `PreviewView.test.tsx` fix above.

Three typecheck failures the merge introduced were fixed in it:
`SETTINGS_CATEGORY_SCOPES` in `settingsSearch.ts` was missing all 9
fork-only settings paths, and two `filterAvailableSettingsSearchItems`
literals in `settingsSearch.test.ts` were missing the fork's
`forgejoEnabled` field.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---
Moatless task:
https://moatless.soaplabstest.com/tasks/e70b41b3-779d-43b8-8f34-7de516548e7c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants