Revert "Stabilize mesh MoA context and tool loops (#820)" - #823
Merged
Conversation
This reverts commit 337f5b8.
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (32)
📝 WalkthroughWalkthroughThis pull request significantly simplifies the mixture-of-agents system by removing the Gemma-4-E4B-it-Q4_K_M model, eliminating parameter-count-based model comparison, removing context-budget reservations, and streamlining tool-call rescue, context packing, answer arbitration, and response handling throughout the codebase. ChangesMoA Simplification and Model Management Cleanup
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reverts PR #820 to restore the previous Mixture-of-Agents (MoA) / virtual mesh behavior after #820 caused model: "mesh" requests on the public console to hang indefinitely.
Changes:
- Reverts MoA context packing, role assignment heuristics, tool-loop handling, and reducer recovery behavior back toward the pre-#820 implementation.
- Simplifies/removes several OpenAI Responses-API normalization and MoA streaming behaviors introduced in #820 (including tool-shape normalization and richer SSE lifecycle events).
- Reverts UI and model-pack/catalog adjustments that shipped alongside #820 (e.g., tooltip touch-disable logic, default 8–24GB pack, catalog entries).
Reviewed changes
Copilot reviewed 32 out of 32 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/openai-frontend/src/responses.rs | Reverts parts of Responses request normalization (content/input handling, tool shape normalization, and related tests). |
| crates/mesh-mixture-of-agents/tests/sim_worker_accounting.rs | Updates test fixtures to match reverted ModelEntry shape. |
| crates/mesh-mixture-of-agents/tests/sim_tool_result_routes_to_reducer.rs | Removes reducer-timeout recovery test and updates model fixtures. |
| crates/mesh-mixture-of-agents/tests/sim_tool_call_text_not_passed_as_content.rs | Updates model fixtures to match reverted ModelEntry shape. |
| crates/mesh-mixture-of-agents/tests/sim_enable_thinking_propagation.rs | Updates model fixtures to match reverted ModelEntry shape. |
| crates/mesh-mixture-of-agents/tests/sim_all_workers_fail.rs | Updates model fixtures to match reverted ModelEntry shape. |
| crates/mesh-mixture-of-agents/src/worker.rs | Reverts role assignment ordering (removes parameter-count-based strength ordering). |
| crates/mesh-mixture-of-agents/src/session.rs | Reverts session behaviors; tool-result pairing logic changed. |
| crates/mesh-mixture-of-agents/src/reducer.rs | Reverts reducer-candidate ordering logic and removes parameter-based test coverage. |
| crates/mesh-mixture-of-agents/src/normalize.rs | Removes several guardrail-rescue normalization tests added in #820. |
| crates/mesh-mixture-of-agents/src/lib.rs | Reverts tool intent/grace logic and reducer failure handling behavior. |
| crates/mesh-mixture-of-agents/src/fanout.rs | Reverts grace/early-exit logic (removes “wait for pending strong worker” behavior). |
| crates/mesh-mixture-of-agents/src/context.rs | Reverts MoA context windowing/anchoring and simplifies reducer/worker packing. |
| crates/mesh-mixture-of-agents/src/backend.rs | Reverts backend request shaping and tool-call extraction behavior. |
| crates/mesh-mixture-of-agents/src/arbiter.rs | Reverts arbitration behavior (removes role-bias tie-breaking + strong-worker pending gate). |
| crates/mesh-llm-ui/src/features/chat/pages/ChatPage.test.tsx | Removes follow-up “same-conversation history” assertion from UI tests. |
| crates/mesh-llm-ui/src/features/chat/components/composer/ChatComposer.tsx | Reverts pointer/touch handling for the send button (mouse handler only). |
| crates/mesh-llm-ui/src/features/chat/api/use-chat.ts | Reverts conversation-reset timing (useEffect instead of useLayoutEffect). |
| crates/mesh-llm-ui/src/features/chat/api/use-chat.test.tsx | Removes message-reset coverage and simplifies stream-draining helper. |
| crates/mesh-llm-ui/src/components/ui/tooltip.tsx | Reverts touch-tooltip disabling logic (always renders tooltip). |
| crates/mesh-llm-node/src/catalog.json | Removes the Gemma-4-E4B entry added in #820. |
| crates/mesh-llm-host-runtime/src/network/openai/transport.rs | Reverts model metadata/context selection behavior for public aliases + virtual mesh context derivation. |
| crates/mesh-llm-host-runtime/src/network/openai/moa_gateway/progress.rs | Reverts progress streaming behavior and failure-tail handling. |
| crates/mesh-llm-host-runtime/src/network/openai/moa_gateway/mod.rs | Reverts MoA config/context budgeting and simplifies Responses SSE emission. |
| crates/mesh-llm-host-runtime/src/network/openai/moa_gateway/context_selection.rs | Reverts MoA context selection + virtual mesh context calculation logic. |
| crates/mesh-llm-host-runtime/src/network/openai/moa_gateway/context_budget.rs | Deletes MoA context reserve logic introduced in #820. |
| crates/mesh-llm-host-runtime/src/network/nostr.rs | Reverts default auto-pack model for 8–24GB tier back to Qwen3-8B. |
| crates/mesh-llm-host-runtime/src/mesh/tests.rs | Removes public-alias context lookup test added in #820. |
| crates/mesh-llm-host-runtime/src/mesh/mod.rs | Reverts peer context-length lookup for public model aliases. |
| crates/mesh-llm-guardrails/src/rescue.rs | Reverts several tool-call rescue parsers and changes candidate scanning behavior. |
| crates/mesh-client/src/network/nostr.rs | Reverts default auto-pack model for 8–24GB tier back to Qwen3-8B. |
| crates/mesh-client/src/models/catalog.json | Removes the Gemma-4-E4B entry added in #820. |
Comments suppressed due to low confidence (1)
crates/mesh-llm-guardrails/src/rescue.rs:99
tool_call_candidatesno longer bounds the input before running the non-JSON rescue parsers.json_candidates()is bounded, but the other parsers will scan the entire model output, which is untrusted and can be very large. Re-applyingbounded_prefixhere restores the intended O(max) behavior and reduces worst-case CPU/memory use.
fn tool_call_candidates(content: &str) -> Vec<Value> {
let mut candidates = Vec::new();
for json_candidate in json_candidates(content) {
if let Ok(value) = serde_json::from_str::<Value>(&json_candidate) {
candidates.push(value);
Comment on lines
+154
to
158
| if let Some(pending) = | ||
| self.pending_tools.iter_mut().find(|p| p.call_id == call_id) | ||
| { | ||
| let pending = &mut self.pending_tools[idx]; | ||
| pending.result = Some(content); | ||
| tracing::info!( |
Comment on lines
+1194
to
+1197
| let data = format!("data: {created}\n\n"); | ||
| let framed = format!("{:x}\r\n{}\r\n", data.len(), data); | ||
| stream.write_all(framed.as_bytes()).await?; | ||
| stream.flush().await?; |
Comment on lines
+1215
to
+1218
| let data = format!("data: {}\n\n", delta_event); | ||
| let framed = format!("{:x}\r\n{}\r\n", data.len(), data); | ||
| stream.write_all(framed.as_bytes()).await?; | ||
| stream.flush().await?; |
Comment on lines
368
to
371
| let body = &moa_result.response_body; | ||
| if is_moa_failure_body(body) { | ||
| return write_failure_as_sse_tail( | ||
| &mut tcp_stream, | ||
| body, | ||
| adapter, | ||
| completion_id, | ||
| continuation, | ||
| ) | ||
| .await; | ||
| return write_failure_as_sse_tail(&mut tcp_stream, body, adapter, completion_id).await; | ||
| } |
Comment on lines
746
to
750
| async fn capture_failure_tail( | ||
| adapter: proxy::ResponseAdapter, | ||
| body: serde_json::Value, | ||
| continuation: Option<ProgressContinuation>, | ||
| ) -> String { | ||
| let listener = tokio::net::TcpListener::bind("127.0.0.1:0") |
Comment on lines
276
to
280
| let name = tc | ||
| .pointer("/function/name") | ||
| .and_then(|n| n.as_str()) | ||
| .map(str::trim) | ||
| .filter(|name| !name.is_empty()) | ||
| .ok_or_else(|| "malformed tool call: missing function.name".to_string())?; | ||
| .unwrap_or("unknown"); | ||
| let args = tc |
Comment on lines
+1238
to
+1242
| let tail = [text_done, completed]; | ||
| for event in &tail { | ||
| let data = format!("data: {}\n\n", event); | ||
| let framed = format!("{:x}\r\n{}\r\n", data.len(), data); | ||
| stream.write_all(framed.as_bytes()).await?; |
Comment on lines
490
to
494
| async fn write_failure_as_sse_tail( | ||
| stream: &mut TcpStream, | ||
| body: &serde_json::Value, | ||
| adapter: proxy::ResponseAdapter, | ||
| completion_id: &str, |
Comment on lines
775
to
779
| let body = serde_json::json!({ | ||
| "error": { "message": "All workers failed", "code": "all_workers_failed" } | ||
| }); | ||
| let raw = capture_failure_tail(proxy::ResponseAdapter::None, body, None).await; | ||
| let raw = capture_failure_tail(proxy::ResponseAdapter::None, body).await; | ||
| assert!( |
michaelneale
added a commit
that referenced
this pull request
Jun 11, 2026
* origin/main: Salvage safe bits from reverted MoA PR (#820) (#824) Revert "Stabilize mesh MoA context and tool loops (#820)" (#823) Stabilize mesh MoA context and tool loops (#820) chore: flip docs pages to canonical URLs (#822) chore: Console public domain (#821) Add meshllm.cloud website, catalog viewer, and onboarding docs (#806) # Conflicts: # docs/index.html
This was referenced Jul 15, 2026
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts #820.
Why
#820 regressed the
mesh/ MoA route on the public console: requests withmodel: "mesh"hang and never return.Verified by rolling back the Fly console image:
model: "mesh"returns 200 in ~1.5s.model: "mesh"request hangs and times out with no response and no routing log activity.A concrete single-model request returns fast either way, so routing works — it is specifically the MoA/virtual-LLM fan-out path introduced/changed by #820 that stalls.
The regression only surfaced against the live relay-heavy public mesh, not in the CI sim tests, which is why it passed checks.
Impact
mainback to a deployable state so the next console deploy does not re-break MoA.Follow-up
#820's MoA context/budget work should be reworked and re-merged with a multi-node confidence test (mesh route against a real mesh) before redeploying.
Summary by CodeRabbit
Release Notes
Model Updates
Bug Fixes
Improvements