Skip to content

Restore llama.cpp upstream canary patch replay - #609

Merged
i386 merged 1 commit into
mainfrom
codex/rebase-qwen35moe-llama-patch
May 21, 2026
Merged

Restore llama.cpp upstream canary patch replay#609
i386 merged 1 commit into
mainfrom
codex/rebase-qwen35moe-llama-patch

Conversation

@IvGolovach

@IvGolovach IvGolovach commented May 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Restores the llama.cpp upstream canary by rebasing the Mesh-LLM patch queue onto current upstream llama.cpp, now replayed on fresh main.

  • Updates the pinned llama.cpp upstream SHA to 6a257d44633d4a752183ed778b88d2924d0a6b9d.
  • Reworks the Qwen35MoE runtime-slice patch for upstream's current NextN/MTP graph shape.
  • Refreshes the dependent Qwen35MoE recurrent-only slice patch.
  • Refreshes later patch contexts that now depend on upstream's added h_pre_norm graph result path.
  • Keeps the PR focused on the llama.cpp patch queue and upstream pin after rebasing onto latest main.
  • Updates the parity helper's inherited-graph allowlist so Mistral4's DeepSeek2-backed staged graph is recognized during validation.

Why

The scheduled llama.cpp upstream canary was failing before it could build or smoke-test Skippy because patch 0009-Support-Qwen35MoE-runtime-slice-packages.patch no longer applied to upstream master.

The root cause is upstream graph churn around Qwen35/Qwen35MoE: the main transformer stack is now separated from NextN/MTP blocks, and the graph also carries a pre-norm output path. Skippy runtime slices need to execute only the main transformer layer range while preserving the upstream MTP/pre-norm behavior for full/output stages.

After review, this branch was rebased onto current main so the patch replay is validated with the latest MoA/runtime and Swift smoke fixes already landed upstream.

Implementation

  • Rebased 0009 so Qwen35MoE slices clamp staged execution to n_layer - nextn_predict_layers, keep token embeddings only on the first slice, and build output row ids only for full/output slices.
  • Rebased 0010 so recurrent-only Qwen35MoE slices still avoid attention-position inputs when the selected slice contains no attention layer.
  • Refreshed 0057 and 0076 around upstream's h_pre_norm graph result addition while preserving RWKV7 and Gemma3n activation sidebands.
  • Rebased 0064 with the same Qwen35 NextN/MTP treatment used for Qwen35MoE.
  • Advanced third_party/llama.cpp/upstream.txt to the validated upstream SHA.
  • Added mistral4 to the parity helper's inherited staged-graph mapping because upstream dispatches Mistral4 through the DeepSeek2-backed graph implementation.

Compatibility

  • No mesh gossip, protobuf, plugin protocol, or client protocol compatibility change.
  • No new Skippy ABI surface is introduced by this PR; this is a patch-queue rebase against a newer llama.cpp upstream.
  • No LLAMA_CPP_SHA file is added; the current repo pin mechanism is third_party/llama.cpp/upstream.txt.

Validation

  • git fetch --no-tags origin main:refs/remotes/origin/main: PASS, validated base 559dc7bc
  • git diff --check origin/main...HEAD: PASS, no output
  • git diff --check: PASS, no output
  • git diff --cached --check: PASS, no output
  • LLAMA_WORKDIR=/tmp/mesh-llm-llama-rebase.DnDzcs scripts/prepare-llama.sh pinned: PASS, upstream 6a257d44633d4a752183ed778b88d2924d0a6b9d, patched e43ab36164c9b94ba0c36a5599ba7262cb0613e5
  • cargo fmt --all -- --check: PASS
  • python3 -m py_compile scripts/skippy-llama-parity.py: PASS
  • scripts/skippy-llama-parity.py --llama-src /tmp/mesh-llm-llama-rebase.DnDzcs validate: PASS

Remote gates:

  • PR Builds: pending rerun on the current rebased SHA.
  • PR Quality Checks: pending rerun on the current rebased SHA.

Not run locally:

  • just build: not required for this fresh-main rebase/update; mandatory PR Builds validates the final pushed SHA.
  • scripts/skippy-ci-smoke.sh: local hf CLI is unavailable here; the upstream canary installs hf and remains the remote smoke gate for this path.

Rollback

Revert this PR.

Related

Fixes #603

michaelneale added a commit that referenced this pull request May 21, 2026
@michaelneale
ci: disable swift_sdk_smoke on this branch (infra flake)
3a3ce00 
macOS runners are rejecting `-fuse-ld=/opt/homebrew/bin/ld64.lld` with
`clang: error: invalid linker name in argument`. Reproduces on
unrelated branches (PR #609) — not introduced by this PR's changes.

Gating with `false &&` so the job stays defined but skips. A
follow-up PR against main will install lld in the swift smoke job
(matching macos_targets) and remove this gate.
@michaelneale

michaelneale commented May 21, 2026

Copy link
Copy Markdown
Collaborator

will jhave to fix swift thing separately.

@michaelneale michaelneale mentioned this pull request May 21, 2026
Merged
michaelneale added a commit that referenced this pull request May 21, 2026
@michaelneale
ci(sdk-smoke): install lld in macOS swift smoke job (#610)
cecc7eb 
* ci(sdk-smoke): install lld in macOS swift smoke job

The Swift SDK smoke job has been failing on macOS runners with:

  clang: error: invalid linker name in argument
  '-fuse-ld=/opt/homebrew/bin/ld64.lld'

The transitive cargo build inside
`sdk/swift/scripts/generate-swift-bindings.sh` runs in a temp dir and
ends up invoking `cc` with `-fuse-ld=/opt/homebrew/bin/ld64.lld`.
Apple clang accepts that flag only when the linker binary actually
exists on disk; the job only installed `jq` so the link step failed.

The `macos_targets` job already does `brew install ... lld` for the
same reason. Install it in the swift smoke lane too.

Reproduced on multiple branches (PRs #566, #609); not specific to any
one change.

* ci(sdk-smoke): re-enable swift smoke gate now that lld install is fixed

This PR installs lld in the macOS swift smoke job, which was the root cause
of the linker failure that prompted the temporary 'if: false &&' bypass in
1b1aaf4. Re-enable the normal gate.

* ci(compute-changes): route sdk-smoke.yml edits into sdk_smoke_required

A PR that only edits .github/workflows/sdk-smoke.yml (e.g. this one)
otherwise can't trigger the swift/linux/kotlin SDK smokes it's trying
to fix — classic catch-22. Add the reusable workflow file itself to
DIRECT_SDK_INPUTS.
@michaelneale

michaelneale commented May 21, 2026

Copy link
Copy Markdown
Collaborator

oh thanks @IvGolovach - worthwhile updating this to main to be sure?

@IvGolovach
Rebase llama.cpp patch queue for upstream canary
3173406 
Validation

* Validation tier: Tier 4 — llama.cpp patch queue/upstream pin maintenance plus narrow parity-helper correction surfaced by the fresh-main rebase.

* git fetch --no-tags origin main:refs/remotes/origin/main: PASS, validated base 559dc7b.

* git diff --check origin/main...HEAD: PASS, no output.

* git diff --check: PASS, no output.

* git diff --cached --check: PASS, no output.

* LLAMA_WORKDIR=/tmp/mesh-llm-llama-rebase.DnDzcs scripts/prepare-llama.sh pinned: PASS, upstream 6a257d44633d4a752183ed778b88d2924d0a6b9d, patched e43ab36164c9b94ba0c36a5599ba7262cb0613e5.

* cargo fmt --all -- --check: PASS.

* python3 -m py_compile scripts/skippy-llama-parity.py: PASS.

* scripts/skippy-llama-parity.py --llama-src /tmp/mesh-llm-llama-rebase.DnDzcs validate: PASS.

* Ledger: not applicable — not required for selected validation tier/change family.

* Version: not applicable — this updates the llama.cpp upstream pin/patch queue, not a mesh-llm release version.

* Not run: just build — not required for this fresh-main rebase/update; mandatory PR Builds validates the final pushed SHA.

* Not run: scripts/skippy-ci-smoke.sh — not required for local PR-ready proof; local hf CLI is unavailable here, and the upstream canary installs hf and remains the remote smoke gate.

Rollback

* git revert HEAD
@IvGolovach
IvGolovach force-pushed the codex/rebase-qwen35moe-llama-patch branch from 46a9efb to 3173406 Compare May 21, 2026 04:23
@IvGolovach

IvGolovach commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the nudge — I rebased #609 onto current main and kept it focused on the llama.cpp patch replay now that the Swift smoke fix has landed separately.

Fresh CI is green now. I also tightened one small parity-helper false positive around Mistral4’s inherited staged graph that showed up during the fresh-main validation. Appreciate the catch 🙂

@i386
i386 merged commit 790b8bc into main May 21, 2026
20 checks passed
@i386
i386 deleted the codex/rebase-qwen35moe-llama-patch branch May 21, 2026 23:21
michaelneale added a commit that referenced this pull request May 21, 2026
@michaelneale
Merge remote-tracking branch 'origin/main' into micn/moa-propagate-en…
8262642 
…able-thinking

* origin/main:
  Rebase llama.cpp patch queue for upstream canary (#609)
  doc(chat): flag AUTO_BACKEND_MODEL as the one-line flip point (#619)
michaelneale added a commit that referenced this pull request May 21, 2026
@michaelneale
Merge remote-tracking branch 'origin/main' into micn/moa-grace-on-div…
3ba818b 
…erse-answers

* origin/main:
  Rebase llama.cpp patch queue for upstream canary (#609)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

llama.cpp upstream canary failing nightly: patch 0009 (Qwen35MoE) needs rebase against current upstream

3 participants

@IvGolovach @michaelneale @i386