[Rust TreeCore] Harden runtime and CI parity

[Jialin]

Motivation

The Rust TreeCore still has a few correctness and build-integration gaps after its initial merge:

• stale node handles can panic while the core mutex is held, poisoning the backend for all subsequent requests;
• Rust accepts shortened SWA prefetch windows below non-root anchors, while the Python TreeCore rejects and releases them;
• Rust extension caches can reuse artifacts built against a different PyTorch installation or stale protobuf sources;
• the Rust TreeCore E2E is registered on AMD even though that platform does not package this extension.

This PR is rebased onto the merged #37290 crate rename and current main.

Modifications

Rust TreeCore boundary handling

• Make NodeArena::resolve the single node-handle lookup API and return a dedicated NodeAccessError for stale handles.
• Remove the parallel optional resolver, TreeCore-specific resolver wrapper, binding-side prevalidation wrappers, and panicking host-insert wrapper.
• Make production and inspection operations propagate node-access failures from the core; PyO3 maps them to KeyError without poisoning the mutex.
• Validate all handles for multi-node mutations before changing tree state, including handles nested in transfer and insert results.
• Preserve explicit absence semantics only where stale work is intentionally dropped (buffer_only snapshots and the three inspector membership predicates).
• Pass already-resolved internal node indices through match finalization instead of resolving public handles again.

Rust SWA parity

• Reject and release shortened SWA prefetch windows below non-root anchors.
• Keep the existing root-anchor behavior unchanged.
• Add the same backend-routed regression for the Python and Rust TreeCore implementations.

Rust extension build and CI

• Include the resolved PyTorch installation in torch-sys rebuild inputs.
• Include proto/** in Rust extension cache keys, sparse checkout inputs, and cache seeding triggers.
• Add metadata-declared external source inputs to the local Rust extension fingerprint.
• Keep the Python and Rust Full E2E subclasses together; the shared file remains registered on CUDA and AMD, while the Rust subclass is skipped only in AMD CI because that platform does not package the extension.

Accuracy Tests

No model kernels or numerical computation are changed.

Validation performed on the consolidated branch:

• Rust native suite: 831 passed, 1 ignored.
• Rust TreeCore adapter/inspection integration: 104 passed.
• Shared shortened-SWA-prefetch regression:
  • Python TreeCore: 1 passed.
  • Rust TreeCore: 1 passed.
• Rust extension discovery and cache tests: 15 passed.
• Clippy with Python and inspection features passed with warnings denied.
• Changed-file formatting, lint, syntax, CI-registration, and cache-key checks passed.

Speed Tests and Profiling

Not applicable. These changes affect error handling, SWA transfer validation, extension cache invalidation, and CI registration; they do not modify model execution or kernel hot paths.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations. No user-facing documentation change is required.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed. Not applicable as described above.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci.
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #34097120801
Latest PR Test (Extra): ✅ Run #34097120429
Latest PR Test (AMD ROCm 7.2): ❌ Run #34097120691

[Jialin]

/rerun-failed-ci

09/01 2:51PM PT: rerun due to tests got cancelled
09/01 4:15PM PT: rerun due to tests got cancelled
09/01 9:47PM PT: rerun due to tests got cancelled
09/02 9:59AM PT: rerun due to tests got cancelled

[bingxche]

Thanks for the fix for AMD CI. May I know the plan of supporting Rust TreeCore on AMD? This PR has conflicts now, can I cherry pick first to skip the failed test TestRustUnifiedFullRadixCache?

[bingxche]

Extracted the AMD-only CI skip from commit 5c9bf17 into focused draft #38287 on current main, since this branch is currently conflicting. If that split is preferred, this commit can be dropped here during rebase; otherwise I can close the draft.

[Jialin]

Thanks for the fix for AMD CI. May I know the plan of supporting Rust TreeCore on AMD? This PR has conflicts now, can I cherry pick first to skip the failed test TestRustUnifiedFullRadixCache?

Let me do a quick rebase now. And I could followup to do the supporting in AMD, ideally, it should be pretty lightweight.

[Jialin]

Extracted the AMD-only CI skip from commit 5c9bf17 into focused draft #38287 on current main, since this branch is currently conflicting. If that split is preferred, this commit can be dropped here during rebase; otherwise I can close the draft.

Please feel free to go ahead landing #38287 for unblocking AMD first. I could do the rebase correspondingly later on.

[Jialin]

/rerun-failed-ci

• 09/08 10:10AM: Retry base-c tests which seems stuck in the queue

[hzh0425]

LGTM, sry for the late

[Jialin]

LGTM, sry for the late

Thanks! No worry :)