Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
94e9d2e
pr1: tinker data format layer — per-token client channels and slot-st…
yushengsu-thu Aug 7, 2026
6bcd8af
pr2: tinker slot pool, run registry, and registration config
yushengsu-thu Aug 7, 2026
d7b8695
pr3: tinker operation ledger — gap-buffered arrival, fingerprinted re…
yushengsu-thu Aug 7, 2026
2efbc30
pr4: tinker control plane — serving identity, backend preflight, cont…
yushengsu-thu Aug 7, 2026
6276d0b
pr5: tinker per-slot Adam — per-call AdamParams, unnormalized gradien…
yushengsu-thu Aug 7, 2026
f45c086
pr6: tinker loss layer — per-slot dispatch, sum reduction, logprob co…
yushengsu-thu Aug 7, 2026
ca91aa5
pr7: tinker trainer verbs — control-op execution, fixed-residency rec…
yushengsu-thu Aug 7, 2026
709dac0
pr8: tinker rollout layer — queue children, homogeneous kind-locked s…
yushengsu-thu Aug 8, 2026
abba6c8
pr9: tinker wiring — driver loop with publish barrier, actor-group ve…
yushengsu-thu Aug 8, 2026
d4c1c46
pr10: delete the adapter-sample-level multi-LoRA path
yushengsu-thu Aug 8, 2026
638abc0
tinker backend: H200 E2E client — lifecycle, forward, ownership-fence…
yushengsu-thu Aug 8, 2026
0486b5a
tinker backend: 4-adapter RL quality client — concurrent GRPO loops o…
yushengsu-thu Aug 8, 2026
c13156d
tinker backend: split protocol-mode predicate from the multi-LoRA exe…
yushengsu-thu Aug 11, 2026
52bd388
rollout: typed postprocess contract — the manager stops sniffing the …
yushengsu-thu Aug 11, 2026
5391f3f
tinker tests: refactor-equivalence capture of the gradient window and…
yushengsu-thu Aug 12, 2026
8735779
tinker backend: registration-keyed GradientWindowTracker — step/dirty…
yushengsu-thu Aug 12, 2026
e03c86f
tinker: batch-local operation lanes carry loss/result correlation — t…
yushengsu-thu Aug 12, 2026
fe20560
tinker: TrainerResidencyPort, ResidentBinding, and the batch executio…
yushengsu-thu Aug 12, 2026
e102a54
tinker: generic control coordinator + MultiLoraParameterExecutor — co…
yushengsu-thu Aug 12, 2026
fba7e63
tinker rollout: OperationQueuePort — the operation-to-batch adapter s…
yushengsu-thu Aug 12, 2026
c5010ee
tinker driver: RolloutComponents role factory + WeightPublisher seam …
yushengsu-thu Aug 12, 2026
67e771c
tinker backend: InferenceAdminPort — engine aborts leave the backend'…
yushengsu-thu Aug 12, 2026
3ee8ed2
tinker tests: align the multi-LoRA CP fake iterator with the real Dat…
yushengsu-thu Aug 12, 2026
0dc5970
tinker: the batch execution lease is metric-inert — log_rollout_data …
yushengsu-thu Aug 12, 2026
f03be50
tinker: failure-path fixes from the external adversarial review — win…
yushengsu-thu Aug 12, 2026
5245468
tinker: delete the dead/duplicate surfaces flagged by the external re…
yushengsu-thu Aug 12, 2026
f830a45
tinker: norm-blind veto — refuse a step whose norm collection is stru…
yushengsu-thu Aug 12, 2026
846798d
tinker tests: black formatting on the new executor regression test
yushengsu-thu Aug 12, 2026
c0def78
tinker: refuse recompute configs that silently zero every adapter gra…
yushengsu-thu Aug 12, 2026
b854773
tinker: relax the multi-LoRA recompute guard to a bridge feature probe
yushengsu-thu Aug 13, 2026
08c26e4
tinker: finalize uncommitted data batches — no operation stays CLAIME…
yushengsu-thu Aug 13, 2026
92e7209
tinker docs: narrow the full-parameter reuse claims to what the code …
yushengsu-thu Aug 13, 2026
a52726b
Merge main @ 456340f86 into tinker-compatible-backend
yushengsu-thu Aug 13, 2026
259eebd
docs: regenerate the examples mirror for the tinker recipe set
yushengsu-thu Aug 13, 2026
66f37a0
tinker: document LayerWise expert ownership
yushengsu-thu Aug 13, 2026
005135d
[multi-lora] integrate LayerWise expert ownership fixes
yushengsu-thu Aug 13, 2026
76ac997
[multi-lora] narrow LayerWise integration scope
yushengsu-thu Aug 13, 2026
c04e709
docker: track Megatron-Bridge @bridge instead of pinning a SHA
yushengsu-thu Aug 13, 2026
8a70cbf
Merge main @ 85e7655f1: docs sidebar restructure, training log-prob r…
yushengsu-thu Aug 13, 2026
699bc97
tests: expect the tinker forward_only kwarg in the exact train-call pin
yushengsu-thu Aug 13, 2026
6347bcf
rollout: cleanup-safe downstream handoff — opaque RolloutFnHandoff re…
yushengsu-thu Aug 14, 2026
62aee3a
tinker rollout: direct-await invocation, typed claim/lease recovery, …
yushengsu-thu Aug 14, 2026
36be7ce
Merge main @ f38655d88: class-based rollout is now the default (#2522…
yushengsu-thu Aug 14, 2026
8deb2f3
Merge branch 'main' into tinker-compatible-backend
yushengsu-thu Aug 15, 2026
7b8c2f9
tinker: delete the batch_id dead field, the test-only tenant property…
yushengsu-thu Aug 17, 2026
20217d7
tinker: type the claim result as ClaimedOperationBatch (0813 review §…
yushengsu-thu Aug 17, 2026
a3fcb62
tinker: collapse TinkerOperationSource + QueueChildRolloutFn into the…
yushengsu-thu Aug 17, 2026
08872dc
tinker: rename TinkerOperationBatchAdapter to TinkerRolloutFn, droppi…
yushengsu-thu Aug 17, 2026
999f499
tinker: gap-timeout the operation stream typed instead of stalling fo…
yushengsu-thu Aug 18, 2026
6444c58
tinker: report loss_weight:sum so SFT per-token loss has a correct de…
yushengsu-thu Aug 18, 2026
3034bc4
tinker docs: gap-timeout contract, SFT loss denominator, and SDK 0.24…
yushengsu-thu Aug 18, 2026
93e0d2e
Merge main @ 62a66297b into tinker-compatible-backend
yushengsu-thu Aug 18, 2026
b95bc46
Revert "tinker rollout: direct-await invocation, typed claim/lease re…
yushengsu-thu Aug 18, 2026
51c9e14
Revert "rollout: cleanup-safe downstream handoff — opaque RolloutFnHa…
yushengsu-thu Aug 18, 2026
326f1c4
Merge branch 'main' into tinker-compatible-backend
yushengsu-thu Aug 18, 2026
1ce8021
[multi-lora] fix mixed Tinker loss channels
yushengsu-thu Aug 18, 2026
70b11d0
tests: cover mixed loss channel boundaries
yushengsu-thu Aug 18, 2026
b82a9fc
multi-lora: separate operation backend names from Tinker
yushengsu-thu Aug 18, 2026
9eb370b
test: cover operation rollout path compatibility
yushengsu-thu Aug 18, 2026
053764c
docs: clarify Tinker protocol flag boundary
yushengsu-thu Aug 18, 2026
1718636
refactor: separate Tinker protocol from operation executors
yushengsu-thu Aug 18, 2026
0b1e3d8
test: align operation E2E paths with multi-LoRA
yushengsu-thu Aug 18, 2026
ec676d8
test: keep allocator fixture local to its test module
yushengsu-thu Aug 18, 2026
515021c
test: isolate rollout object-store fixture
yushengsu-thu Aug 18, 2026
306f447
docs: clarify multi-lora launcher args scope
yushengsu-thu Aug 21, 2026
9f71fa6
refactor: group Megatron API backends
yushengsu-thu Aug 21, 2026
b2af112
refactor: align weight updater naming
yushengsu-thu Aug 21, 2026
013788d
refactor: clarify runtime controller naming
yushengsu-thu Aug 21, 2026
f4a5f1f
refactor: trim redundant multi-lora commentary and tests
yushengsu-thu Aug 21, 2026
e48cd1d
test: trim redundant multi-lora coverage
yushengsu-thu Aug 21, 2026
7a0719e
refactor: make multi-lora operations service-only
yushengsu-thu Aug 21, 2026
72ed124
refactor: make rollout dispatch return explicit
yushengsu-thu Aug 21, 2026
84f7383
refactor: clarify multi-lora launcher limits
yushengsu-thu Aug 21, 2026
7189b1e
test: trim redundant operation-backend coverage
yushengsu-thu Aug 21, 2026
5fa5861
docs: compress the driver comments per review
yushengsu-thu Aug 21, 2026
d9553d4
driver: tolerate consecutive generate failures instead of dying with …
yushengsu-thu Aug 22, 2026
2611c1f
operations: terminal-fail CLAIMED operations that outlive a TTL
yushengsu-thu Aug 22, 2026
7b70a7b
rollout: return a FAILED child runtime to IDLE after a cooldown
yushengsu-thu Aug 22, 2026
8ad4f41
test: give the fake RayTaskError the traceback surface its str() formats
yushengsu-thu Aug 22, 2026
2e4a6c2
fix: point the publish-path local import at api_backends after the re…
yushengsu-thu Aug 22, 2026
a52c730
test: statically pin every miles-internal import to an existing module
yushengsu-thu Aug 22, 2026
dd2fb33
test: tolerate absent optional deps when resolving publish-path imports
yushengsu-thu Aug 22, 2026
83dfc4a
multi-lora: delete the unused parse_adapter helper and its orphan test
yushengsu-thu Aug 22, 2026
bf891bb
ci: pin tinker==0.24.1 into the CPU requirements so the SDK metrics-c…
yushengsu-thu Aug 22, 2026
957c253
multi-lora: remove the stamped-slot fallback and its launch-unreachab…
yushengsu-thu Aug 22, 2026
c7f8593
test: drop the duplicate non-tinker trim test
yushengsu-thu Aug 22, 2026
2af2de4
test: move the update_weight_version abort regression out of this PR
yushengsu-thu Aug 22, 2026
54fe8f2
e2e: RL-quality harness exits non-zero when an adapter loop aborts
yushengsu-thu Aug 22, 2026
7563b7e
style: wrap the lease slot-routing call to black's line length
yushengsu-thu Aug 22, 2026
92ac88a
docs: compress multi-line comments to one line per review
yushengsu-thu Aug 23, 2026
7755776
tests: strip comments from stack-added CI tests
yushengsu-thu Aug 23, 2026
f01778a
multi-lora: bound the operation ledger by the registry's completed ring
yushengsu-thu Aug 24, 2026
0cca03a
multi-lora: skip the base-weight update session for LoRA-only distrib…
yushengsu-thu Aug 24, 2026
93bf84d
ci: verify import targets in the source-resolution harness step
yushengsu-thu Aug 24, 2026
5a7134b
remove the bridge recompute-guard: deployment tracks Megatron-Bridge …
yushengsu-thu Aug 24, 2026
e7e8abf
Merge origin/main into tinker-compatible-backend
yushengsu-thu Aug 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,7 @@ RUN pip install "git+https://github.com/NVIDIA-NeMo/Emerging-Optimizers.git@v0.3
RUN TMS_CUDA_MAJOR=$(python3 -c "import torch; print(torch.version.cuda.split('.')[0])") \
pip install git+https://github.com/fzyzcjy/torch_memory_saver.git@f05a8754daf68238d54e4cf31cb3ba866684bbaf --no-cache-dir --force-reinstall
RUN pip install "nvidia-modelopt[torch]>=0.37.0" --no-build-isolation
# radixark/Megatron-Bridge#24 (TE 2.17 grouped-linear contract), merged to @bridge.
# Pinned by SHA, not branch: buildkit caches this layer on the instruction text, so a
# branch that moves leaves the old revision baked into the image with nothing to show it.
RUN pip install git+https://github.com/radixark/Megatron-Bridge.git@7f0fb3456f8ffe47599b5fd167b454605d85f932 --no-deps --no-build-isolation
RUN pip install git+https://github.com/radixark/Megatron-Bridge.git@bridge --no-deps --no-build-isolation
Comment thread
yushengsu-thu marked this conversation as resolved.
RUN pip install megatron-energon --no-deps
RUN pip install multi-storage-client --no-deps

Expand Down
35 changes: 21 additions & 14 deletions docs/advanced/lora.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,9 +300,9 @@ its own Adam state and independently clocked scheduler. The trainer coalesces
ready prompt-group slices or partial adapter batches and selectively upserts
only changed adapters into SGLang.

Set the slot capacity with `--multi-lora-n-adapters N`. A bounded run registers
repeatable `--multi-lora-adapter NAME PATH` entries at startup; service mode can
start with empty slots and register adapters through the controller HTTP API.
Set the slot capacity with `--multi-lora-n-adapters N`. The operation backend
starts as a long-running service with empty slots and registers adapters at
runtime through the controller HTTP API.
This path currently forces Megatron-Bridge LoRA and requires disaggregated NCCL
broadcast, PP1, THD, Adam, and no train offload. Shared-outer expert adapters are
unsupported, and MoE expert adapters cannot use FP8/FP4 experts.
Expand All @@ -312,20 +312,20 @@ Native multi-LoRA is not implied by the native single-adapter work: both current
Megatron-Bridge. Native multi-LoRA is tracked separately in
[issue #2141](https://github.com/radixark/miles/issues/2141).

### Future Tinker-compatible operation backend
### Multi-LoRA operation backend and Tinker compatibility

[PR #2273](https://github.com/radixark/miles/pull/2273) is the active
Tinker-oriented backend proposal. It changes ownership of the training loop:
Multi-LoRA operation-backend proposal. It changes ownership of the training loop:
instead of the server owning a dataset, reward function, and one-step schedule,
clients submit explicit operations against a registered adapter. Its primary
intended consumer is a Tinker-compatible training service rather than a generic
server-owned dataset scheduler.
consumer today is a Tinker-compatible protocol adapter, but the trainer-side
operation contract is named independently from that client protocol.

```text
Tinker-style client
| register + ordered operations
Tinker client -> Tinker protocol/frontend adapter
| normalized register + ordered operations
v
controller / operation ledger
MultiLoraOperationBackend / operation ledger
| bind one fixed LoRA slot
v
Megatron-Bridge multi-LoRA trainer
Expand All @@ -335,6 +335,12 @@ Megatron-Bridge multi-LoRA trainer
SGLang router + registration-scoped adapter identity
```

The current concrete is `MultiLoraOperationBackend`, with
`MultiLoraOperationBatchFn` and `MultiLoraParameterExecutor` handling adapter
batching and slot execution. `Tinker` remains the wire/SDK compatibility name.
A future full-parameter executor may reuse the normalized operation contract,
but this PR does not implement or claim full-parameter training.

The operation surface separates compute, optimization, and publication:

| Operation | Contract |
Expand Down Expand Up @@ -362,9 +368,10 @@ This backend is implemented in an open PR, not released on `main`; the PR
reports H200 validation. PR #2273 provides the operation backend, but its v1
training operations are still exposed through the controller's Ray API. The
stacked [PR #2346](https://github.com/radixark/miles/pull/2346) adds a REST
frontend compatible with the official `tinker==0.24.1` client; its GPU frontend
E2E is still pending. If #2273 lands as proposed, it replaces the current
dataset-driven driver.
frontend compatible with the official `tinker==0.24.1` client. The stacked
full system has passed both RL and pure-SFT 2xH200 acceptance; those results do
not expand #2273 beyond its fixed-slot Multi-LoRA scope. If #2273 lands as
proposed, it replaces the current dataset-driven driver.
</Warning>

## Compatibility and limitations
Expand Down Expand Up @@ -404,5 +411,5 @@ dataset-driven driver.
- `miles/rollout/session/core.py` attaches the single adapter to agentic session
requests.
- `miles/ray/multi_lora/`, `miles/rollout/multi_lora/`, and
`miles/backends/megatron_utils/multi_lora_*.py` implement the multi-adapter
`miles/backends/megatron_utils/api_backends/multi_lora/` implement the multi-adapter
controller, routing, scheduling, optimization, and checkpoint path.
2 changes: 1 addition & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@
"pages": [
"examples/geo3k-vlm",
"examples/geo3k-vlm/multi-turn",
"examples/multi-lora",
"examples/multi-lora-operations",
"examples/on-policy-distillation",
"examples/on-policy-distillation/qwen3-5-35b-selfdistill",
"examples/ppo",
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/index.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

171 changes: 171 additions & 0 deletions docs/examples/multi-lora-operations.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading