Kimi K2.x online cross-node datagen over RDMA (retain in-process hidden-states generator)#1

Open

k-l-lambda wants to merge 100 commits into

mainfrom

feature/kimi-rdma

Commits on Mar 6, 2026

Cleanup old implementations/features (#332)
fynnsu
authored

Commits on Mar 9, 2026

examples/data_generation_and_training/qwen3_8b_sharegpt_1k_test.py added.
k-l-lambda
committed

Commits on Mar 10, 2026

feat: add MTP acceptance rate evaluation scripts for K2.5

k-l-lambda
and
claude
committed
feat: bundle K2.5-MTP config files; make --model-config optional

k-l-lambda
and
claude
committed
fix: transformers 5.x compatibility for is_torch_fx_available import

k-l-lambda
and
claude
committed
feat: support both K2.5 (INT4 GPTQ) and V3 (FP8) in MTP acceptance eval

k-l-lambda
and
claude
committed
updated gitignore.
k-l-lambda
committed
feat: support dual-model MTP eval (K2.5 INT4-GPTQ + V3 FP8); fix vLLM 0.17 compat

k-l-lambda
and
claude
committed
fix: regex ordering in load_mtp_state_dict_v3 to correctly separate weight from weight_scale_inv

k-l-lambda
and
claude
committed
feat: add MTP training module (Phase A) for K2.5 speculative decoding
k-l-lambda
committed
fix: MTP smoke test bug fixes — causal mask, eval mode, bfloat16, path fixes
k-l-lambda
committed

Commits on Mar 11, 2026

Commits on Mar 12, 2026

Commits on Mar 13, 2026

Commits on Mar 17, 2026

fix: remove eos_token_id from Request() for vLLM 0.17.1 compat; add INT4 hidden states extraction script

k-l-lambda
and
claude
committed

Commits on Mar 18, 2026

feat: INT4 hidden states extraction + eval ablation support

k-l-lambda
and
claude
committed
feat: add MTP KL loss support and fix transformers 5.x compat

k-l-lambda
and
claude
committed
Merge remote-tracking branch 'youyun38/main'
k-l-lambda
committed
feat: greedy dataset gen, draft logits comparison, prefill vs decode test

authored and
k-l-lambda
committed
fix: capture post-norm hidden states for last layer in VllmHiddenStatesGenerator

k-l-lambda
and
claude
committed

Commits on Mar 19, 2026

feat: top-100 verifier logits in dataset + MTP KL loss with sparse logits

k-l-lambda
and
claude
committed
fix: gathered KL loss (no dense [B,S,V] tensor), RMSNorm eps from config, paired field guard

k-l-lambda
and
claude
committed
fix: zero position-0 embedding in MTP forward to match vLLM serving

k-l-lambda
and
claude
committed
fix: remove broken arg parser entries and bare args reference in eval script
k-l-lambda
committed

Commits on Mar 20, 2026

Commits on Mar 22, 2026

feat: add enforce_eager param to VllmHiddenStatesGenerator; enforce_eager=False works for h_t extraction (response cosine 0.995 vs production serving)
k-l-lambda
committed

Commits on Mar 23, 2026

add: standardized benchmark concurrency sweep script (200 prompts, sharegpt format)
k-l-lambda
committed

Commits on Mar 26, 2026

Commits on Mar 27, 2026

add: server-side hidden states extraction script (enforce_eager=False)
k-l-lambda
committed

Commits on Apr 1, 2026

feat: add dynamic hidden states training mode (--dynamic)

k-l-lambda
and
claude
committed
fix: use SDPA attention for kimi_k2 Eagle3 training (eager OOMs on packed sequences)

k-l-lambda
and
claude
committed
feat: support pre-tokenized HF datasets in --dynamic mode

k-l-lambda
and
claude
committed
fix: relax token count assertion in dynamic trainer (allow ±1 for BOS/EOS)

k-l-lambda
and
claude
committed
fix: address review feedback — SDPA guard, prefetcher deadlock, failure tracking

k-l-lambda
and
claude
committed
feat: add --enforce-eager flag for dynamic training, default False

k-l-lambda
and
claude
committed
test: add unit tests for dynamic training components (CPU-only)

k-l-lambda
and
claude
committed
fix: increase skip rate abort threshold from 10 to 50 batches

k-l-lambda
and
claude
committed

Commits on Apr 2, 2026

debug: add batch composition logging to _generate_batch errors

k-l-lambda
and
claude
committed
fix: sort request IDs by original index, not string order

k-l-lambda
and
claude
committed
scripts: add ZClawBench eval scripts and benchmark utilities

k-l-lambda
and
claude
committed

Commits on Apr 3, 2026

feat: add gen_hf_dataset_from_csv.py for dynamic training data prep

k-l-lambda
and
claude
committed
feat: gen_hf_dataset_from_csv — full CSV scan + 16k seq_length

k-l-lambda
and
claude
committed
fix: gen_hf_dataset_from_csv syntax errors + clean rewrite

k-l-lambda
and
claude
committed
fix: right-truncate response instead of left-truncating prefix in gen_hf_dataset_from_csv

k-l-lambda
and
claude
committed

Commits on Apr 4, 2026

Fix loss mask trimming in hidden-state extraction
k-l-lambda
committed

Commits on Apr 8, 2026

feat: add --val-epoch-samples to cap validation set size per epoch
k-l-lambda
committed

Commits on Apr 14, 2026

train: run initial val_epoch before epoch 0 to record pretrain baseline
k-l-lambda
committed

Commits on Jun 3, 2026

Commits on Jun 4, 2026

Commits on Jun 8, 2026

updated documents.
Camus
committed

Commits on Jun 9, 2026

fix: harden FieldInfo materialization in SpeculatorModelConfig serialization
Camus
committed