Skip to content

tests/mlx_parity: 7-probe MLX vs HF bisection + Mac M1 workflow #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
danielhanchen wants to merge 84 commits into
base: main
Choose a base branch
from
Open

tests/mlx_parity: 7-probe MLX vs HF bisection + Mac M1 workflow #119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
5065a35
tests/mlx_parity: 7-probe MLX vs HF parity bisection + Mac M1 workflow
danielhanchen May 17, 2026
c3b0c9e
.github/workflows: drop unrelated workflows on this branch
danielhanchen May 17, 2026
4ab7509
mlx_parity: fan out across 10 parallel Mac jobs + 3 deeper probes
danielhanchen May 17, 2026
83025da
mlx_parity: fix probes 5 + 10, add probe 11 (fp32 trainer curve)
danielhanchen May 17, 2026
8b40544
mlx_parity/probe_5: use mlx.nn.value_and_grad + aggregate norms
danielhanchen May 17, 2026
aa2a7f5
mlx_parity: probe 12 -- pin unsloth-zoo to parent of PR #634
danielhanchen May 17, 2026
7b36a6e
mlx_parity: prune matrix to 4 Mac jobs, add probe 13 (pure mlx-lm)
danielhanchen May 17, 2026
f3f9503
mlx_parity: probe 14 + 15 -- test two fix candidates in parallel
danielhanchen May 17, 2026
a61bf0f
mlx_parity: probe 16 -- mlx-lm NATIVE LoRA trainer, no unsloth
danielhanchen May 17, 2026
796abd9
mlx_parity/probe_16: pad valid.jsonl so mlx_lm.lora loader accepts it
danielhanchen May 17, 2026
4acc5e0
mlx_parity: extended probes -- longer steps, seeds, mlx-lm 50 iters
danielhanchen May 17, 2026
49b6e9d
mlx_parity/probe_17: treat empty env-vars as defaults
danielhanchen May 17, 2026
b721a0f
mlx_parity Round B: scan bc=True convergence boundary + 2 more seeds
danielhanchen May 17, 2026
ba44388
mlx_parity Round C: zoom in on bc=True convergence boundary
danielhanchen May 17, 2026
0ca06a6
mlx_parity Round D: lock in steps=20 across seeds + 50-step stability
danielhanchen May 17, 2026
c1bc687
mlx_parity Round E: max_steps=30 seed-robustness + LR sweep
danielhanchen May 17, 2026
380255d
mlx_parity Round F: investigate seed=12345 generation failure
danielhanchen May 17, 2026
6ac2637
mlx_parity Round G: 60-step universality + 12345 basin entry
danielhanchen May 17, 2026
db0407e
mlx_parity Round H: pivot to mlx-lm reference comparison
danielhanchen May 17, 2026
347d0fb
mlx_parity Round I: mlx-lm at higher iter counts
danielhanchen May 17, 2026
9b98c68
mlx_parity Round J: bc=False vs bc=True end-to-end on PR-663 head
danielhanchen May 17, 2026
9e10b28
mlx_parity Round K: confirm new default + bc=False long-horizon
danielhanchen May 17, 2026
ad4e9cf
mlx_parity Round L: bc=False divergence boundary + default universality
danielhanchen May 17, 2026
d01f6dd
mlx_parity Round M: narrow bc=False NaN boundary + plumbing equivalence
danielhanchen May 18, 2026
deeef59
mlx_parity Round N: final sanity pass on PR-663 head
danielhanchen May 18, 2026
69b37fb
mlx_parity Round O: counterfactual on bc=True at long horizons
danielhanchen May 18, 2026
a2907a0
mlx_parity Round P: bc=False NaN to single-step + 1000-step bc=True
danielhanchen May 18, 2026
a765616
mlx_parity Round Q: LR axis at long horizons
danielhanchen May 18, 2026
598518c
mlx_parity Round R: extend safe-envelope claims from Round Q
danielhanchen May 18, 2026
aa122fc
mlx_parity Round S: fresh-seed pass-rate sampling on PR-663 head
danielhanchen May 18, 2026
d8ca629
mlx_parity Round T: failing-seed basin recurrence at long horizons
danielhanchen May 18, 2026
c7e7618
mlx_parity Round U: confirm 500-step basin recovery generalizes
danielhanchen May 18, 2026
7efd103
mlx_parity Round V: do empty-output seeds (999, 2024) recover at 200/…
danielhanchen May 18, 2026
5ea0ff3
mlx_parity Round W: LR x bc on persistent-failure seeds
danielhanchen May 18, 2026
b35ff58
mlx_parity Round X: hunt for universal (bc, lr, steps) config
danielhanchen May 18, 2026
eee1393
mlx_parity Round Y: expand seed cube at smoke config
danielhanchen May 18, 2026
93885ef
mlx_parity Round Z: determinism check
danielhanchen May 18, 2026
67be132
mlx_parity Round AA: bc=True at lr=1e-4 + 500 head-to-head
danielhanchen May 18, 2026
0f997e1
mlx_parity Round AB: expose grad-clip knobs + start Q1 sweep
danielhanchen May 18, 2026
994fef6
mlx_parity Round AC: value=1.0 multi-seed pass-rate vs norm=1.0
danielhanchen May 18, 2026
a87b0c7
mlx_parity Round AD: finish value=1.0 13-seed sample
danielhanchen May 18, 2026
7bd5de3
mlx_parity Round AE: value=5.0 multi-seed (PR #634 old default)
danielhanchen May 18, 2026
9d8de62
mlx_parity Round AF: value=0.5 multi-seed (does MLX over-clip?)
danielhanchen May 18, 2026
fb5b667
mlx_parity Round AG: complete value=5.0 13-seed sample
danielhanchen May 18, 2026
ae6b085
mlx_parity Round AH: finish value=0.5 + value=5.0 samples
danielhanchen May 18, 2026
0240307
mlx_parity Round AI: fp32 dtype hypothesis test
danielhanchen May 18, 2026
57e503f
mlx_parity Round AJ: bc=True at lr=1e-4 + 500 (Q3 head-to-head)
danielhanchen May 18, 2026
9411d0a
mlx_parity Round AK: confirm bc=True at lr=1e-4+500 underperforms
danielhanchen May 18, 2026
cee07e0
mlx_parity Round AL: bc=True Goldilocks zone search
danielhanchen May 18, 2026
5816ea0
mlx_parity Round AM: bc=False at intermediate envelope
danielhanchen May 18, 2026
ad1b51b
mlx_parity Round AN: complete value=0.5 13-seed sample
danielhanchen May 18, 2026
a121a46
mlx_parity Round AO: norm-clip threshold sweep
danielhanchen May 18, 2026
f8419f6
mlx_parity Round AP: expand norm=0.5 sample
danielhanchen May 18, 2026
5f30f39
mlx_parity Round AQ: value=0.25 (tighter than 0.5 winner)
danielhanchen May 18, 2026
3e23d37
mlx_parity Round AR: value=0.1 (tighter than 0.25)
danielhanchen May 18, 2026
ad14b1f
mlx_parity Round AS: complete value=0.25 13-seed sample
danielhanchen May 18, 2026
2e3d446
mlx_parity Round AT: finish value=0.25 13-seed sample
danielhanchen May 18, 2026
afba8d2
mlx_parity Round AU: mlx-lm with unsloth-zoo-matching settings
danielhanchen May 18, 2026
f51e637
mlx_parity Round AV: validate teacher-forced completion loss on
danielhanchen May 18, 2026
2739fee
mlx_parity Round AW: 2x2 binary-search of mlx-lm-vs-unsloth-zoo gap
danielhanchen May 18, 2026
d4f125d
mlx_parity Round AX: expand n to 15 seeds across mlx-lm vs unsloth-zoo
danielhanchen May 18, 2026
99def42
mlx_parity Round AY: hybrid loader-vs-trainer disambiguation (probe_21)
danielhanchen May 18, 2026
84c2aca
mlx_parity Round AZ: numpy-RNG hypothesis + triple-confirm mlx-lm
danielhanchen May 18, 2026
e1756a2
mlx_parity Round BA: compile-mode hypothesis (probe 23)
danielhanchen May 18, 2026
1614105
mlx_parity Round BB: loss-fn dtype propagation (probe 24)
danielhanchen May 18, 2026
0eaa960
mlx_parity Round BC: bracket trainer-loop vs loss as gap source
danielhanchen May 18, 2026
a39f7df
mlx_parity Round BD: pure-mlx-lm control (probe 26)
danielhanchen May 18, 2026
7d0875b
mlx_parity Round BE: isolate subprocess vs inline mlx-lm at 67% vs 47%
danielhanchen May 18, 2026
7e18b7d
mlx_parity Round BF: THE FIX TEST — seed mx.random AFTER model load
danielhanchen May 18, 2026
ad7d4db
mlx_parity Round BG: THE FIX — num_layers=16 (mlx-lm CLI default)
danielhanchen May 18, 2026
b1be7e3
mlx_parity Round BH: end-to-end fix verification (probe 32)
danielhanchen May 18, 2026
8ac40ae
mlx_parity Round BI: bisect zoo loader vs trainer at num_layers=16
danielhanchen May 18, 2026
b757947
mlx_parity Round BJ: dtype-cast hypothesis (probe 34)
danielhanchen May 18, 2026
c73f796
mlx_parity Round BK: MLXTrainer compile-flag hypothesis (probe 35)
danielhanchen May 18, 2026
93ca90a
mlx_parity Round BL: pin trainer-side cause (probes 36 + 37)
danielhanchen May 18, 2026
956fd1c
mlx_parity Round BM: re-baseline probe 31 to reset confidence
danielhanchen May 18, 2026
b27d4f1
mlx_parity Round BO: pin probe matrix to pad-fix branch (b265d99)
danielhanchen May 18, 2026
e015600
mlx_parity Round BP: strict step-by-step parity probe (probe 38)
danielhanchen May 19, 2026
55e0896
mlx_parity: re-trigger Round BP after spurious cancellation
danielhanchen May 19, 2026
261a66b
mlx_parity Round BP: fix probe 38 callback arg index for grad_norm
danielhanchen May 19, 2026
72fcfbc
mlx_parity Round BQ: FastMLXModel.get_peft_model parity probe (probe 39)
danielhanchen May 19, 2026
12dba6f
mlx_parity Round BR: pin probe matrix to PR #674 seed-ordering fix
danielhanchen May 19, 2026
552ae63
mlx_parity Round BS: bisect MLXTrainer vs manual-loop residual gap
danielhanchen May 19, 2026
c63bc2c
mlx_parity Round BT: test whether elementwise clip-at-1 IS the residu…
danielhanchen May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,265 changes: 0 additions & 2,265 deletions .github/workflows/consolidated-tests-ci.yml
View file
Open in desktop

This file was deleted.

321 changes: 0 additions & 321 deletions .github/workflows/lint-ci.yml
View file
Open in desktop

This file was deleted.

Loading
Loading