Skip to content

fix(ci): make DeviceOffloadPolicy Default derivable (clippy -D warnings) - #575

Merged
justinchuby merged 1 commit into
mainfrom
squad/fix-cuda-clippy-derivable-default
Aug 1, 2026
Merged

justinchuby merged 1 commit into
mainfrom
squad/fix-cuda-clippy-derivable-default

Conversation

@justinchuby

Copy link
Copy Markdown
Owner

CI hotfix — make DeviceOffloadPolicy Default derivable (clippy -D warnings)

Unblocks origin/main — the "CUDA compile (Linux+Windows)" job is RED on every PR (including #574) due to a clippy::derivable_impls failure under -D warnings.

Root cause

#572 flipped DeviceOffloadPolicy::async_pagein's default to false. That made the hand-written impl Default fully equal to a derivable default (all three fields = their type default: bool → false, Option → None), so clippy::derivable_impls now fires at crates/onnx-runtime-ep-cuda/src/weight_paging.rs:117.

Fix (provably behavior-identical)

Replace the hand-written impl Default for DeviceOffloadPolicy with #[derive(Default)] on the struct. The derive produces exactly the same defaults — enabled: false, device_budget_bytes: None, async_pagein: false. from_env() and all runtime behavior unchanged; field doc comments preserved. Net diff: 1 insertion, 11 deletions, one file.

Green confirmation

  • cargo clippy -p onnx-runtime-ep-cuda --features cuda -- -D warnings → exit 0 (the failing gate; no other derivable_impls / -D warnings hits in the crate).
  • CI first step cargo check --locked -p onnx-runtime-ep-cuda -p onnx-runtime-python --features onnx-runtime-python/cuda → exit 0.
  • cargo fmt --all --check → clean.
  • Existing guard test weight_paging::tests::device_policy_defaults_to_disabled (asserts enabled=false, device_budget_bytes=None, async_pagein=false) → passes, locking the derived defaults against field default-drift. No new test needed.

Decision note: .squad/decisions/inbox/cohaagen-cuda-clippy-hotfix.md.

Trivial lint hotfix — fast-track to unblock main.

#572 flipped async_pagein's default to false, making the hand-written
impl Default for DeviceOffloadPolicy fully equal to a derivable default
(enabled=false, device_budget_bytes=None, async_pagein=false). Under the
CI's -D warnings, clippy::derivable_impls then fires
(weight_paging.rs:117), turning the "CUDA compile (Linux+Windows)" job
RED on every PR.

Replace the hand-written impl with #[derive(Default)]. Byte-identical:
the derive produces exactly the same three field defaults. from_env() and
all runtime behavior unchanged; field doc comments preserved.

Verified: cargo clippy -p onnx-runtime-ep-cuda --features cuda
-- -D warnings exits 0; cargo check --locked (CI first step) exits 0;
cargo fmt --check clean. The existing device_policy_defaults_to_disabled
test locks the derived defaults against drift.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@justinchuby
justinchuby enabled auto-merge (squash) August 1, 2026 12:13
@codecov

codecov Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.51%. Comparing base (03f9e60) to head (175e972).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #575      +/-   ##
==========================================
+ Coverage   80.75%   81.51%   +0.76%     
==========================================
  Files         315      315              
  Lines      123737   123737              
  Branches   123737   123737              
==========================================
+ Hits        99920   100869     +949     
+ Misses      19746    18791     -955     
- Partials     4071     4077       +6     
Flag Coverage Δ
cli-ort-linux 86.69% <ø> (ø)
cli-ort-windows 83.84% <ø> (+0.10%) ⬆️
mlas 78.72% <ø> (+0.81%) ⬆️
offline 81.34% <ø> (+0.79%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@justinchuby
justinchuby merged commit 7e8a2ec into main Aug 1, 2026
14 checks passed
@justinchuby
justinchuby deleted the squad/fix-cuda-clippy-derivable-default branch August 1, 2026 12:21
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

⚠️ Benchmark Change Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status Scenario Base PR Change
⚠️ gather/large_f16_threads=1-internal/131072 13.87 µs 17.48 µs +26.0%
⚠️ gather/large_bf16_threads=1-internal/131072 12.54 µs 14.86 µs +18.5%
✅ logit_processing/seven_processor_chain_per_step 1.35 ms 1.51 ms +12.5%
✅ matmul/medium_generic_bf16_threads=8/32x512x512 651.39 µs 729.75 µs +12.0%
✅ matmul/large_generic_f32_threads=8/32x1024x1024 6.30 ms 7.01 ms +11.1%
✅ tokenization/encode_tokens_per_second 424.70 µs 468.86 µs +10.4%
✅ matmul/small_generic_bf16_threads=8/1x256x256 41.69 µs 45.37 µs +8.8%
✅ reduce_mean/large_f32_threads=1-internal/262144 1.11 ms 1.20 ms +8.3%
✅ matmul/large_generic_f32_threads=1/32x1024x1024 10.62 ms 11.19 ms +5.4%
✅ matmul/large_generic_bf16_threads=8/32x1024x1024 2.16 ms 2.23 ms +3.3%
✅ gather/medium_bf16_threads=1-internal/32768 2.81 µs 2.91 µs +3.2%
✅ matmul/small_generic_f32_threads=8/1x256x256 52.89 µs 54.34 µs +2.8%
✅ matmul/small_generic_f16_threads=8/1x256x256 39.87 µs 40.72 µs +2.1%
✅ matmul/medium_generic_bf16_threads=1/32x512x512 593.22 µs 605.17 µs +2.0%
✅ matmul/medium_generic_f32_threads=8/32x512x512 1.62 ms 1.65 ms +1.6%
✅ matmul/medium_generic_f32_threads=1/32x512x512 2.76 ms 2.77 ms +0.4%
✅ matmul/medium_generic_f16_threads=1/32x512x512 41.35 µs 41.10 µs -0.6%
✅ matmul/small_generic_bf16_threads=1/1x256x256 40.81 µs 40.53 µs -0.7%
✅ sampling_latency/top_k_per_token 556.18 µs 548.73 µs -1.3%
✅ sampling_latency/top_p_per_token 1.14 ms 1.13 ms -1.3%
✅ sampling_latency/min_p_per_token 386.36 µs 379.83 µs -1.7%
✅ kv_cache/alloc_dealloc_pages 40.92 µs 40.20 µs -1.8%
✅ tokenization/decode_tokens_per_second 7.56 ms 7.43 ms -1.8%
✅ matmul/medium_generic_f16_threads=8/32x512x512 49.18 µs 47.90 µs -2.6%
✅ matmul/small_generic_f32_threads=1/1x256x256 47.65 µs 46.37 µs -2.7%
✅ matmul/large_generic_f16_threads=1/32x1024x1024 94.26 µs 91.10 µs -3.3%
✅ gather/large_f32_threads=1-internal/131072 40.70 µs 39.00 µs -4.2%
✅ sampling_latency/greedy_per_token 3.77 µs 3.60 µs -4.3%
✅ matmul/large_generic_f16_threads=8/32x1024x1024 113.04 µs 107.88 µs -4.6%
✅ matmul/small_generic_f16_threads=1/1x256x256 42.12 µs 39.89 µs -5.3%
✅ gather/medium_f16_threads=1-internal/32768 2.96 µs 2.78 µs -5.9%
✅ reduce_mean/medium_f32_threads=1-internal/65536 297.55 µs 277.73 µs -6.7%
✅ add/large_f32_threads=1-internal/4194304 52.34 ms 47.90 ms -8.5%
✅ reduce_mean/small_f32_threads=1-internal/4096 19.59 µs 17.44 µs -11.0%
✅ gather/medium_f32_threads=1-internal/32768 5.05 µs 4.48 µs -11.3%
✅ add/large_f16_threads=1-internal/4194304 52.74 ms 46.65 ms -11.6%
✅ add/medium_bf16_threads=1-internal/262144 3.48 ms 3.01 ms -13.6%
✅ gather/small_f32_threads=1-internal/4096 891.3 ns 766.7 ns -14.0%
✅ gather/small_bf16_threads=1-internal/4096 618.5 ns 529.4 ns -14.4%
✅ add/medium_f32_threads=1-internal/262144 3.32 ms 2.82 ms -14.9%
🟢 grammar_masking/llguidance_compute_mask/32 104.39 µs 88.35 µs -15.4%
🟢 matmul/large_generic_bf16_threads=1/32x1024x1024 2.76 ms 2.31 ms -16.5%
🟢 add/large_bf16_threads=1-internal/4194304 58.34 ms 47.35 ms -18.8%
🟢 add/small_bf16_threads=1-internal/1024 17.84 µs 14.31 µs -19.8%
🟢 add/medium_f16_threads=1-internal/262144 3.65 ms 2.89 ms -20.9%
🟢 gather/small_f16_threads=1-internal/4096 689.1 ns 527.6 ns -23.4%
🟢 add/small_f16_threads=1-internal/1024 18.62 µs 14.18 µs -23.9%
🟢 add/small_f32_threads=1-internal/1024 350.3 ns 214.5 ns -38.8%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info
CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.73 3.98 6.06 }
What this cannot catch
  • Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
  • Sub-threshold regressions that compound over multiple PRs
  • Performance changes that only manifest under GPU execution
  • Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant