Skip to content

fit : add --fit-show-mem to print probe table at INFO#23232

Open
Bikkies wants to merge 1 commit into
ggml-org:masterfrom
Bikkies:fit-show-mem
Open

fit : add --fit-show-mem to print probe table at INFO#23232
Bikkies wants to merge 1 commit into
ggml-org:masterfrom
Bikkies:fit-show-mem

Conversation

@Bikkies

@Bikkies Bikkies commented May 17, 2026

Copy link
Copy Markdown

Overview

Adds a flag -fsm / --fit-show-mem [on|off] that surfaces the per-device free-memory table from common_params_fit_impl at the INFO log level.

This is the table that used to show up by default before #23021 demoted it to TRACE. I found it really handy when manually adjusting params like tensor splits and context. Being able to see quickly how much VRAM was free per device made iterating on tensor splits to balance vram quicker without going through a full model load each time.

Adding this flag lets this table show and be easily visible without disappearing into the sea of other output when using higher verbose log output.

Personally, I think it should be on by default. I didn't even know it existed until it was showing up in my
terminal output, and others may now miss out on it the same way now that it's hidden behind TRACE. But
since #23021 deliberately moved it, this PR adds it as opt-in rather than overriding that decision.

Additional information

Requirements

  • I have read and agree with the contributing guidelines
  • YES
  • AI usage disclosure:
  • YES - I used Claude Opus 4.7 to help implement this and make sure what I was doing was correct and I didn't miss anything.

- Opt-in flag that surfaces the per-device free-memory probe table (currently TRACE-only after ggml-org#23021) at INFO when set; default off.
@Bikkies Bikkies requested review from a team and JohannesGaessler as code owners May 17, 2026 21:46
@JohannesGaessler

Copy link
Copy Markdown
Contributor

The console output for --verbosity 4 has unintentionally become a lot more noisy due to a bug. I pushed a fix in #23282 . The fixed version with --verbosity 4 should be equivalent to what you got before which would I think be preferable over adding a completely new CLI argument.

What should or should not be shown by default is I think another issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants