Skip to content

Studio: scope "Remember settings next time" per GGUF quant and apply on every load path - #6594

Merged
wasimysaid merged 5 commits into
unslothai:mainfrom
oobabooga:studio-remember-load-settings-improvements
Jun 23, 2026
Merged

wasimysaid merged 5 commits into
unslothai:mainfrom
oobabooga:studio-remember-load-settings-improvements

Conversation

@oobabooga

@oobabooga oobabooga commented Jun 23, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #6364, which added "Remember settings next time". This scopes those saved settings per GGUF file and applies them no matter how you load the model.

Problem

#6364 keyed saved settings by repo id, so every quant of a GGUF repo shared one entry. But the load knobs it remembers (context override, KV-cache dtype, tensor-parallel) are VRAM-budget settings that differ per quant. The save and restore also only ran through the staged settings sheet. Loading the same model with "Load on selection" on, or from the Hub run button, silently ignored the saved config.

Fix

Key per quant: rememberedLoadSettingsKey(selection) folds the GGUF variant into the key (id::Q4_K_M), so each quant has its own entry. Local .gguf paths and native files key by their existing file-specific id and are unchanged.

Apply on every load path: a shared applyRememberedLoadSettings store action seeds the load knobs the way the sheet's restore effect does. It now also runs on the chat "Load on selection" path and the Hub run path, so a saved config is honored however the model is loaded (immediate, after-download autoload, staged Load button, native drop, Hub). keepSpeculative carries the restored speculative choice across the model switch only when a config exists.

Known limitation

The General Compare view loads models through its own loader (shared-composer.tsx), which doesn't apply remembered settings. It predates this feature and never passed kv_cache_dtype. Bringing that loader to parity is a separate follow-up.

Verification

Typecheck clean, no net-new lint. Checked that a saved config wins over autodetected or inherited values, and that save and load use the same per-quant key on every standard load path.

cc @shimmyshimmer

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

@wasimysaid
wasimysaid merged commit 2193b7f into unslothai:main Jun 23, 2026
34 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants