PR #7074 staging CI: Studio Voice settings tab - #89
Closed
danielhanchen wants to merge 750 commits into
Closed
danielhanchen wants to merge 750 commits into
danielhanchen wants to merge 750 commits into
Conversation
* fix(studio): sort export checkpoints by step * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
…lama.cpp prebuilt selection (unslothai#6584) * Studio: treat data-center Blackwell (sm_100/sm_103) as Blackwell in llama.cpp prebuilt selection _host_is_blackwell gated on _BLACKWELL_MIN_SM = 120, but data-center Blackwell parts report a lower compute capability than consumer Blackwell: B100/B200 are sm_100 and B300/GB300 are sm_103, while RTX 50 is sm_120 and DGX Spark is sm_121. Because 100 and 103 are both < 120, every data-center Blackwell host was classified as non-Blackwell, so two GPU-targeting paths never fired for a B200/B300: - the Linux blackwell_runtime_override that prefers the highest CUDA-major runtime line shipping a bundle covering the host SMs (so a cu12x torch could pin a cuda12 bundle over a native cuda13 one), and - _drop_blackwell_incapable_windows_cuda, which removes cuda-12.4 builds that load and validate but run Blackwell on a slow PTX-JIT path. The result is a B200/B300 being handed a prebuilt that does not natively offload its SM, i.e. the llama.cpp prebuilt is not really for the GPU. The Blackwell floor is sm_100, so set _BLACKWELL_MIN_SM = 100. The toolkit floor (12.8) is unchanged and already correct for sm_100/sm_103. Surfaced loading unsloth/GLM-5.2-GGUF UD-IQ1_S on 8x B200. Adds tests covering the sm_100/sm_103 classification, the Linux cuda13 preference for a data-center host, and the Windows cuda-12.4 drop. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim comments to be succinct (no behavior change) * studio: require CUDA 12.9 for sm_103/sm_121 Blackwell prebuilts sm_103 (B300/GB300) and sm_121 (DGX Spark) have no native compiler target before CUDA 12.9; the family floor of 12.8 only covers sm_100/101/120. Make the Windows-CUDA Blackwell filter SM-aware so a legacy win-cuda-12.8 bundle is dropped on an sm_103/sm_121 host while sm_100/sm_120 hosts keep the 12.8 floor. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…othai#6605) * Studio: fall back to anonymous HF browsing on a malformed token The Discover/Recommended feeds call the Hugging Face JS client (`listModels`/`listDatasets`) directly from the browser. That client throws `Your access token must start with 'hf_'` when handed a non-empty token that isn't a well-formed HF token, instead of falling back to anonymous access. A single bad value left in the HF token field (e.g. a placeholder someone typed) therefore takes down the entire discovery feed even though it works fine with no token at all. Add `hfApiToken()` to the HF token store, which returns the token only when it looks like a real `hf_...` credential and `undefined` otherwise, and route hub-page's four HF call sites through it. Malformed tokens now degrade to anonymous public browsing rather than erroring. The raw token is still stored and shown in the settings field unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Studio: trim hfApiToken comments Collapse the 11-line JSDoc to a 2-line note and drop the redundant call-site comment in hub-page. AST signature check confirms code is unchanged (comments only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
) * Chat: match reasoning thinking icon to the composer bulb The reasoning "Thinking..." indicator used lucide's LightbulbIcon while the composer thinking toggle used a custom bulb glyph, so the two did not match. Move that glyph into lib/bulb-icon.tsx and use it in both places so they render the same icon. * Let BulbIcon take and override svg props --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
PR unslothai#6364 added a branch that overrode the unsloth owner avatar with the bundled circle-logo-small.png sticker. Revert it so unsloth uploads use the live Hugging Face org avatar again, falling back to the colored initial tile. Upstream re-uploads are unaffected since they still resolve through provider logos. Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
…on every load path (unslothai#6594) * Studio: remember load settings per GGUF quant * Studio: apply remembered load settings on load-on-selection * Studio: apply remembered load settings on the Hub load path * Studio: reset staged knobs before seeding remembered settings on Hub load
…ao) (unslothai#6608) torchao has no working Windows ROCm build. transformers.quantizers imports it, and it loads torch's c10d distributed backend at module level, which the AMD Windows wheels omit (no RCCL). The import aborts, transformers can no longer expose PreTrainedModel, and the sentence-transformers embedder silently falls back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected (the c10d ops are present / torchao is real there). The training and export workers already install the shared torchao stub before importing transformers, but the RAG embedder runs in the main backend process, which never did. Two fixes, both no-ops off Windows ROCm: - embeddings.py: install_torchao_windows_rocm_stub() before the first sentence-transformers import, so an already-installed torchao is neutralized (fixes existing venvs). - install_python_stack.py: stop installing torchao on Windows ROCm; it can only crash on import there, so new venvs never ship it. Add tests covering the embedder stub call and the install skip.
…nd add --api-only to `unsloth studio run` (unslothai#6591) * Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run` --secure exposes ONLY the Cloudflare link (it forces a loopback bind), but _cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so `run.py --secure --api-only` started no tunnel and then fail-closed with "A secure Cloudflare link is not allowed". That blocked the natural headless use: serve just the API (no web UI) over the authenticated tunnel. Make --secure start the tunnel regardless of api_only (the non-secure path is unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then expose --api-only on `unsloth studio run` and forward it through both the re-exec args and the in-venv run_server call, so `unsloth studio run --secure --api-only --model ...` is a one-liner secure API server. Verified end to end: `run.py --secure --api-only` now brings up the tunnel and serves /api/health over it (200), with / returning 404 (no UI). Tests: update the tunnel-gate truth table (secure+api-only now tunnels; secure+colab still does not) and add --api-only registration + re-exec/in-venv forwarding coverage to the run CLI tests. * Trim comments to be succinct (no behavior change) * studio: address review on parent --api-only and secure api-only CORS - Reject --api-only on the parent `unsloth studio` group when a subcommand is invoked, with the same redirect guidance used for --parallel/--secure; otherwise the flag was silently dropped and the UI served anyway. - Keep CORS any-origin for secure api-only serving: that mode publishes the API over Cloudflare for remote browser clients, so the Tauri-only lockdown (still applied to plain local api-only) would break preflight. Factored the decision into cors_origins_for_mode() and gate it on api_only and not secure; run_server exports UNSLOTH_SECURE before importing main. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: suppress TAURI_PORT and de-dup test for headless run --api-only - run_server gains emit_tauri_port (default True, unchanged for the Tauri/ desktop path). The new headless `run --api-only` path passes False so the Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key banner (it ran even under --silent and could break one-liner parsers). - Remove a duplicate test_reexec_forwards_api_only that shadowed the parametrized one; fold the --secure --api-only case into it so the secure headless path is actually collected. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…nslothai#6581) unslothai#6579 reworded these comments to attribute the failure to a half-resolved install and claimed a clean 4.14 is fine on 3.13. That is wrong: unslothai#6483 is a genuine anyio 4.14 + Python 3.13 regression. 4.14 added a per-task cancel scope in its asyncio backend (TaskHandle/_run_coro) that gets exited in the wrong task under starlette's collapsing task group, raising the cancel-scope RuntimeError on streaming; 4.13 has no such code and is unaffected (the reporter confirmed 4.13.0 fixes it). The TaskHandle ImportError is only the secondary macOS-arm symptom from the mlx-vs-cap version fight. Comments only.
…models (unslothai#6521) * studio: report the true reasoning duration and fix the Stop button for thinking models For a local GGUF the "Thought for N" label was timed entirely on the client by a brittle edge-detector, so an always-think model (Qwen3 MTP) that buffers its whole reasoning and flushes it in one chunk showed "1 second" instead of the real minute-plus. The client cannot time reasoning it receives atomically, so make the timing backend-authoritative. Backend: generate_chat_completion_with_tools measures wall-clock reasoning and emits a Studio reasoning_summary event (duration_ms) at the moment reasoning ends -- the first answer token, or end-of-stream for a reasoning-only reply -- for both the tool-detection pass and the final-answer pass. Timing resets per tool iteration so the final answer's thinking time wins on the client (which takes the latest reasoning_summary). routes/inference.py forwards the event in the GGUF tool stream. Frontend: parse the reasoning_summary SSE into a _reasoningDurationMs chunk and use it as the authoritative reasoning duration (last write wins), clamped to >= 0 and guarded to a finite number so a malformed or proxied chunk cannot produce a NaN label; the persisted value wins for the final "Thought for N" label, with the previous live timer kept only as a fallback when no metadata arrives. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Gemma 4 GGUF OpenAI API streams * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Avoid duplicate Responses stream disconnect watcher * Keep reasoning-only Responses output hidden * Address Gemma stream review comments * Avoid Responses stream task-group cleanup * Harden OpenAI chat completion streams * Address OpenAI stream review issues * Clean up Studio OpenAI stream helpers * Fix Studio passthrough cold stream timeout * Fix tool parser compatibility exports lint * Preserve audio stream disconnect cancellation * Avoid synthetic finish after passthrough errors * Address stream cleanup and Gemma parser reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gemma 4: parse bare-string tool args and keep safetensors tools for native <|tool_call> - Quote bare unquoted string values in Gemma native tool-call args (e.g. {location:Tokyo,unit:celsius}) so they parse; JSON scalars stay typed. - Stop _detect_safetensors_features from suppressing supports_tools for templates that emit Gemma native <|tool_call>, which the shared parser now reads. - Add tests for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden Gemma tool-call parsing and stream-error detection Address three issues in the Gemma-native tool-call path: - _quote_gemma_object_keys stopped a bare (unquoted) string value at the first comma, so an argument like `location:New York, NY` was split mid-value and the synthesized JSON failed to parse, dropping the whole tool call. A bare value now ends only at `}` or a comma that begins the next `key:` pair. - parse_tool_calls_from_text scanned the entire response for Gemma markers even inside a tool call already parsed from a `<tool_call>{...}` JSON block, so a marker-like string inside an argument (data) was promoted to a second, unintended tool call. Matches inside an already-consumed call span are now skipped. - _openai_passthrough_stream relied on _monitor_openai_sse_line to flag a stream error, which returns early when monitor_id is None (skip_api_monitor), so an upstream error chunk left saw_stream_error unset and the synthetic-finish guard emitted a successful finish_reason after a failed stream. Error chunks are now detected independently of API monitoring. Adds tests/test_gemma_tool_parse_edge_cases.py covering the comma and marker-injection cases. * Emit the terminal finish_reason chunk in GGUF streams The OpenAI chat-completions GGUF tool stream and plain stream both built a final ChatCompletionChunk carrying finish_reason but never yielded it, so clients received the optional usage chunk and [DONE] with no chunk carrying finish_reason. OpenAI-compatible consumers rely on that terminal choice to distinguish stop/length/tool_calls. Yield it before the usage chunk and [DONE], matching the other streaming paths. * Parse tool calls in document order and skip nested markers both ways Unify the JSON- and Gemma-format tool-call passes into a single position-ordered scan: - Calls are now emitted in byte order across both formats, so a mixed output like `<|tool_call>call:create{...}<tool_call|> ... <tool_call> {"name":"read",...}</tool_call>` executes create before read, matching the order they appear in (tools run in returned order). - A candidate that starts inside an already-accepted call's span is skipped, in both directions: a JSON marker inside a Gemma argument and a Gemma marker inside a JSON argument are treated as data, not promoted to a second executable tool call. Extends tests/test_gemma_tool_parse_edge_cases.py with the ordering and JSON-in-Gemma nesting cases. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Quote bare Gemma array elements; order finish before trailing usage - _quote_gemma_object_keys skipped array values, so a Gemma call with a bare-string array argument like labels:[bug,ui] produced invalid JSON and the whole tool call was dropped. Array values are now scanned and bare string elements quoted, while numbers, quoted strings, and JSON literals are preserved. - In the OpenAI passthrough stream, a trailing usage-only chunk (stream_options.include_usage) that arrived before any finish chunk was relayed before the synthetic finish, producing usage -> finish -> [DONE]. Emit the synthetic finish before that usage chunk so the order matches the other streams (finish -> usage -> [DONE]). Extends tests/test_gemma_tool_parse_edge_cases.py with the bare-array cases. * Harden Gemma array parsing, XML-parameter guard, and stream teardown Address five review findings on the Gemma tool-call and OpenAI passthrough streaming paths: - parse_tool_calls_from_text collected JSON and Gemma markers without the _inside_open_parameter guard, so a marker embedded in an existing <function=...><parameter=...> value was promoted to a separate tool call. Candidates that start inside an open XML parameter are now skipped, matching the guard the XML-style parser already applies. - _quote_gemma_array_elements preserved array elements starting with { or [ verbatim, so an array of objects (items:[{path:a}]) or a nested array failed json.loads and the whole call was dropped. Object and nested-array elements are now normalised recursively. - _openai_passthrough_stream synthesized a finish chunk before a trailing usage-only chunk and set saw_finish_reason, which made the EOF guard skip the [DONE] sentinel. The EOF path now emits [DONE] whenever the upstream omitted it, even after a finish chunk was already synthesized. - /generate/stream drove generation through asyncio.to_thread with no disconnect watcher, so a client disconnect during a long generation went unnoticed until the next send. It now runs _await_disconnect_then_cancel against the request, matching the other local streaming endpoints. - _SameTaskStreamingResponse closed the body iterator with aclose() on a send-side disconnect, raising GeneratorExit so the generators' cancellation handlers (which finish the api_monitor entry) never ran. It now throws CancelledError, falling back to aclose() when athrow is unavailable. Extends tests/test_gemma_tool_parse_edge_cases.py with array-of-objects, nested-array, and marker-inside-XML-parameter cases. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Watch disconnects on Anthropic streams; keep timestamps in Gemma values Two follow-ups on the streaming and tool-parse paths: - _anthropic_tool_stream and _anthropic_plain_stream drove generation through asyncio.to_thread(next, gen, ...) and only polled is_disconnected() between events, so a client disconnect during prefill or a long generation/tool step held the decode slot until the next event or a failed send. Both now run the _await_disconnect_then_cancel watcher used by the other local streams, stop it in finally, and break promptly when cancel_event is set. - _GEMMA_NEXT_KEY_RE treated any comma followed by word-chars-then-colon as the next key, so a bare value such as "meet at 10:00, 11:00 tomorrow" was split into bogus keys. The next-key token must now be identifier-shaped (start with a letter or underscore), so a comma before a timestamp, ratio, or other numeric-then-colon text stays part of the value. Adds a timestamp-in-bare-value regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard nested markers, reset on disconnect, clean unstarted streams Three follow-ups on the tool-parse and streaming paths: - parse_tool_calls_from_text only skipped markers that fell inside a span it had already parsed successfully, so when an unquoted Gemma argument contained a literal marker (code:<|tool_call>call:terminal{...}<tool_call|>) the outer object failed to normalize, its span was never recorded, and the inner marker was promoted to a standalone terminal call. Candidates nested inside any other candidate's brace span are now skipped regardless of whether the enclosing candidate parsed, so a marker in malformed outer data is never executed. - /generate/stream skipped backend.reset_generation_state() when the disconnect watcher set cancel_event between chunks: the loop broke and the finally's reset is guarded on cancel_event being unset. A subprocess backend kept decoding after the client left. The cancel-break path now resets the backend. - _SameTaskStreamingResponse threw CancelledError / called aclose() on the body iterator on a send-side disconnect, but neither runs the try/finally of a generator that never started (early disconnect on http.response.start), so the passthrough's eagerly-opened upstream httpx stream and cancel-registry entry leaked. It now tracks whether the body started and, when it did not, runs an optional unstarted_cleanup hook; the OpenAI passthrough wires it to close the upstream resp/client and exit the cancel tracker. Adds a nested-unquoted-marker regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
… blocked (unslothai#6596) * Studio: lazy-import matplotlib so the server starts when the wheel is blocked matplotlib.pyplot was imported at the top of core/training/training.py, on the server boot path. When matplotlib's native extension fails to load (e.g. an unsigned wheel blocked by Windows Smart App Control), that import crashed the whole Studio server at startup instead of just disabling loss plots. Move it into a lazy _load_pyplot() helper called from _create_loss_plot, using the headless Agg backend, and return None when matplotlib is unavailable so plotting degrades gracefully. The plot return was already Optional, so callers need no changes. Keep the type-only import under TYPE_CHECKING and quote the annotations. Fixes unslothai#6588 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: pin matplotlib==3.11.0 Pin matplotlib to the current latest so a new unsigned release does not reintroduce the Smart App Control block on Windows. Belt-and-suspenders on top of the lazy import. Pinned in both studio.txt and extras.txt. * Pin matplotlib to 3.10.9 so Studio still installs on Python 3.10 matplotlib 3.11.0 requires Python >=3.11, so the pin had no installable wheel on Python 3.10 (still supported) and pip install failed there. 3.10.9 is the latest 3.10.x (requires-python >=3.10) and covers Python 3.10 through 3.13. Also tighten the lazy-import docstrings. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>
…lothai#6545) * studio: add sidebar update button (static design only) Adds a clock-icon update card above the account button in the sidebar footer. Visual/layout only; update detection and click behavior are wired in follow-ups. * studio: show installed version in sidebar update card + collapse to icon Replaces the placeholder version with the real installed app version via @tauri-apps/api/app getVersion() (Tauri-only; hidden in browser). Collapsed sidebar now shows just the clock icon instead of hiding the card. * studio: open Settings About (update section) when the sidebar update card is clicked * studio: i18n the sidebar update button label and add aria-label Address Gemini Code Assist review on unslothai#6545: - wrap the hardcoded "Update available" label in t() (shell.updateAvailable, en + zh-CN), matching the rest of the sidebar - add aria-label so the collapsed icon-only button has an accessible name Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * studio: hide sidebar update card unless an update is available Gates the card on useWebUpdateCheck so it stays hidden by default on both web and desktop, appearing only when the installed PyPI version is behind the latest release. Includes a TEMP localStorage dev override (devForceUpdateCard) to preview the card where there is no real update; remove before merge. Keeps the i18n label/aria-label; desktop (Tauri updater) detection not wired yet. * Polish Studio sidebar update affordance --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>
…3N audio LoRA) (unslothai#6558) * vision/llama: thread finetune_audio_layers through get_peft_model Pass finetune_audio_layers to get_peft_regex so FastModel / FastVisionModel can adapt the Gemma 4 / Gemma 3N audio encoder and the audio/vision embedder projectors. Enabled automatically under target_modules="all-linear", included in the explicit-target layer-scope filter, and added to the UNSLOTH_USE_NEW_MODEL kwargs defaults. Requires the matching unsloth-zoo get_peft_regex change. * vision: address review feedback on finetune_audio_layers threading - Drop finetune_audio_layers from the explicit-target_modules filter condition: it defaults False, so including it forced every explicit list through get_peft_regex and printed the scope warning even for text / vision models that never opted in. - Guard the kwarg by get_peft_regex's signature so an older unsloth_zoo without the parameter does not raise TypeError; warn once if requested but unsupported. * vision: honor finetune_audio_layers with explicit targets; fail fast on old zoo - Route an explicit target_modules list through get_peft_regex when audio is opted in (positive `or finetune_audio_layers`), so finetune_audio_layers=True is no longer a no-op for explicit lists. The scope warning is printed only when a layer family is actually disabled, so default text/vision calls are not spammed. - Raise (instead of warn) when finetune_audio_layers=True but the installed unsloth_zoo lacks the parameter, rather than silently training a language-only adapter. * vision: keyword-safe audio flag, graceful all-linear on old zoo - Move finetune_audio_layers to the end of get_peft_model's signature so positional callers of finetune_last_n_layers (and the params after it) keep their slots. - Track whether the caller EXPLICITLY requested audio before all-linear flips the flag on, and only raise the 'needs newer unsloth_zoo' error for explicit requests. A plain all-linear (text/vision) run now degrades gracefully on an old unsloth_zoo instead of failing.
…6522) * Fix CPOTrainer crash with multimodal processors CPOTrainer shares build_tokenized_answer/tokenize_row and __init__ with ORPOTrainer, but the ORPO replacement functions that route tokenization through the underlying text tokenizer and resolve pad_token_id were only registered for orpo_trainer. With a multimodal processing class (e.g. Gemma4Processor) the positional self.processing_class(prompt, ...) call binds prompt to images=, leaving text=None and raising TypeError: 'NoneType' object is not subscriptable. Register the existing orpo_trainer_text_tokenizer and orpo_trainer_processor_pad_token under cpo_trainer as well so CPO/SimPO fine-tuning of multimodal models works. No change for plain tokenizers. * Add CPO processor tokenizer regression test Static, CPU-only checks that cpo_trainer registers the same orpo_trainer_text_tokenizer and orpo_trainer_processor_pad_token rewriters as orpo_trainer, and that the rewriter drops the broken positional self.processing_class(prompt, ...) call. Guards against issue unslothai#4952 regressing. * Format CPO test assert for ruff line length (pre-commit) * Bind CPO __init__ pad/eos token reads to underlying tokenizer TRL 0.28+ CPOTrainer.__init__ reads bare processing_class.pad_token and processing_class.eos_token before pad_token_id, which raises AttributeError for multimodal processors (e.g. Gemma) where those live on .tokenizer. Extend orpo_trainer_processor_pad_token to route that block through the underlying tokenizer, and add a regression test. * Tighten code comments (no logic change) * Make CPO/ORPO rewriters reach the trainer on TRL 1.x TRL 1.x moved CPOTrainer and ORPOTrainer out of trl.trainer into trl.experimental.<algo> and dropped the trl.trainer.<algo>_trainer shim that older TRL (0.26 - 0.28) kept. patch_trl_rl_trainers() discovers trainers via dir(trl.trainer), so on TRL 1.x cpo_trainer and orpo_trainer are never found and the multimodal-processor tokenization fix (unslothai#4952) silently stops applying, even though the rewriters themselves still match the source. Re-expose experimental-only trainers that Unsloth has rewriters for (RL_FUNCTIONS keys) under trl.trainer before discovery, so the existing patch machinery and its thin-wrapper resolution work unchanged. The alias is a no-op on older TRL where trl.trainer.<algo>_trainer already exists. Also rebind the patched Trainer/Config into every already-imported trl.* module that holds the original class so the fix is visible at the experimental import site (from trl.experimental.cpo import CPOTrainer), not only via trl.trainer. Verified on transformers 4.57.6 + trl 0.22.2, transformers 4.57.6 + trl 0.27.1, and transformers 5.12.1 + trl 1.6.0: CPOTrainer with a multimodal processor tokenizes through the underlying text tokenizer with no crash on all three, and the SFT/GRPO/DPO patch paths are unchanged. * Format for ruff (pre-commit) * Simplify CPO fix to mirror ORPO registrations (unslothai#4952) Register the existing ORPO row-tokenizer/pad-token rewriters for cpo_trainer. Under the trl<=0.24.0 pin CPOTrainer lives in trl.trainer.cpo_trainer (found by dir(trl.trainer)), shares ORPO's build_tokenized_answer and uses processing_class.pad_token_id, so the two registrations are sufficient. Drop the trl 1.x experimental aliasing/rebind machinery in rl.py and the bare-pad_token rewriter: trl 1.x (CPO in trl.experimental) and the bare pad_token pattern (trl>=0.28) are not installable under the pin. * CPO: route bare pad_token/eos_token default through inner tokenizer TRL 1.x CPO/ORPO __init__ (the trl.experimental source unsloth resolves on TRL 0.26+) defaults processing_class.pad_token from processing_class.eos_token before tokenizing. Multimodal processors (Gemma3/Gemma4 Processor) expose those attributes on .tokenizer, not on the processor, so that bare access raises AttributeError during __init__ even with the pad_token_id fallback registered. Extend orpo_trainer_processor_pad_token to rewrite that defaulting block to run on the inner tokenizer. The pinned TRL range (<=0.24.0) has no such block, so the regex is a no-op there and only the existing pad_token_id fallback applies. Verified the rewrite against the real trl 1.6.0 experimental CPOTrainer.__init__ (bare access removed, result compiles, a processor without pad_token no longer raises) and added offline regression tests for both the rewrite and its no-op. --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
…ad_token (unslothai#6524) * Fix Qwen3 NaN: self-heal vision pad_token in load_correct_tokenizer Text-only Qwen3 (and Qwen2.5) models share Qwen3-VL's vocab, so their Hub tokenizer configs ship <|vision_pad|> as pad_token. Padding text-only training with a vision token corrupts attention/loss and produces NaN losses and gradients on affected stacks. patch_tokenizer already heals this, but only when a model with config is passed. The standalone load_correct_tokenizer path (and custom training loops) still returned <|vision_pad|>. This adds a model-independent guard in load_correct_tokenizer that replaces a vision pad_token on text-only tokenizers with the first safe text token (<|endoftext|>, <pad>, [PAD], <unk>), falling back to eos_token only if it differs from pad_token. The result now matches upstream Qwen configs (pad_token <|endoftext|>, id 151643) with no new token added. Vision processors (image_processor present) and non-vision pad tokens (Llama, Qwen2) are left untouched. Fixes unslothai#3155 * Format pad_token helper for ruff kwarg-spacing hook (pre-commit) * Harden vision pad_token fix: drop unk candidate, guard get_vocab, skip vision eos * Tighten code comments (no logic change) * Delegate pad_token fix to shared unsloth_zoo.pad_token Generalize the narrow _fix_vision_pad_token by delegating to unsloth_zoo's shared fix_pad_token (single source of truth, AGPL-3.0), which scans the reserved-token families instead of only the vision-pad case. A guarded import keeps this working against an older unsloth_zoo that has not shipped the module yet: on ImportError it falls back to _fix_vision_pad_token. allow_add=False is passed so the early load_correct_tokenizer call stays side-effect free (no model here to resize embeddings); the later model-aware patch_tokenizer call finishes the job and is idempotent. Adds tests/python/test_pad_token_fix.py covering both dispatch paths offline.
…lothai#6599) * Harden MLX self-heal install against supply-chain execution The Apple Silicon MLX self-heal runs uv pip install on a daemon thread during Studio startup, default-on with only an env opt-out, before the post-install stack check. Two things widened the supply-chain surface: - it accepted source distributions, whose PEP 517 build backends run arbitrary code at install time; and - it forwarded the full process environment, exposing Studio secrets to that code and letting a poisoned env (UV_FIND_LINKS / UV_DEFAULT_INDEX) repoint the install at a hostile source. Require pre-built wheels (--only-binary=:all:) and forward only an allowlist of variables uv needs (PATH/HOME, proxy + CA settings, cache dir), setting UV_OVERRIDE ourselves. mlx/mlx-metal ship wheels only and mlx-lm/mlx-vlm publish py3-none-any wheels, so a healthy self-heal is unaffected; an unavailable wheel just leaves Studio chat-only as before. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop cache-dir env vars from the self-heal allowlist Address review: a poisoned process env could set UV_CACHE_DIR / XDG_CACHE_HOME to redirect uv at an attacker-staged cache (cache poisoning, symlink writes), which partly undercut the index-redirect protection. Drop them from the allowlist; uv falls back to its safe user-owned default cache, still reused across runs, so there is no normal-path cost. Test now asserts both are excluded from the install env. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: drop OpenEnv and unused ExecuTorch/open_spiel install deps * Studio: drop 8 more unused install deps from extras * Studio: restore tomli<3.11 for kernels; tidy dep-cleanup comments and tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: refresh scan-packages baseline for scipy _external + unsloth-zoo tests scipy moved its vendored array_api_compat from scipy/_lib to scipy/_external, so the four allowlisted array_api_compat __init__.py entries stopped matching and resurfaced as unsuppressed CRITICAL "Downloads and executes remote code" findings on all three pip scan-packages shards (extras, hf-stack, studio). Add the _external paths next to the existing _lib ones so both scipy layouts stay covered. Allowlist two unsloth-zoo test-file false positives now present in the hf-stack shard: tests/test_mlx_save_export_regressions.py (writes to /tmp dropper) and tests/test_mlx_trainer_internals.py (obfuscation plus exec/eval). Drop nine stale entries for packages removed from the Studio requirements and no longer in any shard closure (evaluate, pytest, hypothesis, kgb, langid), confirmed absent via with-deps resolution of all three shards. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Bumps the npm-frontend-security group with 1 update in the /studio/frontend directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite). Updates `vite` from 8.0.10 to 8.0.16 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite) --- updated-dependencies: - dependency-name: vite dependency-version: 8.0.16 dependency-type: direct:development dependency-group: npm-frontend-security ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…othai#6595) * Fix SyntheticDataKit.chunk_data dropping single-chunk documents chunk_data turns the n boundary points from np.linspace into n-1 ranges via the boundaries[:-1] / [1:] pairing. When a document fits in a single chunk (n_chunks == 1) that produces zero ranges, so the loop writes no files and the whole document is silently dropped. Emit the full [0, length] range when n_chunks <= 1; the multi-chunk path is unchanged. Added a regression test covering the single-chunk and multi-chunk cases. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chunk_data: emit nothing for an empty document (no empty chunk file) Addresses review feedback: when the input document is empty (length == 0), return no chunks instead of writing a single empty chunk file. Added a regression test for the empty-document case. * chunk_data: reject overlap >= chunk size (non-positive stride) Per review feedback: when overlap >= max_tokens the chunk stride is non-positive, which would divide by zero or silently emit one oversized chunk. Raise a clear RuntimeError for that unusable configuration. Added a regression test. * Broaden single-chunk guard to length <= max_tokens (also fixes sub-overlap docs); expand tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
…e chat template (unslothai#6531) * Fix construct_chat_template leaking {INPUT}/{OUTPUT} sentinel into the template In construct_chat_template's inner process() helper, the branch handling a section that starts with the {INPUT}/{OUTPUT} sentinel sliced the part from part.find(which) (which is 0 in that branch), so the literal sentinel was re-included in the generated Jinja chat template. The endswith branch already slices correctly with part[:part.find(which)]; this slices past the sentinel with part[len(which):], so a template whose input or output section begins with the sentinel (for example a user turn that starts with {INPUT}) renders correctly instead of emitting a literal {INPUT}/{OUTPUT}. Added a regression test covering {INPUT}-leading and {OUTPUT}-leading sections. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: slim the GLM-5.2 thinking menu width The high|max effort menu has short labels, so drop it from min-w-44 to min-w-40 for GLM-5.2 only. Other models keep the wider dropdown. * Studio: keep Preserve thinking menus at full width Only narrow the effort menu when it has no Preserve thinking row. That row is longer than the high|max labels and wraps to two lines at the min-w-40 floor, so gate the skinnier width on no preserve-thinking.
* Studio: confirm saved Hugging Face token with a tick Pasting a token and clicking away saved it silently with no feedback. Show a green tick in the field once a non-empty token is committed and the input still matches the stored value, so the save is visible. Adds the tokenSaved label to the en and zh-CN locales. * Studio: let clicks pass through the saved-token tick The decorative tick sat over the input and swallowed clicks, so clicking it would not focus the field. Add pointer-events-none so clicks reach the input; drop the now-unreachable title and keep an aria-label via role=img.
…slothai#6627) * Fix _SameTaskStreamingResponse disconnect test bypassing __init__ test_same_task_response_closes_body_iterator_on_send_disconnect builds the response via __new__ to skip Starlette's __init__, then wires body_iterator, background, and stream_response by hand. It never set _unstarted_cleanup, so the disconnect-before-first-chunk branch of __call__ raised AttributeError instead of ClientDisconnect, failing the Backend CI "Repo tests (CPU)" job on main. Set response._unstarted_cleanup = None in the manual construction, matching the default __init__ assigns. * Shorten the _unstarted_cleanup comment to one line
…es it (unslothai#7023) * Silence torch._check_is_size FutureWarning and shim it if torch removes it bitsandbytes 4-bit dequant calls torch._check_is_size, which torch deprecated with a FutureWarning ("Use _check(i >= 0) instead") that prints on every bnb-4bit load. Silence that warning in suppress_cuda_printf, and add fix_torch_check_is_size so a future torch that removes _check_is_size gets it shimmed to _check(i >= 0) (honoring the max bound) and bitsandbytes keeps working. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten fix_torch_check_is_size docstring Lead with what the shim does and drop the redundant line; two lines instead of three, same intent. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…thai#7014) * unsloth start: add --resume to persist and reopen agent sessions `unsloth start <agent>` launches a coding agent whose home is a throwaway temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their whole home there) cannot resume a conversation after you quit. opencode and claude keep their session data in a fixed user dir, so they already resume. Add an opt-in --resume/--no-resume flag: it routes the launch to the stable Unsloth agents dir (the same one --no-launch already uses) so the session survives the exit, never touching the user's own ~/.<agent>. A bare --resume also reopens the last conversation via the agent's native flag (codex `resume --last`, opencode/claude/pi `--continue`). The default is unchanged: a plain launch still uses a temp dir and persists nothing. Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the real launch path and asserts the split: codex/pi are wiped without --resume and persist with it, while opencode/claude persist either way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * unsloth start: rename --resume to --persist The session flag collided with agents' own resume flags. `unsloth start claude --resume <id>` used to forward `--resume <id>` straight to Claude (which keeps its history in ~/.claude regardless), so a boolean --resume on unsloth start would have swallowed the session id and turned it into a stray prompt. Name the persistence flag --persist instead, so every agent's native resume flag (claude --resume <id>, codex resume, opencode --continue, ...) still passes through untouched. Behavior is otherwise identical: --persist keeps a launched agent's session under the Unsloth agents dir, and a bare --persist reopens the last conversation. Add a regression test that `--resume <id>` passes through verbatim, and in the CI resume experiment skip the redundant second pass for opencode/claude (they persist either way, and a second CPU turn only risks a timeout). * unsloth start: correct --persist help and drop the buggy auto-resume Reword the --persist help to be accurate: claude and opencode keep sessions in the user's own stores and resume regardless, so --persist only stabilizes the otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the bare-launch auto-append of native resume tokens: it errored on a first launch with no prior session, and was inconsistent between launch and no-launch. --persist now only keeps the session dir; resume via the agent's own command (e.g. `unsloth start codex --persist resume`), which now finds it. In the CI resume experiment, fail the pass when the launched turn exits non-zero, so a write-then-error is not misread as PERSISTED. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…othai#7017) * Studio /v1/messages: accept thinking and unknown content blocks The Anthropic-compatible /v1/messages endpoint modeled a message's content as Union[str, list[{text|image|tool_use|tool_result}]], so any other block type made Pydantic reject the whole request with `messages.N.content.str: Input should be a valid string`. Resuming a Claude session commonly replays assistant turns that carry `thinking` (extended thinking) blocks, and sometimes a null content for a tool-only turn, both of which tripped this and returned a 400. Accept them: - Add a permissive AnthropicUnknownBlock fallback (any block whose type is not one of the four known ones), so thinking/redacted_thinking/provider-specific/ future blocks validate. A validator keeps known types on their typed models, so a malformed known block (e.g. a tool_use without id) still fails cleanly. - Coerce a null message (and tool_result) content to "" so the converter's `for block in content` stays safe. The converter already drops block types it does not translate, so a thinking block is not forwarded to the model. * Studio /v1/messages: keep user content validation strict Make the thinking/null leniency role-aware so it never silently drops real user input. Assistant turns (replayed history) still accept unknown/thinking blocks and coerce a null tool-only turn to empty. User turns keep the strict boundary: a null user content is rejected, and a content block the converter cannot translate is rejected instead of being dropped into an empty prompt. Also remove an empty file committed by accident. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio /v1/messages: coalesce resumed user turns and tighten content checks - The /v1/messages count and generation paths now coalesce the adjacent user turns that dropping an empty or null assistant turn can leave behind, so a strict GGUF chat template no longer 400s on non-alternating roles. - A user content block with a non-string type (list / dict) is rejected as a clean 400 instead of raising TypeError and escaping as a 500. - The assistant null-to-empty coercion only applies to an explicit null; an assistant turn that omits content entirely still fails required-field validation instead of being silently coerced to an empty string. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio /v1/messages: tighten comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Vulkan llama.cpp support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address gemini's feedback * Studio: move the Vulkan VRAM probe into a standalone script * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Improve Vulkan probe error reporting * Resolve llama-server symlink so Vulkan build is detected * Drop unreachable Vulkan fallback in GPU free-memory dispatcher * Skip the Intel GPU probe when NVIDIA or ROCm is present * Reserve host RAM headroom for Vulkan integrated GPUs * Add a `UNSLOTH_FORCE_VULKAN` environment variable * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honor GGML_VK_VISIBLE_DEVICES, reserve discrete Vulkan VRAM headroom, and clear Intel GPU on --cpu-fallback * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Route Intel and forced-Vulkan hosts to the upstream Vulkan prebuilt, add arm64 Vulkan, keep Vulkan out of RAG auto-detect * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Clear the fork release pin when routing a Vulkan host to the upstream repo * Gate auto-Vulkan routing on no physical NVIDIA so hidden CUDA devices aren't used * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin Vulkan launches with --device Vulkan<i> instead of the raw GGML_VK_VISIBLE_DEVICES index space * Let user --device override the Vulkan pin, and gate direct Vulkan asset picks on no physical NVIDIA * Update RAG auto-backend test mocks for the _resolve_auto binary and Vulkan probes * Keep the add_dll_directory handle alive through the Vulkan probe DLL loads * Revert RAG auto Vulkan guard, guard multi-backend Vulkan detection, and preserve forced Vulkan across updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use getattr for RTLD_GLOBAL in the Vulkan probe CDLL mode * Skip CUDA/ROCm APU and datacenter GPU tuning on Vulkan builds On a Vulkan llama.cpp build gpu_indices are ggml compact ordinals, not CUDA/ROCm physical ids, so _amd_apu_wants_unified_memory and _apply_datacenter_env were reading the wrong device. On a mixed AMD APU plus discrete GPU host that could raise a spurious system-RAM shortfall and block a valid discrete-GPU load. Gate all three call sites on not is_vulkan_backend; the Vulkan path already reserves iGPU host headroom and the backend ignores GGML_CUDA_* anyway. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten Vulkan-guard comment in load_model * Reduce comments in Vulkan support to be more succinct * Resolve shell-wrapper llama-server entrypoint to the real lib dir create_exec_entrypoint falls back to a #!/bin/sh wrapper at the install root when it cannot symlink into build/bin. _find_llama_server_binary returns that root entrypoint, but Path.resolve() does not follow a shell wrapper, so _llama_lib_dir returned the install root and _is_vulkan_backend missed libggml-vulkan.so -- silently skipping the Vulkan probe and --device pin on an otherwise valid Vulkan install. Follow the wrapper's exec target to build/bin. Regression test: test_shell_wrapper_entrypoint_resolves_to_real_lib_dir. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Fix Windows installer torch index override * Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (unslothai#6898) * Harden setup.ps1 index-var clearing to truly remove vars (unslothai#6898) * Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (unslothai#6898) * Neutralize all uv index env vars for pinned torch installs (unslothai#6898) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…7025) * Fix fast_gemv crash on compressed-tensors FP8 models Loading a compressed-tensors FP8 checkpoint (for example unsloth/Llama-3.2-1B-Instruct-FP8-Block) with fast_inference=False and running a forward crashed with 'Parameter object has no attribute absmax' inside fast_gemv. A compressed-tensors CompressedLinear exposes an already dequantized bf16 weight at forward time while keeping a weight_scale Parameter. The quant state resolution in get_lora_parameters/get_lora_parameters_bias fell back to that weight_scale, so a bf16 weight was routed into the bitsandbytes fast_gemv/fast_dequantize path, which expects a bitsandbytes QuantState with an absmax attribute. Only fall back to weight_scale_inv/weight_scale when the weight is still fp8. A decompressed bf16 weight then resolves to no quant state and flows through the normal bf16 path, which already handles bias and the LoRA backward. Real fp8 and bitsandbytes 4bit weights are unchanged. * Skip the fast_gemv dispatch test before importing unsloth when bitsandbytes is absent
… linear (unslothai#7028) * Keep native RoPE scaling when extending context; carry rope_theta for linear When max_seq_length exceeds a model's native window, the loader overwrote the model's rope_scaling with linear scaling. For models that already ship a scaled RoPE (llama3/yarn/longrope) that is far worse for long context, and on transformers v5 the linear dict omitted rope_theta (v5 keeps it under rope_parameters), so the rotary base fell back to 10000 and broke past ~8K tokens. Keep the native scaling and just widen the window; only synthesize linear for plain-RoPE models, and carry rope_theta so v5 keeps the real base. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only preserve native llama3 when extending context; keep linear fallback otherwise The patched attention constructor (patch_llama_rope_scaling) rebuilds only linear, llama3 and longrope and its longrope branch reads a top-level original_max_position_embeddings, so preserving yarn or a nested-only longrope config would raise during construction on transformers <= 4.47.1. Keep only llama3 native; yarn/longrope/other types fall back to the linear override, still carrying rope_theta. * Correct long-context extension comment to match llama3-only preservation --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* scripts: refresh scan_packages allowlist baseline Regenerate scripts/scan_packages_baseline.json against the current resolved dependency set so the blocking pip scan-packages gate matches what the scanner now finds. Refreshes evidence hashes for benign findings whose code shifted lines (unsloth-zoo mlx loader, gguf/mlx test /tmp fixtures) and adds two mainstream-library entries that were newly surfaced (torch inductor codecache base64+subprocess compile cache, torch testing common_utils socket import). Stale entries whose matching code changed and no longer triggers are dropped. All entries remain CRITICAL/HIGH findings manually judged benign; matched on (package, file, check, evidence_hash). * ci(security-audit): re-run scan when the allowlist baseline changes The security-audit pull_request trigger listed the scanners but not their allowlist baselines, so a baseline-only edit never re-ran the scan that consumes it. A refreshed baseline could therefore merge without CI confirming its evidence hashes match what the scanner finds. Add scan_packages_baseline.json and scan_npm_packages_baseline.json to the paths filter so baseline changes are validated on their own PR.
parse_direct_linux_release_bundle and direct_linux_release_plan are no longer reached by any live code path. Fork Linux installs resolve through _fork_manifest_release_plans -> _linux_published_attempts, and the upstream (ggml-org) path uses direct_upstream_release_plan. The dead parser also called _resolve_linux_bundle_profile, which no longer exists, so its CUDA branch would raise NameError if ever executed. Drop both functions and the obsolete TestDirectLinuxNvidiaCpuGate; its live equivalent TestLinuxPublishedAttemptsNvidiaCpuGate already covers the NVIDIA no-silent-CPU behaviour.
* Restore dropped FP8 weight_scale_inv tensors on load Some block-scale FP8 checkpoints (for example Qwen3.6-27B-FP8, issue unslothai#6200) load with transformers leaving an mlp.gate_proj as a plain bf16 Linear instead of an fp8 module. Its raw quantized values are read into the bf16 weight and the weight_scale_inv is dropped as an unexpected key, so the weight is used un-scaled and the base model is garbage (perplexity around 2 million). After load, for every checkpoint weight_scale_inv whose live weight is not fp8, dequantize the orphaned weight in place using the block scale from the checkpoint index. Modules that were converted correctly keep an fp8 weight and are skipped, so healthy checkpoints and single-file checkpoints are a no-op. Verified on Qwen3.6-27B-FP8: 64 gate_proj scales restored, perplexity 2028902 to 8.9. No-op on Qwen3-8B-FP8 (all scales already live). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden FP8 weight_scale_inv restore from review - Skip restore when the model has no fp8 weights, so an intentionally dequantized load (load_in_16bit) is never re-scaled and corrupted. - Thread revision, subfolder and cache_dir through the index and shard downloads so scales come from the same snapshot as the weights. - Cover unsharded single-file model.safetensors checkpoints (no index). - Handle transposed block-scale layouts and skip on a true grid mismatch instead of applying a wrong scale. - Match text-only VLM loads where the language_model prefix was stripped. - Restore on the FastLanguageModel text path too, not only vision. - Handle a scalar weight_block_size; per-tensor error handling so one bad tensor cannot abort the rest or hide a partial mutation. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address second review round on FP8 scale restore - Bound peak memory: dequantize block views in place with the fp32 scale broadcast instead of materializing a full expanded scale and fp32 copy, so a near-VRAM-limit load is not pushed into OOM by the repair. - Restore on the sequence-classification load path too. - Cover more VLM key remappings (language_model.model.* to model.language_model.*) when matching modules. - Skip the restore for variant loads (variant=...) rather than risk applying default-checkpoint scales to variant weights. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align FP8 scale restore revision with the loaded weights and warn on disk-offloaded layers In llama.py the CausalLM/SequenceClassification weight loads resolve model_name on its default branch (revision is not forwarded there), so read the dropped weight_scale_inv tensors from the same default branch instead of the requested revision, avoiding rescaling default-branch weights with scales from another revision. In loader_utils.py a disk-offloaded layer keeps its weight on the meta device until the offload hook materializes it, so the scale cannot be applied in place. Skip such layers explicitly and print a warning rather than silently leaving them unscaled. * Tighten comments in the FP8 scale restore path --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…nslothai#7031) * Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename The Gemma 4 QAT GGUF repos renamed the higher-precision MTP/ subdir copies from gemma-4-...-<quant>-MTP.gguf to mtp-gemma-4-...-<quant>.gguf, so their basenames now start with the same mtp- prefix as the small repo-root drafter (mtp-gemma-4-E4B-it.gguf). The drafter selectors filtered candidates by a mtp- basename prefix and took the first in sort order. With the new names the MTP/ copies also match, and because MTP/ (uppercase) sorts before the lowercase root file, selection flipped to the large BF16 copy under MTP/ instead of the root drafter both functions document they should pick. Restrict both selectors, and the companion byte estimate, to root-level mtp-*.gguf so the MTP/ copies stay explicit-selection only: - core/inference/llama_cpp.py _pick_mtp (loader auto-download) - hub/utils/gguf_plan.py preferred_mtp_sibling (Hub variant plans) - routes/inference.py _remote_gguf_companion_bytes (VRAM headroom) Also reuse a drafter already in the local cache before downloading, so a device that already holds a copy on disk does not re-fetch it. Old-scheme names keep working (they have no root-level mtp- sibling to mis-select). Adds regression tests for the new naming, both selection paths, and the on-disk reuse. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate MTP drafter cache reuse to offline mode Reuse the cached drafter only when HF is offline. Online, route back through _download_companion_gguf/hf_hub_download so the current revision is checked (etag) and a changed drafter is refetched, matching the offline-only cross-snapshot reuse already used for the main GGUF. This avoids pairing freshly downloaded weights with a stale cached draft. Make the reuse tests offline and add an online-skips-reuse test. * Studio: prefer a root MTP drafter across all cached snapshots Offline reuse scanned snapshots one at a time and returned the first snapshot that held any drafter, only preferring root within it. A newer partial snapshot with just the MTP/ copy could shadow the small root drafter in an older snapshot. Collect drafters across all snapshots and prefer any repo-root file before an MTP/ copy. * Studio: keep newest-first snapshot order when reusing cached drafters Collecting root candidates and sorting by absolute snapshot path could pick a drafter from an older snapshot. _iter_hf_cache_snapshots yields newest first and the main GGUF is resolved in that order, so preserve it (root still preferred over MTP/ copies) to avoid pairing a fresh main weight with a stale drafter revision. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
--------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: auto-load last used local model * Studio: handle missing GGUF quant in last-used autoload * Studio: tighten last-used autoload handling * Fix * Honor last-used autoload settings * Skip recording LoRA auto-loads * Mirror auto-load runtime state --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>
…nslothai#7007) * Clarify in README that -H 0.0.0.0 starts a public Cloudflare tunnel * Hedge tunnel URL wording and restore trusted-network caution * Tighten the 0.0.0.0 tunnel note * Drop trust-the-network caution from tunnel note * Restore trusted-network note on the raw-bind sentence * Use Cloudflare's quick tunnel terminology and consolidate the trust warning
) * CI: retry transient HTTP timeouts in Studio smoke probes The post() helper in the Studio inference smoke workflows does a single urlopen with a 240s timeout against the local Studio server. On shared runners this sporadically hits TimeoutError while the server is stalled, failing the whole job for a transport hiccup; the same flake has recurred across unrelated PRs on Linux and Windows (JSON/images and tool-calling jobs) and passes on rerun. Retry the probe up to 3 times on transport-level failures only (TimeoutError, ConnectionError, non-HTTP URLError), 15s apart. HTTP status errors still surface immediately, so genuine server failures are unaffected. post_sse() is left unchanged: it has a 600s budget and has not flaked. * CI: retry only short probes so worst case fits the job budget Some json-images calls pass timeout=600; three attempts there could spend 30 minutes in one step and hit the job's timeout-minutes instead of failing with the Python error. Retry (3 attempts) only when timeout <= 300s, which covers the observed flaky 180-240s probes; longer probes keep the pre-PR single attempt. * CI: give long smoke probes one capped retry Round two of bounding the retries: timeout>300s probes previously got a single attempt, so a transient stall in the 600s JSON-mode probes still failed on first occurrence. Give them one retry with the attempt timeout capped at 300s. Worst cases stay inside timeout-minutes: 240s probes 12.5 min, one 600s probe 15.25 min, the Windows JSON job's two long probes 30.5 min against its 35 minute budget.
…er (unslothai#7047) --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR.
The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer.
Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery.
Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior.
The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere.
Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| function writeStoredLocale(locale: Locale): void { | ||
| try { | ||
| globalThis.localStorage?.setItem(LOCALE_STORAGE_KEY, locale); |
| yield "data: [DONE]\n\n" | ||
|
|
||
| return StreamingResponse( | ||
| gen(), |
| pytest.skip( | ||
| f"Server failed to start within 30 seconds. Output:\n{server_output}" | ||
| ) | ||
| server_output = stdout.decode(errors = "replace") + stderr.decode(errors = "replace") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disposable CI run for unslothai#7074 (Studio Voice settings tab). Do not merge; closed after CI.