Skip to content

desktop: disable the webkitgtk dmabuf renderer on nvidia, not only on wayland - #8884

Merged
danielhanchen merged 14 commits into
unslothai:mainfrom
mahiatlinux:fix/studio-nvidia-dmabuf-renderer
Aug 24, 2026
Merged

danielhanchen merged 14 commits into
unslothai:mainfrom
mahiatlinux:fix/studio-nvidia-dmabuf-renderer

Conversation

@mahiatlinux

Copy link
Copy Markdown
Collaborator

Two Ubuntu users report Unsloth Desktop freezing and crashing. Their hosts share
nothing on the GPU side, one an RTX 4050 laptop with 6 GB and one a 96 GB card,
so memory pressure does not explain the second. Both run NVIDIA.

configure_wayland_renderer applied its workaround only on Wayland, and
x11_session_keeps_webkit_defaults asserted that an X11 session kept WebKit's
defaults. WebKit's renderer selection has no session branch:
AcceleratedBackingStore::rendererBufferTransportMode reads
WEBKIT_DISABLE_DMABUF_RENDERER and WEBKIT_DMABUF_RENDERER_FORCE_SHM and
picks the hardware transport on X11 the same way, so an Ubuntu X11 session on
the proprietary driver got no workaround at all.

WebKit's own fix for this (bug 262607)
adds isNVIDIA() and drops every buffer transport for the proprietary driver
regardless of display server. Distributions carry it as a patch, so it cannot be
assumed present in the host library.

configure_linux_renderer now probes /proc/driver/nvidia/version before GTK
initialization and applies the workaround on either display server. The probe
reads a file rather than an EGL vendor string, so it touches no GL state before
GTK comes up. nouveau publishes nothing there and renders through Mesa, so it
stays on the defaults.

On NVIDIA the variable is WEBKIT_DISABLE_DMABUF_RENDERER, matching what
upstream produces: isNVIDIA() returns before
mode.add(RendererBufferTransportMode::SharedMemory), so the transport set is
empty rather than shared-memory. Operator overrides still win, and non-NVIDIA
Wayland keeps WEBKIT_DMABUF_RENDERER_FORCE_SHM.

configure_linux_renderer returns the reason alongside the variable, and the
startup log now records which of the two triggered.

Testing

Four new cases in linux_webkit.rs: NVIDIA on X11, NVIDIA under an explicit
GDK_BACKEND=x11, NVIDIA on Wayland taking the stronger switch, and an operator
override outranking the NVIDIA default.

… wayland

WebKit's renderer selection has no session branch:
AcceleratedBackingStore::rendererBufferTransportMode reads
WEBKIT_DISABLE_DMABUF_RENDERER and WEBKIT_DMABUF_RENDERER_FORCE_SHM and picks
the hardware transport on X11 the same way it does on Wayland, so an X11 session
on the proprietary driver got no workaround.

configure_linux_renderer now probes /proc/driver/nvidia/version before GTK
initialization and applies the workaround on either display server, matching
WebKit's own isNVIDIA fix for bug 262607. nouveau publishes nothing there and
stays on the defaults.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 15a185957b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +97 to +98
fn nvidia_driver_loaded() -> bool {
std::path::Path::new(NVIDIA_DRIVER_VERSION_PATH).exists()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Detect the GPU actually used for rendering

On PRIME/Optimus hybrid systems, /proc/driver/nvidia/version exists whenever the NVIDIA kernel module is loaded even when GTK/WebKit is rendering on the integrated Intel or AMD GPU. In that common laptop configuration this probe falsely selects WEBKIT_DISABLE_DMABUF_RENDERER, disabling WebKit's DMABUF renderer on X11 (and replacing the less disruptive shared-memory workaround on Wayland) despite the affected NVIDIA renderer not being in use; base the workaround on the selected rendering GPU rather than system-wide module presence.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The premise is right: /proc/driver/nvidia/version reports module presence, not the selected renderer, and on most Optimus laptops the internal panel is driven by the iGPU. The consequence is a slower buffer transport rather than a defect, though. WebKit falls back to its legacy path and rendering stays correct, which is the same fallback the Tauri docs recommend for this driver.

Basing it on the selected rendering GPU is not available at this call site. configure_linux_renderer runs before GTK initialization, and resolving the active renderer needs an EGL/GL context, which is exactly what WebKit's isNVIDIA() does inside rendererBufferTransportMode(). WebKit can afford that because the function is called lazily once GTK is already up. Creating a GL context here would defeat the ordering the workaround depends on.

The two failure directions are not symmetric. Over-triggering costs the accelerated transport on a hybrid host. Under-triggering brings back the compositor freeze and the session kill this PR is fixing, on the same laptops, since the reported host is itself a hybrid RTX 4050.

An affected user already has an exit: the operator override is tested before the NVIDIA branch, so WEBKIT_DISABLE_DMABUF_RENDERER=0 preserves the environment and WebKit keeps the DMABUF path (its own check treats "0" as off). Covered by an_operator_override_outranks_the_nvidia_default.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and the code now says so. One correction to the reply above, because it changes the weight of the trade: on WebKitGTK 2.44 and later the cost of over-triggering is not a slower buffer transport. WEBKIT_DISABLE_DMABUF_RENDERER returns before mode.add(RendererBufferTransportMode::SharedMemory), so the transport set is empty, checkRequirements() is false and accelerated compositing is off for the whole process (#9393). A hybrid host presenting on the integrated GPU loses compositing, not a transport.

The probe stays on module presence, since resolving the rendering GPU needs a GL context and this function runs before GTK is initialized precisely so that no GL state exists yet. 94d2c95 names the PRIME case in the code and points at the WEBKIT_DISABLE_DMABUF_RENDERER=0 opt-out, which on a patched library hands the decision back to the library's own GL_VENDOR probe.

Measured on an NVIDIA X11 host here, running libwebkit2gtk-4.1-0 2.52.3-0ubuntu0.24.04.1, which carries the patch: the GBM and surfaceless EGL contexts both report NVIDIA Corporation, so the two probes agree. That box has two discrete cards and no integrated GPU, so it cannot exercise the disagreement this comment is about.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 15a185957b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f266a9a2bf

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@@ -52,6 +59,11 @@ fn rendering_plan(
return RenderingPlan::PreserveEnvironment;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Override inherited Force-SHM on NVIDIA relaunches

When an NVIDIA/Wayland user updates from the previous build, that running build has already set WEBKIT_DMABUF_RENDERER_FORCE_SHM=1; the inspected update flow in studio/frontend/src/hooks/use-tauri-update.ts lines 416-427 starts the replacement as a child, which inherits this environment. This early return therefore prevents the replacement from setting the stronger WEBKIT_DISABLE_DMABUF_RENDERER workaround that this change requires for NVIDIA, so the first post-update process remains exposed to the freeze/crash until a clean external launch. The NVIDIA branch must distinguish inherited app state from a deliberate operator override or clear/replace the old app-applied variable during relaunch.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fixing. The mechanism is right: tauri 2.11.5 process::restart runs Command::new(path).args(...).spawn() with no env_clear, so the replacement inherits the environment. The scenario still is not fixable from this side. The inherited WEBKIT_DMABUF_RENDERER_FORCE_SHM=1 was set by the previous build, and that build exports no marker saying it applied it, so a marker added here can only tag state written by builds that already carry the marker. The other option, treating a bare FORCE_SHM=1 on NVIDIA as app state instead of an override, breaks the override contract this change tests in an_operator_override_outranks_the_nvidia_default: an operator on NVIDIA who prefers shared memory over an empty transport set has to be able to say so.

The gap is one session, NVIDIA plus Wayland only, and it leaves that session on the pre-update behaviour rather than regressing it. NVIDIA on X11, the case this change exists for, inherits nothing because the previous build applied nothing there, so the workaround takes effect on the first post-update launch. Same-build relaunches are already idempotent, since the inherited variable equals the one the new process would set.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 69781084c1: inherited FORCE_SHM=1 is replaced on NVIDIA relaunches; all 377 Tauri tests pass.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and 6978108 fixed it. 94d2c95 replaces that fix with a simpler one: the NVIDIA branch no longer reads WEBKIT_DMABUF_RENDERER_FORCE_SHM at all, rather than carving out the single value 1.

WebKit reads the variable as forceSHM && g_strcmp0(forceSHM, "0"), so =0 is indistinguishable from unset. The carve-out therefore honoured the one value that states nothing and overrode the one WebKit acts on, which left a stale FORCE_SHM=0 able to suppress the workaround on an unpatched NVIDIA host with nothing in the startup log. Ignoring the variable outright covers the relaunch case this comment raised and that one together.

WEBKIT_DISABLE_DMABUF_RENDERER=0 alongside FORCE_SHM=1 is still how an operator asks for shared memory on NVIDIA, covered by explicit_disable_override_preserves_force_shm_on_nvidia.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 69781084c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member

Went through this one carefully because I was looking at the same file from the performance side. The NVIDIA probe is the right instinct and I am not arguing with the direction. One factual correction though, and it changes which variable you want.

The transport set is not empty on NVIDIA.

The PR body says isNVIDIA() returns before mode.add(RendererBufferTransportMode::SharedMemory), so the set is empty rather than shared-memory. It is the other way around. From Debian's disable-nvidia-dmabuf.patch, the hunk is:

         mode.add(RendererBufferTransportMode::SharedMemory);

+        if (isNVIDIA())
+            return;
+
         const char* forceSHM = getenv("WEBKIT_DMABUF_RENDERER_FORCE_SHM");
         if (forceSHM && g_strcmp0(forceSHM, "0"))
             return;

The early return lands between the SharedMemory add and the FORCE_SHM check, so on NVIDIA the set is {SharedMemory}. That is byte for byte what WEBKIT_DMABUF_RENDERER_FORCE_SHM=1 produces. Upstream's intended behaviour on NVIDIA is FORCE_SHM, not DISABLE_DMABUF.

Why that matters: DISABLE_DMABUF turns accelerated compositing off entirely.

WEBKIT_DISABLE_DMABUF_RENDERER returns before the SharedMemory add, so the set really is empty, and an empty set propagates. At webkitgtk-2.50.6, AcceleratedBackingStore.cpp:140:

bool AcceleratedBackingStore::checkRequirements()
{
    if (!rendererBufferTransportMode().isEmpty())
        return gtkCanUseHardwareAcceleration();
    return false;
}

and HardwareAccelerationManager.cpp:47:

const char* disableCompositing = getenv("WEBKIT_DISABLE_COMPOSITING_MODE");
if ((disableCompositing && strcmp(disableCompositing, "0")) || !AcceleratedBackingStore::checkRequirements())
    m_canUseHardwareAcceleration = false;

So since 2.44, WEBKIT_DISABLE_DMABUF_RENDERER=1 has the same end effect as WEBKIT_DISABLE_COMPOSITING_MODE=1. Michael Catanzaro puts the user visible consequence plainly in bug 303811: "blur (and other 3D transforms) is not expected to work with dmabuf renderer disabled." This PR applies that to every NVIDIA user on both display servers, which is a wider blast radius than the Wayland only rule it replaces. I could not reproduce the blur loss myself, but only because this box has no DRM access and every transport collapses to the same path, so treat that as untested rather than disproven.

Suggestion: use FORCE_SHARED_MEMORY for the NVIDIA branch. It matches upstream's own fix exactly, keeps accelerated compositing on, and still removes the hardware dmabuf transport. If you have a reason to believe shared memory is insufficient on these two users' hosts, that would be worth stating, because it would mean upstream's patch is also insufficient.

Two smaller things from reading the same patch.

The patch ships a documented opt-out, WEBKIT_FORCE_DMABUF_RENDERER, checked first inside isNVIDIA(). Worth preserving as an override alongside the two we already honour.

Upstream detects NVIDIA by creating a throwaway GL context and testing whether GL_VENDOR contains NVIDIA, that is, the GPU actually doing the rendering. That is the difference the review comment about Optimus laptops was pointing at: /proc/driver/nvidia/version reports module presence, so a hybrid laptop rendering on the Intel iGPU reads as NVIDIA and gets the workaround it does not need. Not necessarily worth doing a GL probe before GTK init, but the log line should probably not claim more than "NVIDIA module loaded".

On distro patch coverage, which cuts both ways. I checked two:

  • Arch's webkit2gtk-4.1 2.52.5-2 carries nothing. source=() is the pristine upstream tarball, prepare() is an empty cd, and there are no .patch files in the packaging repo at all. So Arch NVIDIA users genuinely have no auto fallback and need this.
  • Ubuntu 24.04's libwebkit2gtk-4.1-0 2.50.4 does carry it. strings on the shipped library hits WEBKIT_FORCE_DMABUF_RENDERER, which only exists in the Debian patch, and the changelog syncs d/p/disable-nvidia-dmabuf.patch forward. So on the distro both reporters are running, WebKit already selects shared memory on NVIDIA by itself.

That does not invalidate the probe, since we cannot assume the patch, but it does mean that on Debian and Ubuntu this PR's only real change over upstream is the escalation from shared memory to compositing off. Which is the part I would drop.

Last one, and it is a genuine open question rather than a correction. I could not find a single report anywhere of FORCE_SHM=1 fixing the Error 71 (Protocol error) startup crash. Every confirmed fix in 280210 and friends uses WEBKIT_DISABLE_DMABUF_RENDERER=1 or __NV_DISABLE_EXPLICIT_SYNC=1. Mechanically that is not surprising: the surface that trips the explicit sync violation is the application's own toplevel wl_surface, which is not the surface FORCE_SHM switches. Since you can presumably reach the two affected users, it would be worth having them try __NV_DISABLE_EXPLICIT_SYNC=1 on its own, with no WebKit variable set. It is reported to fix the same crash with no rendering cost at all ("smooth without low FPS" in 280210 comment 6), against "performance is very bad" for the WebKit variable. If that holds it is a strictly better lever than either transport switch, and nobody loses GPU compositing.

Whichever variable you land on, could the PR body spell out the failure path? Something like: if the workaround is wrong for a given host the symptom is X, and the user recovers by setting WEBKIT_DISABLE_DMABUF_RENDERER=0 (or the relevant variable) themselves. Rendering transport changes are invisible when they work and total when they do not, so the recovery step should be written down somewhere a user can find it.

@danielhanchen

Copy link
Copy Markdown
Member

One follow up, because it may be decisive for the variable choice and it is cheap to check.

Your description says both crash reporters are on Ubuntu. If that means Ubuntu 24.04 with the archive libwebkit2gtk-4.1-0, then WebKit had already restricted them to shared memory on NVIDIA before Studio set anything, because Ubuntu carries the Debian patch. I verified that on a 24.04 box here: strings on the shipped libwebkit2gtk-4.1.so.0 (2.50.4-0ubuntu0.24.04.1) hits WEBKIT_FORCE_DMABUF_RENDERER, and that symbol appears nowhere in upstream (grep on AcceleratedBackingStore.cpp at tag webkitgtk-2.50.6 gives zero hits), so it only comes from the patch. The changelog agrees: the patch is refreshed at 2.46.6 and synced forward at 2.48.0.

If that holds for their machines, then they crashed while already running SHM-only on NVIDIA, which would mean shared memory does not prevent this failure and FORCE_SHM=1 was never going to help them. That is consistent with the mechanism: the surface that trips the explicit-sync violation is the application's own toplevel wl_surface, which is not the surface either transport switch controls. It would also mean the reason DISABLE_DMABUF=1 works for them is that it disables accelerated compositing outright, not that it changed the buffer transport.

Worth confirming before landing, since it decides between the two variables:

  • apt policy libwebkit2gtk-4.1-0 on both reporters' machines, to confirm the version and that it is the archive build rather than a PPA, Flatpak or Snap
  • whether their symptom is the Error 71 (Protocol error) startup exit or a later freeze, since those may be different bugs
  • __NV_DISABLE_EXPLICIT_SYNC=1 alone, with no WebKit variable set

Related precedent that argues against ever assuming this one is fixed upstream: Debian removed the patch at 2.46.0-1 with the note "all Nvidia-related bugs are supposed to be fixed upstream", then brought it back in the very next revision, 2.46.0-2, "Bring back this patch (Closes: #1082139)". So the "upstream fixed it" hypothesis has been tried once and reverted within a single revision. Not an argument against your PR, just a reason to keep the probe and to be careful about which lever it pulls.

@danielhanchen

Copy link
Copy Markdown
Member

Correcting myself on the part of my first comment that was advice rather than fact. The facts stand; the recommendation I drew from them was probably wrong, and I would rather say so before you act on it.

What stands. The patch ordering is not in question. I have now read the raw hunk three times:

         mode.add(RendererBufferTransportMode::SharedMemory);

+        if (isNVIDIA())
+            return;
+
         const char* forceSHM = getenv("WEBKIT_DMABUF_RENDERER_FORCE_SHM");

Upstream's patch leaves NVIDIA at {SharedMemory}, and DISABLE_DMABUF really does end up disabling accelerated compositing outright. Both of those are still worth fixing in the PR body.

What I got wrong. I told you to switch the NVIDIA branch to FORCE_SHM because it matches that patch. Matching upstream's patch is not the same as fixing the crash, and I conflated them. Three things point the other way:

The root cause looks like it is in GTK3, not in WebKit's transport. GNOME/gtk#8056, "Wayland: Attaching buffer to same EGL surface that window is backed by causes a crash (Error 71 (Protocol error) dispatching to Wayland display)", puts it in gdk_wayland_window_attach_image(): GTK attaches a buffer to a surface that is already backed by EGL, which is illegal and trips the protocol error. Reporter is on Arch, KDE Plasma, RTX 2070 SUPER, reproducible "extremely consistently". There is a proposed patch and no maintainer response yet.

If that analysis is right, then the crash is on the shared-memory attach path. FORCE_SHM routes more frames through exactly that path, so it is not obviously a fix and could plausibly make things worse. WebKit bug 315436 is titled "Explicit sync enabled for wl_shm surface commit causing protocol disconnect on NVIDIA", which is consistent. I want to be clear that this last step is my inference from the two bug reports, not something anyone has measured.

It also fits the field evidence, which I had already noticed but read the wrong way round: I could not find a single report anywhere of FORCE_SHM=1 fixing Error 71. Every confirmed fix uses WEBKIT_DISABLE_DMABUF_RENDERER=1 or __NV_DISABLE_EXPLICIT_SYNC=1. Tauri's own Linux graphics page does not list FORCE_SHM at all.

So your choice of DISABLE_DMABUF is the defensible one for crash prevention, and I withdraw the suggestion to change it. There is also direct prior art for your exact design, which I should have found earlier. Wails ships it in-framework, v3/pkg/application/application_linux.go:

if os.Getenv("WEBKIT_DISABLE_DMABUF_RENDERER") == "" && isNVIDIAGPU() {
	_ = os.Setenv("WEBKIT_DISABLE_DMABUF_RENDERER", "1")
}

func isNVIDIAGPU() bool {
	if _, err := os.Stat("/sys/module/nvidia"); err == nil {
		return true
	}
	return false
}

Same probe, same variable, same "both display servers" decision. It was originally gated on XDG_SESSION_TYPE == "wayland" and the guard was deliberately removed in commit 37a69238675, "fix(linux): apply WEBKIT_DISABLE_DMABUF_RENDERER on X11 for NVIDIA GPUs", closing wailsapp/wails#4985. That is your PR's reasoning arrived at independently, and worth citing in the body.

The one thing I would still change is not the variable but the trigger order. Tauri's Linux graphics doc, added 2026-06-15, ranks the levers:

Set __NV_DISABLE_EXPLICIT_SYNC=1. This often fixes the Wayland Error 71 crash without a performance cost.

Set WEBKIT_DISABLE_DMABUF_RENDERER=1. Fixes the DMABUF framebuffer error and the Error 71 crash, at the cost of the faster rendering path.

and warns:

Only ship an unconditional override like this if you have verified your app is affected. It disables a faster path for everyone, including users on working setups.

Worth trying __NV_DISABLE_EXPLICIT_SYNC=1 first on NVIDIA and only falling back to DISABLE_DMABUF if it does not hold, since the first costs nothing and the second costs compositing for every NVIDIA user. Separately, the perf complaints about DISABLE_DMABUF are real and specific, so the PR body should say plainly that NVIDIA users trade smoothness for stability and how they opt back out. Sorry for the churn on the recommendation.

@danielhanchen

danielhanchen commented Aug 19, 2026 •

Copy link
Copy Markdown
Member

Confirmed this hits studio/src-tauri/src/linux_webkit.rs, where the workaround is still gated on a Wayland session and NVIDIA X11 launches keep the dmabuf transport.

@mahiatlinux mahiatlinux self-assigned this Aug 19, 2026

@mahiatlinux mahiatlinux left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

studio/src-tauri/src/linux_webkit.rs:70: the NVIDIA branch honours the one FORCE_SHM value WebKit ignores, and overrides the one it acts on

WebKit reads the variable as a value, not a presence:

const char* forceSHM = getenv("WEBKIT_DMABUF_RENDERER_FORCE_SHM");
if (forceSHM && g_strcmp0(forceSHM, "0"))
    return;

Same shape in every shipped version: AcceleratedBackingStoreDMABuf.cpp:78 at webkitgtk-2.44.0 and :97 at 2.48.0, AcceleratedBackingStore.cpp:99 at 2.50.0 and 2.52.0. So WEBKIT_DMABUF_RENDERER_FORCE_SHM=0 is indistinguishable from the variable being unset: it forces nothing and suppresses nothing.

Line 70 treats it as an operator override regardless. On an NVIDIA host whose WebKitGTK does not carry the Debian disable-nvidia-dmabuf.patch (Fedora, Arch, openSUSE, Nix, the Flatpak runtime, upstream tarballs; nothing of this is upstream, see below), a stale WEBKIT_DMABUF_RENDERER_FORCE_SHM=0 in environment.d or a launcher gives:

  1. line 72 returns PreserveEnvironment, so nothing is applied and configure_linux_renderer returns None;
  2. WebKit adds SharedMemory, skips the return because the value is "0", and adds Hardware;
  3. the host runs the full hardware dmabuf transport on NVIDIA, the state this change exists to prevent, with no line in the startup log, because nothing was applied.

The polarity is inverted. =1, the only value that states a preference, is overridden as inherited app state; =0, which states nothing, wins. WEBKIT_DISABLE_DMABUF_RENDERER is already tested first at line 57 and is the only variable WebKit accepts as "keep the dmabuf renderer" (=0 short-circuits its own check, AcceleratedBackingStore.cpp:87 at 2.50.0), so the escape hatch documented at line 68 survives without this branch.

Delete lines 70-73. an_operator_override_outranks_the_nvidia_default (:246) then asserts the opposite of the code and should become:

for value in ["0", "1", "", "true"] {
    assert_eq!(
        plan_on_nvidia(&[(FORCE_SHARED_MEMORY, value)]),
        RenderingPlan::Apply(RenderingWorkaround::DisableDmabuf, NVIDIA_REASON)
    );
}

With that edit the module still compiles and all 16 tests pass, explicit_disable_override_preserves_force_shm_on_nvidia included, so the DISABLE_DMABUF=0 hatch is unaffected.

studio/src-tauri/src/linux_webkit.rs:222 and :63: the patch this reasoning rests on is described backwards, and it is not a WebKit fix

// isNVIDIA returns before SharedMemory is added, so FORCE_SHM is not equivalent

The shipped patch does the opposite. Debian webkit2gtk 2.53.90-1, debian/patches/disable-nvidia-dmabuf.patch, whose headers are Bug: https://bugs.webkit.org/show_bug.cgi?id=262607 and Origin: https://github.com/WebKit/WebKit/pull/18614:

         mode.add(RendererBufferTransportMode::SharedMemory);
 
+        if (isNVIDIA())
+            return;
+
         const char* forceSHM = getenv("WEBKIT_DMABUF_RENDERER_FORCE_SHM");

The early return lands between the SharedMemory add and the FORCE_SHM check, so a patched build leaves NVIDIA on {SharedMemory}, exactly what FORCE_SHM=1 produces. WEBKIT_DISABLE_DMABUF_RENDERER=1 returns before the add and leaves {}, a different and stronger state. The same sentence appears in the PR body.

Line 63 calls it "webkit's isNVIDIA fix (bug 262607)". Bug 262607 is RESOLVED WONTFIX, PR 18614 was closed unmerged, and the string NVIDIA appears in neither backing-store file on any release branch from 2.42 to 2.52. It is a distribution patch. The PR body's next sentence already says so; the code comment contradicts it.

studio/src-tauri/src/linux_webkit.rs:140: remove_var(FORCE_SHARED_MEMORY) cannot change what WebKit selects

WEBKIT_DMABUF_RENDERER_FORCE_SHM has exactly one reader in WebKit, and it sits after the WEBKIT_DISABLE_DMABUF_RENDERER check that returns first: lines 66 and 78 at webkitgtk-2.44.0, 85 and 97 at 2.48.0, 87 and 99 at 2.50.0 and 2.52.0. The next statement, line 142, sets WEBKIT_DISABLE_DMABUF_RENDERER=1, so the removal is never observed. On the other path that reaches DisableDmabuf (non-NVIDIA Wayland below 2.44) the variable cannot be set, because line 77 returns first. Nothing else in the repository reads it.

Deleting lines 139-141 leaves all 16 tests green, the only coverage statement available for it: every new test asserts on rendering_plan, and configure_linux_renderer has none.

studio/src-tauri/src/main.rs:1784: the startup log no longer records the display server for the hosts this change targets

The NVIDIA branch returns at linux_webkit.rs:74, before the Wayland/X11 determination at :82-99 ever runs, so an NVIDIA X11 host and an NVIDIA Wayland host emit the identical line:

NVIDIA driver loaded; set WEBKIT_DISABLE_DMABUF_RENDERER=1 for WebKitGTK compatibility

The merge base always recorded Wayland detected. The two triggers are not mutually exclusive, so "the startup log now records which of the two triggered" holds only for the first match. Nothing else carries the display server: studio/src-tauri/src/diagnostics/report.rs:32-46 records app_version, os, arch and the AppImage flag, and tauri.log enters the support bundle as a 1000-line, 200 KiB tail (studio/src-tauri/src/diagnostics/mod.rs:30-31), so the startup line ages out of any long session. Compose both when both hold.

tests/studio/playwright_research_freeze.py:10: stale after the rename

studio/src-tauri/src/linux_webkit.rs forces that webview onto the SHM software renderer on Wayland

Both halves are now wrong on an NVIDIA host: linux_webkit.rs:74 selects WEBKIT_DISABLE_DMABUF_RENDERER, not the shared-memory transport, and it applies on X11 as well. The docstring exists to state what the harness cannot prove about the reporter's machine, so the wrong transport undercuts the caveat it is making.

PR body, "Testing": six new cases, not four

grep -c '#\[test\]' gives 10 at the merge base and 16 at 69781084c. The body lists four and was not updated after 69781084c added nvidia_relaunch_replaces_inherited_force_shm (:230) and explicit_disable_override_preserves_force_shm_on_nvidia (:238). Those two carry the override-contract change, the part a reviewer most needs pointed at.

@oobabooga

Copy link
Copy Markdown
Member

I went back through the transport question, and the correction earlier in this thread is wrong for the versions Studio actually ships against. Your PR body has the ordering right. Sorry for the detour.

The patch ordering depends on the WebKit version

The disable-nvidia-dmabuf.patch that Debian and Ubuntu carry moved the isNVIDIA() early return recently. Read out of the source packages themselves:

package placement NVIDIA result
webkit2gtk 2.50.4-0ubuntu0.22.04.1 (what the AppImage bundles) isNVIDIA() before mode.add empty set
webkit2gtk 2.52.3-0ubuntu0.24.04.1 (Ubuntu 24.04) before empty set
Debian webkit2gtk 2.52.6-1 before empty set
Debian webkit2gtk 2.53.90-1 (development) after {SharedMemory}

From webkit2gtk_2.50.4-0ubuntu0.22.04.1.debian.tar.xz:

+        if (isNVIDIA())
+            return;
+
         mode.add(RendererBufferTransportMode::SharedMemory);

Reproducible with:

curl -sLO https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/webkit2gtk/2.50.4-0ubuntu0.22.04.1/webkit2gtk_2.50.4-0ubuntu0.22.04.1.debian.tar.xz
tar -xf webkit2gtk_2.50.4-0ubuntu0.22.04.1.debian.tar.xz
sed -n '/rendererBufferTr/,$p' debian/patches/disable-nvidia-dmabuf.patch

2.53.90 is a development release on the way to 2.54, so the reordered form is not what Studio currently ships. The v0.1.801-beta AppImage bundles 2.50.4 (read from the bundled library through webkit_get_major_version and friends, alongside GTK 3.24.33 and GLib 2.72.4), which is the Ubuntu 22.04 package above and matches the ubuntu-22.04 builder in release-desktop.yml.

What that changes about the change itself

Three populations, and they behave differently:

1. Patched library, vendor probe returns NVIDIA. The library already returns an empty transport set on either display server, with no help from Studio. Setting WEBKIT_DISABLE_DMABUF_RENDERER there does not alter rendering, so on those hosts this change cannot be the difference between crashing and not crashing. That should normally include AppImage and Ubuntu .deb launches whose WebKit probe successfully creates a context on the NVIDIA renderer. Worth establishing whether either reporter is in that population. If the symptom is specifically the Wayland Error 71 disconnect, GNOME/gtk#8056 is one candidate to compare, since its failure is on the toplevel wl_surface rather than the WebKit transport surface.

2. Patched library, module loaded, vendor probe returns non-NVIDIA. PRIME and Optimus laptops rendering on the integrated GPU. Here the library deliberately keeps the DMA-BUF transport and this change turns compositing off. This is the case the automated review flagged, and it is also the case where the change could genuinely be doing something for the RTX 4050 report. Same mechanism, opposite signs, and which applies depends on whether the discrete GPU is involved in presenting that window. That is the decision this PR needs to make out loud.

3. Distros without the patch. Fedora 44 ships webkit2gtk4.1-2.52.5-1.fc44 from the upstream source without disable-nvidia-dmabuf.patch; its library also has no WEBKIT_FORCE_DMABUF_RENDERER string. There the library can take the hardware transport on NVIDIA and Studio adds no X11 fallback unless an AppImage cannot load GLES. This is where the choice between the two variables actually costs something.

The two variables are not interchangeable, measured

I ran the shipped v0.1.801-beta AppImage under Xvfb in a container with no host GTK or WebKit, and compared the GL objects mapped into WebKitWebProcess:

default vs WEBKIT_DMABUF_RENDERER_FORCE_SHM=1:  identical
default vs WEBKIT_DISABLE_DMABUF_RENDERER=1:    4d3
                                                < libGLESv2.so.2.1.0

DISABLE_DMABUF ends accelerated compositing rather than degrading the transport, which is what #9393 describes from the source side. The missing libGLESv2 supports that source trace. The default-versus-FORCE_SHM equality is narrower evidence: Xvfb was rendering through Mesa without an active hardware DMA-BUF transport, so this run cannot measure the transport or performance difference on an NVIDIA desktop. For population 3 the source-level compositing cost is still the cost to weigh. Debian moved NVIDIA to {SharedMemory} deliberately in 2.53.90-1: its changelog says "Use the SHM transport with NVIDIA GPUs (Closes: #1142771)." This makes the empty set the older Debian and Ubuntu patch behavior and {SharedMemory} the current Debian development-package choice.

It also means the reply to the hybrid-GPU comment, that the consequence is a slower buffer transport rather than a defect, does not hold on 2.44 and later. The objection still needs an answer, just not that one.

What I need before this can merge

  1. Merge the latest main branch. The branch is 206 commits behind and conflicts with Desktop: ship a complete Linux AppImage #9113, which added an AppImage branch and a GLES probe to rendering_plan in the same function. The merged/rebased version has to order the NVIDIA probe against those two as well, and the AppImage case is the one where the bundled WebKit version is known rather than guessed.
  2. Scope the PR body's claim to a version. The empty-set behavior is what the current Debian and Ubuntu patched stable packages do; 2.53.90 changes it. Since the AppImage bundles its own WebKit, a hardcoded DISABLE_DMABUF becomes stronger than the library's own choice if that ordering reaches the 2.54 bundle.
  3. An explicit decision on module presence versus the rendering GPU, with the PRIME case named in a comment either way. If the intent is to catch the laptop report, say so, because that is exactly the configuration where the library's own probe declines on purpose.
  4. Verify the affected hosts. For each reporter, record the exact Studio asset or package, WebKitGTK version, display backend, and failure signature, then confirm whether WEBKIT_DISABLE_DMABUF_RENDERER=1 alone changes it. For a Wayland Error 71 failure, compare __NV_DISABLE_EXPLICIT_SYNC=1 as well. Without that, we cannot tell whether the PR fixes the reported failure or only broadens a workaround for a related one.
  5. State the cost in the code. Nothing in linux_webkit.rs today tells a reader that one of these two switches turns off accelerated compositing while the other does not.

One adjacent note: whichever variable you settle on, the same argument applies to the AppImage-on-Wayland branch that #9113 added, which sets DISABLE_DMABUF for every AppImage user on Wayland regardless of GPU. That is #9393, and if this lands with FORCE_SHM the two should move together.

@oobabooga

Copy link
Copy Markdown
Member

I pushed two commits to this branch rather than only asking for them, because the conflict with #9113 had to be resolved before anything else could be checked. Revert either if you disagree with it.

26ff1df1fa, merge of current main

The conflict was semantic, not textual. #9113 reshaped rendering_plan with an AppImage branch and a GLES probe, and renamed configure_wayland_renderer to configure_renderer. The resolution keeps main's name, since it already carries the meaning this PR's rename was for, and orders the NVIDIA probe ahead of both the FORCE_SHM override and the GLES branch, so an inherited variable cannot suppress it. The PR is mergeable again and CI is running on a current base.

94d2c954ba, the review findings

  • The NVIDIA branch no longer reads WEBKIT_DMABUF_RENDERER_FORCE_SHM at all. Carving out the single value 1 honoured =0, which WebKit treats as unset (forceSHM && g_strcmp0(forceSHM, "0")), so a stale FORCE_SHM=0 in a launcher or environment.d left an unpatched NVIDIA host on the full hardware transport with nothing in the startup log. Ignoring the variable there covers the inherited-=1 case as well, so the carve-out was not needed for it.
  • std::env::remove_var(FORCE_SHARED_MEMORY) is gone. WebKit reads WEBKIT_DISABLE_DMABUF_RENDERER and returns before it ever reads FORCE_SHM, so the removal could not change what the library selects.
  • The reason now carries the display server. An NVIDIA Wayland host and an NVIDIA X11 host used to emit the same startup line, because the NVIDIA branch returned before the session was determined. Main had dropped the display server from that line entirely, and nothing in the support bundle names it otherwise.
  • The comments were rewritten to say that this is a distribution patch and not a WebKit fix, that the empty-set behaviour is version-scoped, that DISABLE_DMABUF ends accelerated compositing rather than degrading a transport, and that the probe is module presence with the PRIME case named. That is requirements 2, 3 and 5.

On the patch itself, read out of the source packages rather than out of a description of them:

package isNVIDIA() placement NVIDIA transport set
Ubuntu webkit2gtk 2.50.4-0ubuntu0.22.04.1 before mode.add(SharedMemory) {}
Debian webkit2gtk 2.52.6-1 before {}
Debian webkit2gtk 2.53.90-1 after {SharedMemory}

isNVIDIA appears nowhere in AcceleratedBackingStore.cpp at webkitgtk-2.50.4 or -2.52.5, so it only ever arrives as a distribution patch. Your PR body has the ordering right for every version Studio ships; it is 2.53.90 that changed it, which is why the code comment now states a version rather than a behaviour.

Verification

19 unit tests pass. Separately I ran the merged rendering_plan against main's across 6720 non-NVIDIA cases, every combination of GDK_BACKEND, WAYLAND_DISPLAY, FORCE_SHM, DISABLE_DMABUF and APPIMAGE values, both GLES states and four WebKitGTK versions, and got zero differences. So nothing outside the NVIDIA branch moved. On NVIDIA the only transitions against main are PreserveEnvironment -> DisableDmabuf in 1476 cases and ForceSharedMemory -> DisableDmabuf in 60; no host loses a workaround it had.

Requirement 4 is still open, and it is the only one that cannot be closed from a machine

It needs the reporters. The three populations behave differently, so which one each reporter is in decides whether this change fixes the failure they hit or broadens a workaround for a related one, and that is a fact about their hosts.

For the record on what a lab can and cannot settle here: the box I used is an NVIDIA X11 host running libwebkit2gtk-4.1-0 2.52.3-0ubuntu0.24.04.1, which does carry the patch (WEBKIT_FORCE_DMABUF_RENDERER is present in the shipped library and appears nowhere upstream). Its GBM and surfaceless EGL contexts both report NVIDIA Corporation, so the module probe and the library's own probe agree, which puts it in population 1 where this variable changes nothing. It has two discrete cards and no integrated GPU, so it cannot exercise the PRIME disagreement, and it has no Wayland session, so the Error 71 disconnect cannot occur on it at all. Neither absence is fixable by testing harder.

So please still collect, per reporter: the exact Studio asset or package, the WebKitGTK version, the display backend, the failure signature, and whether WEBKIT_DISABLE_DMABUF_RENDERER=1 alone changes it. For a Wayland Error 71, __NV_DISABLE_EXPLICIT_SYNC=1 on its own is worth the same test.

On your own review of 08-19

The :70 finding, the :140 finding and the main.rs:1784 finding are all implemented above. The :63 half of the second finding is implemented, since bug 262607 is WONTFIX and PR 18614 was closed unmerged; the :222 half is not, because the ordering it calls backwards is the ordering Ubuntu 2.50.4 and Debian 2.52.6 actually ship. I left tests/studio/playwright_research_freeze.py:10 alone: its docstring is already imprecise on main, since the AppImage branch selects DISABLE_DMABUF on Wayland too, so it belongs with #9393 rather than here.

Two things left that are yours to write: the body still says four new cases, and the file is now at 19 tests, six more than main; and the ordering claim in the body wants the same version scope the code comment now carries.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 94d2c954ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@oobabooga

Copy link
Copy Markdown
Member

CI on 94d2c954ba has settled at 31 pass and 4 fail, and none of the four is this branch. Recording it so the red checks do not read as yours.

Rust unit tests (windows) and Chat UI Tests each failed once and passed on re-run. The first was desktop_backend_owner::tests::stale_blank_observer_cannot_delete_a_competing_callers_id panicking on Access is denied. (os error 5) against a Windows temp path, with the other 392 tests green. mod linux_webkit is #[cfg(target_os = "linux")], so nothing this PR changes is even compiled on that runner. The second was a Playwright timeout in the Windows UI lane waiting for chat bubbles.

GGUF inference smoke twice and Chat UI, API, Update and GGUF inference all fail on TypeError: Messages.create() got an unexpected keyword argument 'temperature', the Anthropic 1.0 SDK break that #9431 and #9432 are handling. I did not re-run those, since that failure is deterministic rather than flaky and will stay red until one of them lands.

Repo tests (CPU) fails on frontend structure assertions about chat-adapter.ts and the serverTuningState symbols. This branch differs from e91f9720b1 in exactly two files, both under studio/src-tauri/src/, so every input that test reads is byte for byte main's and it fails there too.

So the checks that can speak to this change are green: Tauri Linux debug build, Rust unit tests (macos) and Rust unit tests (windows) all pass.

…r overrides

Three things, all found while checking this branch against the WebKit sources
rather than against descriptions of them.

## The two switches are not interchangeable, and X11 wants the lighter one

Traced in webkitgtk-2.50.4, the version the AppImage bundles:
DISABLE_DMABUF returns before mode.add(SharedMemory), so the set stays empty,
checkRequirements() is false, AcceleratedBackingStore::create() returns nullptr
and webkitWebViewBaseEnterAcceleratedCompositingMode() dereferences that behind
an ASSERT that release builds compile out. FORCE_SHM returns after the add and
leaves a valid backing store.

That is not only a smoothness question. block/buzz#3654 reports a SIGSEGV on
Ubuntu 24.04, WebKitGTK 2.52.3, X11, NVIDIA module loaded but EGL on Mesa, with
DISABLE_DMABUF, and no crash with FORCE_SHM. That is exactly the iGPU-presenting
topology the module probe over-triggers on.

So the branch now picks per failure mode rather than per GPU. Wayland keeps
DISABLE_DMABUF: the failure there is the explicit-sync disconnect, FORCE_SHM
routes every commit down the wl_shm path that trips it (WebKit bug 315436), and
DISABLE_DMABUF is the switch with reports of fixing Error 71. X11 on 2.44+ takes
FORCE_SHM, which drops the hardware transport that fails there without emptying
the set. Old WebKitGTK and the missing-GLES AppImage keep the stronger switch.

## Operator overrides win again, without pinning a stale decision

Ignoring every FORCE_SHM value on NVIDIA also discarded a deliberate setting,
and it was the setting that avoids the crash above. Both variables are operator
overrides again. The relaunch case that motivated ignoring them is handled by
naming the variable we set in UNSLOTH_WEBKIT_RENDERER_WORKAROUND, so a launch
can tell its own inherited output from an instruction, re-decide, and clear the
previous variable when the new decision differs. Without that a launch reads its
own output back and the first decision is pinned for the life of the process
tree; a new test asserts the plan is a fixed point instead.

WEBKIT_FORCE_DMABUF_RENDERER is honoured too. It is the opt-out
disable-nvidia-dmabuf.patch ships for itself, checked inside isNVIDIA(), which
WebKit never reaches once DISABLE_DMABUF is set, so this branch was silently
outranking the only lever a Debian or Ubuntu user has. It stands the NVIDIA
branch down and nothing else: the missing-GLES fallback answers a packaging
failure, not a GPU policy, and that launch cannot render without it.

## Diagnostics

An NVIDIA AppImage that cannot load GLES reported only NVIDIA and hid the
packaging defect. It now reports both. Two harness docstrings still said this
module forces SHM on Wayland; both switches and both display servers are now in
scope, so they say so.

19 tests to 27; the whole crate is 415 passing. Verified against the extracted
Ubuntu 2.50.4 and Debian 2.52.6 patches (isNVIDIA() before the SharedMemory add,
empty set) and 2.53.91 (after it, {SharedMemory}). A differential over 4860
host, session, packaging, inherited-environment and version combinations shows
no host losing a workaround it had, and no operator override ignored.
@danielhanchen
danielhanchen self-requested a review as a code owner August 24, 2026 05:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aec95dcd4a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/src-tauri/src/linux_webkit.rs Outdated
Comment on lines +108 to +111
// Same rule for the shared-memory switch: do not combine it with legacy
// instructions a user may already carry in a launcher or environment.d file.
if env(FORCE_SHARED_MEMORY).is_some() && !ours(FORCE_SHARED_MEMORY) {
return RenderingPlan::PreserveEnvironment;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Migrate unmarked workaround state from older relaunches

When an NVIDIA/Wayland user installs this revision through the in-app updater, the old process has already set WEBKIT_DMABUF_RENDERER_FORCE_SHM=1, and relaunch() starts the replacement as a child that inherits it. Fresh evidence in this revision is that ownership now depends solely on APPLIED_WORKAROUND, but the parent revision never emitted that marker, so this check classifies the inherited app value as an operator override and skips the required NVIDIA/Wayland DISABLE_DMABUF decision. Handle the legacy unmarked FORCE_SHM=1 state during the first post-update relaunch rather than preserving it.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fixing this one, and the mechanism is not in dispute: an in-app update does relaunch as a child, the parent revision never wrote a marker, and so that first relaunch does read its inherited WEBKIT_DMABUF_RENDERER_FORCE_SHM=1 as an operator value and stands down.

Three reasons it stays.

The cost is one session, and it is the previous release's own behaviour. That user was already on FORCE_SHM before the update; they are not worse off, they just pick up the new decision one launch later. The next start from the launcher has a clean environment and decides normally, so it self-heals without the user doing anything.

The migration cannot be written without guessing. An unmarked FORCE_SHM=1 on NVIDIA/Wayland is byte for byte what an operator gets by exporting it, so any rule that reclassifies it as app state also silently overrides that operator. That is the rule 94d2c95 had and the one this branch removed. It matters more now than it did then: on NVIDIA, FORCE_SHM=1 is the setting that keeps a valid backing store, so overriding it is the direction that can crash rather than merely slow things down.

The right place is not this module. If we want a post-update relaunch to start clean, the updater should not hand the replacement the old process's renderer decision at all, alongside the PYTHONHOME/PYTHONPATH scrubbing update.rs already does. That fixes the class rather than special-casing one inherited value, and it is a separate change from this one.

Happy to be argued out of it if there is a signal that separates our own inherited value from an operator's without a marker, but I could not find one.

Same decisions, fewer lines: 94 comment lines out, 58 back. Verified behaviour
preserving by comparing the plan and the reason string against the previous commit
over 420000 input combinations, 0 differences. 415 crate tests pass, rustfmt clean.
…nd-down for PR unslothai#8884

Two genuine Codex findings, both reproduced against the shipped sources.

FORCE_SHM alone never reached selection on a patched library: isNVIDIA() returns
before mode.add(SharedMemory) in Ubuntu 2.50.4 and Debian 2.52.6, so the X11 branch
still handed NVIDIA an empty transport set. Its own opt-out is checked first inside
isNVIDIA(), so ForceSharedMemory now sets WEBKIT_FORCE_DMABUF_RENDERER alongside
WEBKIT_DMABUF_RENDERER_FORCE_SHM. Unpatched libraries ignore it, and 2.53.91, which
moved isNVIDIA() after the add, lands on the same {SharedMemory}.

Standing down dropped the ownership marker but left the variable set, so a relaunch
that no longer wanted a workaround kept it, and the launch after that read the now
unmarked value as an operator override and preserved it for good. release_claimed
clears the values we set, never an operator's.

The marker holds the variable list rather than one name, so ownership still covers
a workaround that takes two, and our own FORCE_DMABUF no longer reads back as an
operator opting out.

419 crate tests pass. The 4860-case host matrix still shows no regression, and the
NVIDIA X11 rows now reach {SharedMemory} instead of the empty set.
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
chatgpt-codex-connector[bot]

This comment was marked as resolved.

danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 24, 2026
chatgpt-codex-connector[bot]

This comment was marked as resolved.

danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 24, 2026
@danielhanchen

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a411741f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return true; // a TCP or otherwise unusual display; not ours to judge
}
let path = format!("{socket_dir}/X{number}");
if UnixStream::connect(&path).is_ok() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Complete the X11 handshake before selecting its backend

When DISPLAY points to a reachable X socket but the X11 setup is rejected—for example because XAUTHORITY is missing or stale—this bare UnixStream::connect succeeds even though GDK's X11 opener fails and proceeds to the next backend. With GDK_BACKEND=x11,wayland, the plan therefore misclassifies the actual Wayland session as X11: non-NVIDIA hosts omit the Wayland workaround, while NVIDIA hosts choose the X11 FORCE_SHM path instead of the required Wayland workaround. Verify the X11 setup/authentication response rather than treating transport-level connection success as an opened display.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not taking this one, and the reason is the direction of the risk rather than the premise.

The premise is right: a reachable socket is not an opened display, and a stale or missing XAUTHORITY makes GDK skip the x11 opener. But the default configuration cannot reach the misclassification. With GDK_BACKEND unset the backend list is *, and that arm answers wayland before x11, so a live compositor already wins whatever DISPLAY says. It takes an explicit x11-first GDK_BACKEND and broken X authentication and a live compositor to land here.

Against that, the only correct implementation is the real thing: parse the .Xauthority binary format, pick the entry matching family, address, display and name, send the setup request and read the reply. That is an X11 protocol implementation running before GTK init, and its failure mode is the bad one. The probe is fail-safe on purpose, because calling a live server closed selects the Wayland branch, which on NVIDIA is WEBKIT_DISABLE_DMABUF_RENDERER on an X11 webview, and that is the empty transport set and the startup SIGSEGV the rest of this file is written around. A cheap version does not exist either: a setup request sent with no auth data gets Failed back from any server with access control on, which is every healthy session.

So the trade is a third-order misclassification that costs the wrong renderer switch, against new protocol code whose bugs cost a crash. Leaving the transport-level probe as the line, and it is documented as fail-safe for exactly this reason.

@danielhanchen

Copy link
Copy Markdown
Member

@codex review

1 similar comment
@danielhanchen

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 5a411741f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen
danielhanchen merged commit ce0fa66 into unslothai:main Aug 24, 2026
44 of 46 checks passed
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
danielhanchen added a commit to Datta0/unsloth-staging-3 that referenced this pull request Aug 25, 2026
@mahiatlinux
mahiatlinux deleted the fix/studio-nvidia-dmabuf-renderer branch September 25, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants