Typing break monitor doesn't quite recognize idle time properly with niri

[jjramsey]

This issue reflects a problem described here: rcaelers/workrave#565 (comment)

On LabWC, Hyprland, and the new COSMIC desktop alpha, installing xdg-desktop-portal-gtk is enough to get time watching video on Firefox (without keyboard or mouse input) to be recognized as idle time in the Workrave typing break monitor, probably because Firefox uses the dbus org.freedesktop.impl.portal.Inhibit interface from the GTK portal (rather than Wayland's idle-inhibit protocol, which works poorly with typing break monitors).

However, what works with LabWC, Hyprland, and COSMIC doesn't work with Niri, and I don't know why.

System Information

• niri version: niri 0.1.8 (unknown commit)

• Distro: Arch Linux

• GPU: AMD Radeon RX 7800 XT

• CPU: Intel Core i5-12400

[YaLTeR]

Niri does implement the freedesktop Inhibit interface and Firefox video watching is supposed to trigger it. You can test it by starting swayidle timeout 1 'echo hi', then starting a YouTube video on Firefox. For me it doesn't print while the video is playing, meaning that idle is successfully inhibited.

[jjramsey]

I think you misunderstand. Firefox will try (at least?) two methods of idle inhibition: first, the one involving the Freedesktop org.freedesktop.impl.portal.Inhibit interface -- which doesn't interfere with typing break monitors like Workrave, while still preventing idle monitors from timing out -- and second, the Wayland idle inhibit protocol extension -- which is a lot more coarse-grained and prevents typing break monitors from recognizing intervals without user input as idle.

Your test can't tell the difference between the two methods of idle inhibition.

Something in the setup for Niri's seems to prevent the org.freedesktop.impl.portal.Inhibit interface from becoming active, causing a fallback to the more coarse-grained Wayland idle-inhibit protocol.

[YaLTeR]

I looked into this when implementing inhibiting, and Firefox pretty much cannot ever use the Wayland inhibit because the portal Inhibit never fails. I believe I opened some issues in the portals about this, but they always returned success. This was why I even implemented the freedesktop inhibiting in niri in the first place.

You can actually tell these apart. The Wayland inhibiting is only active when the inhibiting window is on screen. If I do the swayidle test with a Firefox YouTube video, then switch to a different workspace, swayidle still does not print anything, which indicates that Firefox uses the freedesktop inhibit on my system.

[jjramsey]

Then why is Workrave acting as if the Wayland inhibit protocol is being used?

There's a pretty straightforward check:

• Install Workrave. Unfortunately, you'll need the 1.11 git code for this, compiling it with the -DHAVE_WAYLAND:BOOL=TRUE CMake flag (see Workrave Wayland support (meta) rcaelers/workrave#523 (comment))
• Run Workrave in a Niri session
• Start Firefox, start watching a YouTube video, and keep away from the keyboard and mouse. You should see Workrave stop incrementing its timers in its status window as it recognizes that you aren't doing anything. If you aren't, then the Wayland idle inhibit protocol is being used. (Note that swayidle should still treat this time as not idle, because it's receiving signals from the org.freedesktop.impl.portal.Inhibit implementation to not time out.)

(ETA: I did try switching to another workspace, and Workrave's timer did still count down, so I guess Niri's org.freedesktop.impl.portal.Inhibit implementation is being used instead of the one from the GTK portal. That would indicate that Niri's implementation has the same weakness as the Wayland inhibit protocol of failing to deal with the case where the computer is not idle but the user is. The GTK and KDE implementations of the org.freedesktop.impl.portal.Inhibit interface do not have this problem.)

[YaLTeR]

swayidle should still treat this time as not idle, because it's receiving signals from the org.freedesktop.impl.portal.Inhibit

I'm fairly sure that swayidle only receives idle events from the ext-idle-notify-v1 Wayland protocol, i.e. exactly what niri tells it.

Niri's implementation has the same weakness as the Wayland inhibit protocol of failing to deal with the case where the computer is not idle but the user is.

The most common use for the idle protocol is to power off the screens after some time of inactivity, and the most common use for inhibiting idle is to prevent this powering off from happening as you're watching a movie. So arguably the fault is with the program that assumes it can use this idle indicator for user inactivity.

[jjramsey]

I'm fairly sure that swayidle only receives idle events from the ext-idle-notify-v1 Wayland protocol, i.e. exactly what niri tells it.

IIRC, swayidle also listens for D-Bus signals that tell it not to trigger its idle timeouts. (I think this works via systemd.) I already tested this in Hyprland. If I run swayidle timeout 1 'echo hi', it stops repeating 'hi' when playing video in Firefox even though Workrave still recognizes that the user is idle.

The most common use for the idle protocol is to power off the screens after some time of inactivity, and the most common use for inhibiting idle is to prevent this powering off from happening as you're watching a movie. So arguably the fault is with the program that assumes it can use this idle indicator for user inactivity.

If you look at the discussion at https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/29, you can see that RSI tracking (i.e., tracking user activity to help deal with Repetitive Stress Injury to hands and wrists) is part of the use case for the ext-idle-notify protocol. So no, it's not an abuse of the protocol, but rather an expected (though niche) use.

There already is a way to keep the computer from powering off without falsely indicating that the user is typing or using the mouse. That's what the org.freedesktop.impl.portal.Inhibit D-Bus interface is good for. The one from the GTK portal works just fine in LabWC, Hyprland, and COSMIC, and the one from the KDE portal works just fine in, well, KDE. Niri is the outlier here.

[YaLTeR]

I'm not sure what you're suggesting. Niri does not inhibit idle through the dbus interface or "interfere" with it. It listens to it to relay the inhibiting to the Wayland idle notify protocol.

[jjramsey]

I'm not sure what you're suggesting. Niri does not inhibit idle through the dbus interface or "interfere" with it. It listens to it to relay the inhibiting to the Wayland idle notify protocol.

So it isn't trying to implement org.freedesktop.impl.portal.Inhibit at all, and just uses whatever implementation comes from an XDG portal?

(I thought you indicated that you did implement it, but I guess I misunderstood.)

[YaLTeR]

Niri doesn't implement any portals. What it does implement is an org.freedesktop.ScreenSaver inhibit interface https://github.com/YaLTeR/niri/blob/main/src%2Fdbus%2Ffreedesktop_screensaver.rs#L23 which the gtk inhibit portal impl calls, if I'm not mistaken.

[jjramsey]

Gotcha.

If I wanted to test a version of the Niri code without that org.freedesktop.ScreenSaver interface implementation, how would I go about it? Loosely speaking, what would I need to comment out without breaking anything I didn't want to break?

[YaLTeR]

Should be fine to just comment this:

niri/src/dbus/mod.rs

Lines 56 to 57 in 9608384

	let screen_saver = ScreenSaver::new(niri.is_fdo_idle_inhibited.clone());
	dbus.conn_screen_saver = try_start(screen_saver);

[jjramsey]

That did it. I tried a version of Niri with those lines commented out, and I got the behavior with Workrave that I get with those other compositors that I mentioned.

ETA: After I got the behavior of Niri to be how I wanted it, I got a bit fancier. In niri/niri-config/src/lib.rs, I added

#[knuffel(child, default)]
    pub prefer_no_screensaver_impl: bool,

to the Config struct, and in niri/src/dbus/mod.rs, I now have

            if !(config.prefer_no_screensaver_impl) {
                let screen_saver = ScreenSaver::new(niri.is_fdo_idle_inhibited.clone());
                dbus.conn_screen_saver = try_start(screen_saver);
            }

My modified Niri now does what I want it to do if I have prefer-no-screensaver-impl in the config file. I don't know if I'd really want the option to be called prefer-no-screensaver-impl, but it's at least a proof-of-concept for a reasonable solution.

[YaLTeR]

Well, the unfortunate thing is that without it, flatpak Firefox has no way to stop swayidle from turning off the monitors. Due to the aforementioned bugs in the portals that always return success, so Firefox never falls back to the Wayland inhibiting. This goes for any flatpak app that only knows about the inhibit portal, too.

[jjramsey]

Yes, but the needs of those who have RSI and use typing monitors to keep it in check are worth considering as well, and my proposed fix with the prefer-no-screensaver-impl can help accommodate them.

Consider it a workaround for the messy state of idle monitoring on Wayland, which kinda sorta distinguishes between the user being not idle (by providing input) and the computer being not idle (because it's playing video or whatnot), except when it doesn't. Basically, we have something like this:

• There's the Wayland ext-idle-notify extension protocol, which measures idleness with the only inputs it has available: keyboard and mouse input, and signals from apps using the Wayland idle-inhibit protocol. If apps avoid the Wayland idle-inhibit protocol, ext-idle-notify can reliably measure if the user is idle. The ext-idle-notify protocol is also the only game in town for Wayland-based typing break monitors to keep track of user input.

• There are D-Bus-based protocols (e.g. https://systemd.io/INHIBITOR_LOCKS/) that apps like swayidle and can use (and at least to some degree do use, see here: Handle logind inhibitors swaywm/swayidle#38) to supplement the input from ext-idle-notify in order to determine whether to time out. You can have a scenario where the Wayland compositor indicates that the user is idle (via ext-idle-notify and a lack the use of the Wayland idle-inhibit protocol) but other D-Bus signals can indicate to swayidle that the computer isn't idle, and thus shouldn't do into a time out to run a screen locker or whatnot.

Arguably, in a more ideal world, we'd have the Wayland compositor take the responsibility for measuring user activity and the D-Bus protocols to indicate that the computer is doing something that shouldn't be interrupted via suspension, screen locking, etc. In the current world, we almost have this, but the Wayland idle-inhibit protocol complicates the former, and dodgy implementations of the org.freedesktop.impl.portal.Inhibit interface complicate the latter.

My proposed fix with the prefer-no-screensaver-impl option would help navigate this mess somewhat -- though it would have its tradeoffs -- at least until the current state of idle monitoring gets better.

ETA: I'm up for creating a pull request with the proposed fix, with perhaps a better name for the option than prefer-no-screensaver-impl.

[YaLTeR]

I can't promise I'll merge that because really it is a hacky workaround that just happens to work for your particular scenario. Had Firefox successfully used the Wayland inhibit instead, you would end up with the exact same problem where idle time watching a video is not counted as idle time. Or with any other client that uses the Wayland inhibit.

You may want to just keep running the patched niri version for this, it's not like this delete 2 lines patch is gonna break.

I can see how a way for monitoring the user activity is useful. It would not even be hard to implement, in fact it would be strictly easier to implement (the same as the current idle-notify but with no inhibiting). If there's another D-Bus interface or protocol for this, we can implement that. Or we could create one (but then the target app also needs to know about it).

Actually, it seems that Workrave can use a Mutter-specific D-Bus interface: https://github.com/rcaelers/workrave/blob/main/libs/input-monitor/src/unix/MutterInputMonitor.cc I don't know if it handles inhibiting better, maybe it does.

[jjramsey]

I can't promise I'll merge that because really it is a hacky workaround that just happens to work for your particular scenario.

I agree that it would be a very imperfect workaround for the reasons that you mentioned, but it would at least allow Niri to be as accommodating to Workrave (and similar typing break monitors) as the rest of the Wayland compositors that implement ext-idle-notify. I would consider it a simplistic, relatively noninvasive short-term solution that lets other Workrave users besides me use Niri.

Longer term solutions would unfortunately likely require changes to Wayland protocols. 😬 Perhaps, if you are willing to go down a rabbit hole, you could modify Niri to provide proof-of-concept implementations of what the updated protocols could look like. I can conceive of a few possible ideas, though how wise or feasible they'd be is up in the air:

• You could, for example, provide a debug option that would "break" Niri's implementation of the ext-idle-notify protocol so that it didn't respond to idle-inhibit requests from Wayland clients, leaving apps to resort to other (probably D-Bus-based) protocols to account for the computer not being idle even if the user is. Since the "broken" implementation still presents itself as an ordinary ext-idle-notify protocol, it would work with current Wayland apps that use it.

• You could provide a debug option that would cause Niri to interpret idle-inhibit requests from Wayland clients as requests for Niri to send systemd inhibitor lock signals (e.g., https://systemd.io/INHIBITOR_LOCKS/).

• For something that might be even more of a headache, you could provide a debug option that would cause Niri to interpret idle-inhibit requests from Wayland clients as requests for Niri to send some Niri-specific signal to some fork of swayidle that could accept such a signal. Said signal would indicate to this modified swayidle to not time out.

Again, I don't know how good or feasible any of these ideas would be, and I'm not entirely comfortable with proposing ideas that I wouldn't know how to implement. Take them for whatever they're worth.

[TelarinT]

Due to the aforementioned bugs in the portals that always return success, so Firefox never falls back to the Wayland inhibiting. This goes for any flatpak app that only knows about the inhibit portal, too.

There is a way to force the Inhibit portal to fail - by using org.freedesktop.impl.portal.Inhibit=none; in niri-portals.conf.
I tested by adding the line and checking dbus activity using Bustle, and by calling swayidle timeout 1 'echo hello world'.
Bustle did throw an error for org.freedesktop.impl.portal.Inhibit and swayidle is able to print only if (flatpak) firefox is in a different window

If this is the case could we use this method to force (flatpak) firefox to use the wayland protocol?

[YaLTeR]

If I recall correctly, this error from the impl is not propagated to the application. The portal call (without .impl.) still returns success.

[TelarinT]

This is the screenshot from Bustle when the Inhibit portal was disabled in portals.conf. You can see that the portal threw an error (without the Impl, my bad on saying that org.freedesktop.impl.portal.Inhibit threw an error). You can also see when the video state changes (those mpris2 calls), it stops calling the Inhibit portal entirely.

Maybe this is because I am using fedora flatpak instead of flathub (flathub doesn't have aarch64 builds), but they should still use xdg-portals.

I think when you were implementing the ScreenSaver interface it was an issue but it seems to be fixed in flatpak/xdg-desktop-portal#1255
Edit: gtk inhibit portal is still broken, we are just bypassing it by setting it to none

[YaLTeR]

Oh yeah, I remember bumping into that issue. That must've been it for this particular problem.