Wayland autotype not possible

[jm355]

I'm currently using Sway and was hoping to be able to use keepmenu to input passwords, however it looks like it's impossible to use autotype on wayland since autotype is currently implemented using PyUserInput, which uses xlib, which can't pass through to wayland even when using dmenu-wayland.

Would it be possible to switch from using PyUserInput to something a bit friendlier to different display server protocols, such as evemu which works on both wayland and x11? Here's a quick tutorial that has a pretty straightforward example, and I believe it could be pretty simple to swap in, although I haven't looked through how keepmenu currently does things to know how easy it really would be.

Edit: After a bit more looking another possibility would be ydotool

Thanks!

[firecat53]

A recent update added ydotool support. If that is installed and you select that as your input method in the config file `type_library = ydotool`, it should just work! Edit: just pushed a commit updating the documentation to reflect ydotool support.

[jm355]

Hmm, it still doesn't seem to work for me. ydotool on it's own works just fine, but it looks like keepmenu isn't even getting to the point of running ydotool. When I run the ydotool daemon in one terminal and have it type something in another, the ydotoold will output "accepted client" and the text I was testing with will be typed out. When I try to use keepmenu, ydotoold doesn't output "accepted client", no text is typed, and keepmenu just closes (stops running when run from terminal). I have the type library set to ydotool in the config, but it still doesn't seem to work

[firecat53]

I haven't had the time to install or try out Wayland yet. @danieljampen did the PR to add ydotool support...perhaps he'd care to weigh in?

[jampe]

@jeremiah-miller have to admit, I never used ydotool with the daemon. Does it work if you don't use the daemon? When you start keepmenu from a console, do you see any output / stacktrace you could provide? I'll try to test it with the deamon when I find some time, maybe at the weekend.

[jm355]

keepmenu doesn't work with or without the daemon, but running ydotool from a terminal works either way. When I run keepmenu from a console, it doesn't output anything, no warnings or stacktraces or anything, unless there's a verbose option I'm missing? I'm running sway on manjaro linux, with ydotool installed from the aur. I'm also using dmenu-wayland for dmenu, let me know how I can help figure out the problem!

[bwidawsk]

Using ydotool works for me and several others. The most recent version of ydotool in the AUR even packages a systemd service file for the daemon.

[fbearoff]

Can't get this working in a pure wayland environment under Sway installed via AUR. I'm using the wayland fork of rofi (1.7.0+wayland1-3-g2025b73e (makepkg)).

Ydotool works perfectly fine in the console. It appears keepmenu is ignoring "type_library" in the config and always using pynput.

Error message upon launch:

Traceback (most recent call last):
  File "/usr/bin/keepmenu", line 35, in <module>
    from pynput import keyboard
  File "/usr/lib/python3.9/site-packages/pynput/__init__.py", line 40, in <module>
    from . import keyboard
  File "/usr/lib/python3.9/site-packages/pynput/keyboard/__init__.py", line 31, in <module>
    backend = backend(__name__)
  File "/usr/lib/python3.9/site-packages/pynput/_util/__init__.py", line 76, in backend
    raise ImportError('this platform is not supported: {}'.format(
ImportError: this platform is not supported: ('failed to acquire X connection: Bad display name ""', DisplayNameError(''))

Try one of the following resolutions:

 * Please make sure that you have an X server running, and that the DISPLAY environment variable is set correctly

config:

[dmenu]
dmenu_command = /usr/bin/rofi

[database]
database_1 = ~/PWs.kdbx
autotype_default = {USERNAME}{DELAY 200}{TAB}{PASSWORD}{ENTER}
pw_cache_period_min = 5
type_library = ydotool

[firecat53]

Have to admit, I have yet to use anything with wayland long enough to test any of my dmenu/Rofi based projects. It's on my list!

[fbearoff]

Have to admit, I have yet to use anything with wayland long enough to test any of my dmenu/Rofi based projects. It's on my list!

Happy to help debug! This project is seriously the best implementation of keepass I've used on any platform!

[siikamiika]

There is also wtype which is a bit simpler to use and doesn't require adding the user to the input group:

# raw text input
printf %s password | wtype - # subprocess.Popen(['wtype', '-'], stdin=subprocess.PIPE)
# username + tab + password
printf %s username | wtype -
wtype -k tab # /usr/include/X11/keysymdef.h without the `XK_` prefix
printf %s password | wtype -
# username, sleep 200ms, tab, password
printf %s password | wtype username -s 200 -k tab -
# modifiers or other combinations with -P ctrl -p ctrl; available: "shift", "capslock", "ctrl", "logo", "win", "alt", "altgr" (from manpage)

Though one annoying thing about Wayland is that you must use the appropriate autotype program for each client depending on if it's Wayland native or XWayland. xdotool works on Wayland if you're entering text to an XWayland client.

Edit: oops, this seems to be fixed in wtype git master: atx/wtype#1

[siikamiika]

Update: I have a POC here that I'm currently using
https://github.com/siikamiika/os-setup/blob/53054396dde3d25b64903965b27fbd615b222a65/linux/arch/custom/passwords/password_menu.py#L143-L149

While researching this I also found that pynput is looking into implementing the same API that wtype uses
moses-palmer/pynput#331

[firecat53]

I finally played around with Sway for long enough to at least get ydotool
working with keepmenu/bwm. Rofi and dmenu both seem to work (I'm assuming
they're using Xwayland). I added some notes to the docs about enabling ydotool.

I'd be open to a PR allowing wtype to be used. I'm not intending to switch to
Wayland yet, so I'll need some community support in testing.

I also haven't done any testing under Gnome (non-wlroots Wayland) yet.

[siikamiika]

I haven't tested under Gnome either, but I doubt it would work: moses-palmer/pynput#331 (comment) atx/wtype#22

I can look into this when I get a chance, another project is taking a lot of time at the moment.

[firecat53]

I did end up trying Gnome/Wayland, and none of the menu options that I'm aware
of will work there. They all need a wlroots compositor like Sway. Gnome/X11 is
fine: Rofi, dmenu, bemenu all work normally.

I'd love to hear anyone with a solution for Gnome/Wayland. That's a pretty
substantial user base nowadays.

[jm355]

Looks like ydotool just had a big refactor, maybe v1.0.0 will fix this? https://github.com/ReimuNotMoe/ydotool https://github.com/ReimuNotMoe/ydotool/releases/tag/v1.0.0

[fictitiousexistence]

Getting the same error as @fbearoff but on gentoo.

"Please make sure that you have an X server running, and that the DISPLAY environment variable is set correctly"

Is it not possible to run keepmenu on a pure wayland setup?

Also using ydotool in config and bemenu.

Thanks.

[firecat53]

@fictitiousexistence I did get it working on Sway with ydotool last November, but honestly haven't tried it again since. It's on my todo list though. Make sure you're running the most recent version of keepmenu. Also, ydotool is now on version 1.0.0 and I haven't used it since then to check if my README instructions still work.

Please specify exactly what window manager you are using and post your keepmenu config so I can try to replicate.

[fictitiousexistence]

Window manager is "labwc"

Config:

[dmenu]
dmenu_command = bemenu

[dmenu_passphrase]
obscure = True
obscure_color = #555555

[database]
database_1 = /home/user/example.kdbx
keyfile_1 =
pw_cache_period_min = 360
autotype_default = {USERNAME}{TAB}{PASSWORD}{ENTER}
editor = vim
terminal = foot
gui_editor = subl
type_library = ydotool

[firecat53]

I was again able to check that keepmenu works with ydotool under both Sway and labwc using both bemenu and Rofi.

Instructions for ydotool are under docs/install.md.

If you are using ydotool >= 1.0.0, you will also need to grab the latest version of keepmenu from Github with a just merged PR for keycodes in ydotool (autotyping will work, but special characters like and will not).

Here are my relevant ENV variables under sway if that helps (I started it via GDM on Fedora 35):

I3SOCK=/run/user/1000/sway-ipc.1000.3162625.sock
DESKTOP_SESSION=sway
XDG_SESSION_DESKTOP=sway
XDG_SESSION_TYPE=wayland
SWAYSOCK=/run/user/1000/sway-ipc.1000.3162625.sock
WAYLAND_DISPLAY=wayland-1
ALACRITTY_SOCKET=/run/user/1000/Alacritty-wayland-1-3163724.sock
GDMSESSION=sway

My keepmenu config file is basically the same as yours. I don't see any issues there. I'm suspecting it's a setup issue.

[firecat53]

I did discover while playing with this the past couple of days that I need to either lazy load pynput or just make it optional. Otherwise keepmenu fails when the pynput module fails to load if xwayland is disabled. As long as xwayland is present, it works fine.

Also, ydotool apparently works under X as well as Wayland. TIL.

[fictitiousexistence]

@firecat53 That would be great. I noticed some issues with xwayland on my system. I have xwayland installed and can launch it. But ultimately if it doesn't need to load at all that would be optimal.

[firecat53]

I think I'm finally able to close this issue. There are options that work on both X and Wayland, including (mostly) Gnome/Wayland. Lots of Wayland caveats though.

	X	Wayland/Xwayland	Pure wlroots Wayland	Gnome/Wayland(3)	Unicode Support
Launchers
Dmenu	Yes	No	No	No
Rofi	Yes	Yes	No	No
Bemenu	Yes	Yes	Yes	No
Wofi	No	Yes	Yes	Yes
Typing Tools
Pynput	Yes	No	No	No	No
Xdotool	Yes	No	No	No	Yes
Ydotool (1)	Yes	Yes	Yes	Yes	No (4)
Wtype (2)	No	Yes	Yes	No	Yes

(1) Ydotool has a bug currently that incorrectly types ^ as ~.
(2) Wtype has a bug currently that prevents it from typing into web based applications such as Firefox, Chromium, VSCode. It does work in terminals, pinentry dialogs, LibreOffice and Qutebrowser.
(3) Gnome modal dialogs for SSH/GPG key entries are unusable with any password manager that performs autotyping. You have to copy the password to the clipboard before you need it and paste into the field because the dialog does not allow you to navigate away once it's open. Copying to clipboard is not currently possible in Keepmenu, although I intend to add support at some point.
(4) Supposedly you can change the keyboard language of the ydotool virtual device in the Sway config to enable support for characters on that keyboard. Did not test that.

Most flexible option for X (i3, Gnome on Xorg): Rofi with xdotool
Best option for pure Wayland with English US keyboard: Bemenu with ydotool (except for (1)).
Best option for pure Wayland with non-English US keyboard: Bemenu with wtype (except for (2)).

So, good luck! I hope Wayland support for all these tools improves at some point.