Winit Features and Scope

[Osspial]

Rendered

Adds FEATURES.md in place of SCOPE.md, as discussed in #674. Subsumes #252. I've replaced the tier 1/2/3 structure with a new Core, Platform, and Usability tier structure, as I think that aligns more with what we're trying to accomplish.

Discussion points highlighted in the document:

• Do we want to support WASM without Emscripten?
• I've introduced the concept of upgrading features from Platform to Core, as some features (window resize increments, child windows) may end up becoming core. Do we want to have this be an official process as I've described?
• Do we want to support child windows?
• Do we want to support gamepad and joystick events?
• Do we want to support device movement (gyroscope and joystick) events?
• The 1.0 and stability section is based on my understanding of our end goal for Winit. Is is accurate?
• Some features in the core matrix are marked as partially complete, but don't have an associated issue listed. Do such issues exist?

[Osspial]

cc @francesca64 @vberger @mitchmindtree @tomaka.

Relevant issues that may or may not be addressed in the document right now:

• Input handling and filtering
  • Double-clicks: How are double clicks handled? #642
  • IME: WM_IME_* support in win32 #508
  • Key repeats: A way to disable key repeats #310, Key presses report "virtual" key presses (including repetition), not physical key down events #52
• Device/gamepad events: Joystick events #624, Gamepad support #119, Consider expanding scope of raw input #212
  • Pen/tablet input: Pen/Tablet Input Support #99
• Window information:
  • Window creation w/ physical dimensions: Create window with physical dimensions #615
  • Fixed window aspect ratio: Allow setting minimum and maximum or fixed aspect ratio #584
  • Window::get_fullscreen() Add Window::get_fullscreen() -> bool #579
• Window appearance:
  • Glassy windows: Blur-Behind / Glassy Windows #538
• wasm32-unknown-unknown: Support wasm32-unknown-unknown backend? #395

Other relevant issues that I'm inclined to close as stuff we won't be changing from its current API state:

• Screensaver toggle: Enable/Disable screensaver toggle #616 (out of scope)
• Native widgets and menus: Native Widget Handling #446, Noob Question on Menus etc #427, Pop-up menus #403 (out of scope)
• Windows other messages: windows - support for other messages #450 (supported by subclassing)
• Configuration files: Configuration file support #304 (supported by serde)

[Osspial]

My assorted thoughts on some of those issues, in no particular order:

Generalized gamepad and joystick support is something I think we should support. It's a feature that most people expect us to support, falls well within the category of cross-platform input handling abstraction, and is implemented within the platform's event loop in several places.

Proper child window support is really tricky to get right outside of Winit, and requires platform-specific code to do properly. On Wayland it's impossible without platform calls, and Winit doesn't expose any way of converting client-space coordinates to screen-space coordinates. Additionally, disabling a window in favor of a child window is a common behavior for desktop applications (see file dialogs and are you sure you want to close without saving dialogs) so it makes sense to expose that here.

Gyroscopes and accelerometers are fairly commonly used in mobile games, so I see no reason not to expose them as events.

Double-click and key repeat notifications are both things that could be handled outside of Winit, but neither of them take up a significant amount of API surface area on our end. I think it makes for a better abstraction to track those numbers internally and deliver them in click and keypress events.

[MatusT]

I would like to see added what everyone usually forgets about in every windowing API(or any new cross-platform GUI library) and that is split between Win32 and WinRT APIs on Windows.

I don't know if anyone else is interested but I would love to have a windowing library in Rust with which I could make a UWP native application on Windows(as with WinRT/C++) that I could deploy to Microsoft Store. Bonus: render with vulkan on directx 12 through gfx-hal into the window.

Although all this is stuck on contextfree/winrt-rust#53 which I have no idea how to solve.

[elinorbgr]

I believe this can be in scope for winit. As you said (at least on wayland) creating child window require direct platform interaction in general, and can be difficult to expose without exposing a lot of winit's internals.

[mitchmindtree]

Proper child window support is really tricky to get right outside of Winit, and requires platform-specific code to do properly. On Wayland it's impossible without platform calls, and Winit doesn't expose any way of converting client-space coordinates to screen-space coordinates. Additionally, disabling a window in favor of a child window is a common behavior for desktop applications (see file dialogs and are you sure you want to close without saving dialogs) so it makes sense to expose that here.

What it really means for a window to be a child window is still slightly unclear to me, maybe a description of what a child window can offer that another regular window cannot would be worth adding?

Reading the current entry, It seems to me that the relative positioning could be solved using the difference between get_position on two regular windows. Disabling a window in favour of another (e.g. disallowing user input for a popup) sounds to me like something could be handled by application logic (e.g. filter out input events associated with WindowIds that aren't popup_window_id).

I guess I just want to make sure we're not exposing something that could otherwise be done using application logic in order to reduce the winit surface area. I'm also a little concerned that the concept of a "child" window and what that means might be different per platform, though I haven't actually looked into this at all yet - it could very well have a very specific meaning across each platform.

All that said, I have read some users mention that child windows are simply essential in some use cases. One example that comes to mind is the VST API, though I don't know much about it.

[elinorbgr]

Yes, this is a point I discussed in an earlier issue / PR. I don't know how this works for other platforms, but in wayland, drag'n'drop is unified for all kind of contents. The way it works is that the application starting the d'n'd and the application receiving it negociate a mime type for the data transfer. Dragging files from a file explorer is just a special case of this.

Can this kind of general behavior be transposed to other platforms?

[Osspial]

I assume? I know that it works on Windows, and I'm pretty sure I've done it before on macOS and X11 but I can't quite recall.

[francesca64]

I can confirm that it's similar on X11 and macOS, so I think we can reasonably provide this functionality.

Also, what about knowing the hover position and being able to decide whether to accept or reject a drop (based on both position and type)? Anyone building GUIs on top of winit will demand this, and it's at least easy to implement on X11.

[elinorbgr]

Also, what about knowing the hover position and being able to decide whether to accept or reject a drop (based on both position and type)?

That's also the natural way it works on wayland. With the added case that the sender application can actually provide a list of available mime types that the receiver can choose from.

[Osspial]

From skimming the Windows code, that also seems to be how Windows does it. The one question I have is how do we implement communication from the callback back to the operating system to accept or reject the drop? The best way I can think of doing it would be to pass a mutable reference along with the event that the callback can modify (as otherwise we're getting into the realm of having different return types for different events), but that isn't possible on Windows before Event Loop 2.0 gets merged.

[Osspial]

Another thing - Winit supports dragging content into windows. Should we support dragging content from windows? In principle I'm not opposed to it, but I don't know what the API would look like and it starts to turn Winit from windowing and user input to general cross-platform desktop interfacing which is a substantially bigger project.

Perhaps we could publicly expose some of Winit's internals in a permanently unstable way (say, hide them behind a feature or make them all #[doc(hidden)]) and split that into a separate crate, alongside non-windowing stuff like clipboard handling? Doing that would have issues and I'm honestly not entirely sold on it myself, but it would allow us to keep the core of Winit simple and build more specialized libraries around it.

[elinorbgr]

Agreed. I believe this was already decided that this is in scope for winit.

[Osspial]

I've opened an issue to discuss how that should work at #704

[Osspial]

The more I think about it, the less comfortable I am declaring this as in-scope for Winit. Right now, Winit does a decent job of keeping the scope of its API small as exclusively a window creation library, and adding clipboard support starts to turn it into a grab-bag of assorted desktop interfacing tools. I don't want to be in a situation where Winit starts snowballing and gaining more and more features that aren't directly related to windowing/user input because it doesn't expose enough of its internals.

@vberger If we exposed Winit's pointers to the Wayland environment through unix::EventsLoopExt (so the Wayland registry, display, compositor, subcompositor, shell, data device manager, and decorations manager), could the implementation work for this be moved out of Winit? Admittedly that's a fairly significant number of internals, but they're only necessary to expose because Wayland doesn't use global state nearly as much as other APIs. If we can do that, we can move substantial amounts of design and maintenance work outside Winit's core.

[elinorbgr]

While on X11 this can be done without winit, on wayland this requires at least some collaboration from winit to be possible. I don't know about other platforms.

[francesca64]

This is something I've gone back and forth on a lot in my head, though I've ultimately been leaning in favor of it. As @Osspial said, it's something people expect, and it's still largely an open problem in the Rust ecosystem.

I'm pretty happy with the results I've had implementing Raw Input and XInput gamepad support on Windows. Since supporting Raw Input requires a window to receive events (which is why it was said to be out of scope for gilrs), and we already have a bunch of Raw Input abstractions in master, winit is really the simplest place this could be handled.

While XInput is polling-based, XInput devices also send Raw Input events, so I was able to rely on that both to only poll when I know we won't have to wait, and to dynamically register which XInput devices are connected and disconnected (attempting to poll a nonexistent XInput device is very slow, and AFAIK there's no part of the XInput API for enumerating devices). I think all of this is a great boon, and it's something I don't think could be implemented as elegantly if done outside of winit.

There's the issue of other platforms, though. On Linux, X11 and evdev are rather orthogonal, and different implementations will be required for BSD/etc. I haven't researched macOS at all, or the situation with the non-desktop platforms. Before pursuing a feature like this, I'd want us to have a clear view of how it would be integrated into the event loop on each platform. It's also worth noting that device-specific code is often needed to support force feedback, though Chromium is a good reference for that and it doesn't seem that extensive.

I think that any gamepad support provided by winit should be "low level", which is to say, a stream of events that's been transformed only enough to have a consistent format across platforms. Something like SDL mappings would have to be applied downstream, though we'd still have to research what info we'd need to provide for that to be possible.

[elinorbgr]

I suppose this is mostly relevant for iOS/Android platforms?

I'd say that if we decide Gamepad/Joystick is in scope for winit, then this should be as well.

[Osspial]

Yeah, and it looks like it's supported on Emscripten as well. I don't know if it works on desktop platforms, though.

I agree that it makes sense to talk about these in the same category as gamepad/joystick events.

[elinorbgr]

On Wayland it's impossible without platform calls, and Winit doesn't expose any way of converting client-space coordinates to screen-space coordinates.

Screen-space coordinates are by design not available to clients on wayland. So while it'd be possible to define a parent surface for dialogs, hey cannot be explicitly placed wrt to their parent.

It's an other matter if we are considering tooltips. These are fundamentally different from windows in wayland, and they can be placed wrt to their parent. I think not all platforms make such a distinction, this will require some discussion to unify the platforms on that.

[Osspial]

@MatusT WinRT compatibility is a feature that I'm not opposed to adding, and it's certainly something that Winit should, in the long run, aim to support! The thing is, implementing it correctly is a huge undertaking (in terms of work seems like it's essentially adding support for another operating system) and I think it's a niche enough target that I'm not willing to put in the work to learning how the API works and supporting it. If somebody is willing to make a PR for it and agrees to maintain it going forward, I don't think you'll find much objection to it getting merged. However, without that I'm not comfortable promising future support for it.

@vberger Could Wayland tooltips be implemented by a downstream library with what Winit currently exposes? That's starting to nudge into the realm of GUI stuff, and my gut instinct is to say that we shouldn't support it in this codebase.

[elinorbgr]

@vberger Could Wayland tooltips be implemented by a downstream library with what Winit currently exposes? That's starting to nudge into the realm of GUI stuff, and my gut instinct is to say that we shouldn't support it in this codebase.

No, winit does not expose enough to make tooltips yet. Though if we support making then in platforms where tooltips and just regular windows with a parent (I think I understood some platforms behave like this?), we may need to support them on wayland as well, for platform compatibility.

[Osspial]

@vberger Hmm. How much more of the Wayland internals would we need to expose to make it possible to implement tooltips externally?

[elinorbgr]

@Osspial

The "nice" way (for users) would imply exposing SCTK as a public dependency of winit.

The "conservative" way would be to just provide a raw pointer to the C shell surface, but it can be of 3 different possible types, so either the method would return an enum, or there would be 3 methods.

[Osspial]

@vberger I'd be most comfortable with exposing the raw pointer and wrapping it with an enum.

[elinorbgr]

This will require some work from sctk, which does not currently expose it. I'm not extremely thrilled about that to be honest, but I understand the tradeoff.

To make sure we're on the same page, you're proposing that everything about tooltips (creating them and drawing their contents) is out of scope for winit?

For example, stuff like "creating a second winit window and using a platform-dependent routine to make it into a tooltip" would not be possible on wayland this way. I don't know how creating tooltips work in other platforms, so I don't know if it may be something users might want to do.

[Osspial]

@vberger Yes, that's what I'm proposing. In general my feeling is that all platform-specific GUI and I/O work is out of Winit's scope, and we should focus exclusively on event loop management, renderable window creation, and input handling. If more specialized stuff can't be done with Winit's API that means that Winit isn't exposing enough of its internals.

Also, what part does sctk not expose? It looks like the shell surfaces are exposed through Proxys, which let you obtain a pointer to the underlying C type. If there's more work that has to be done I'd be happy to try and do some of it, as I have access to a machine that can run Wayland and the Rust side of things seems fairly understandable.

[elinorbgr]

Currently SCTK exposes the underlying wl_surface of a window, but not the "shell surface" object, which is and other object. It can be a wl_shell_surface, a zxdg_toplevel_v6 or a xdg_toplevel depending on the protocol (the two first are kind-of deprecated but kept in SCTK for compatibility, as the stabilization of the last on is still fairly recent). It'll require some small adaptation in SCTK to be able to expose it. And to avoid unnecessary overhead, winit might also want to expose the shell global it uses (which is already a field of the event loop).

[Osspial]

Oooooh, I was misinterpreting SCTK's shell global as the shell surface! That makes sense.

[est31]

Emscripten is tough to install, and furthermore there is a lot of attention on the wasm without emscripten ecosystem. I think that it certainly should be supported.

[Osspial]

@vberger Since we're going to be exposing those, I think it makes sense to expose the rest of SCTK's Wayland environment objects. If we do that, is it possible to cleanly support a Wayland clipboard outside of Winit? If it is (or if it only requires us to expose a little bit more to do so) I'd advocate for declaring clipboard support as out of scope and implementing it in a downstream library, since AFAIK Wayland is the only platform where clipboard support can't already be implemented downstream.

[elinorbgr]

@Osspial Well, exposing all the environment objects means exposing 7-8 objects in total. those can already be retrieved via the display (that is already exposed), just inducing some slight overhead.

As such, clipboard handling can and has been implemented outside of winit by @trimental, see alacritty/alacritty#1583.

The absolute minimum winit needs to expose is the display, the surface (both are already exposed), and the shell surface (for tooltips support mostly), I think.

Though, having the clipboard implemented outside of winit requires some nontrivial considerations which boils down to forcing the user to manage a second event loop (and thus having to manually advance it from the winit one, or spinning a new thread to run it). The implementation would overall be much simpler if it were in winit.

However, I agree that as clipboard support is possible outside of winit, it could be possible to keep it out of its scope, and reconsider the question later, once all the rest is completely consolidated. But this opens the question of drag'n'drop.

The machinery from copy/paste and drag'n'drop is mostly the same in wayland, so from my point of view it'd b strange to support one and not the other. But maybe the separation between them is more pronounced in other platforms?

[mitchmindtree]

Thanks so much for putting this together @Osspial! For the most part this looks great to me, the only thing I'm kind of unclear/concerned with is the concept of "child" windows, but I left a comment inline and it's probably purely because I haven't ran into a use-case for them myself yet.

[mitchmindtree]

Proper child window support is really tricky to get right outside of Winit, and requires platform-specific code to do properly. On Wayland it's impossible without platform calls, and Winit doesn't expose any way of converting client-space coordinates to screen-space coordinates. Additionally, disabling a window in favor of a child window is a common behavior for desktop applications (see file dialogs and are you sure you want to close without saving dialogs) so it makes sense to expose that here.

What it really means for a window to be a child window is still slightly unclear to me, maybe a description of what a child window can offer that another regular window cannot would be worth adding?

Reading the current entry, It seems to me that the relative positioning could be solved using the difference between get_position on two regular windows. Disabling a window in favour of another (e.g. disallowing user input for a popup) sounds to me like something could be handled by application logic (e.g. filter out input events associated with WindowIds that aren't popup_window_id).

I guess I just want to make sure we're not exposing something that could otherwise be done using application logic in order to reduce the winit surface area. I'm also a little concerned that the concept of a "child" window and what that means might be different per platform, though I haven't actually looked into this at all yet - it could very well have a very specific meaning across each platform.

All that said, I have read some users mention that child windows are simply essential in some use cases. One example that comes to mind is the VST API, though I don't know much about it.

[Osspial]

Reviving this now, to aid with the marketing push discussed in #830. I've done some minor status updates, but I don't see anything blocking this getting merged. I will do so on April 11th if no objections are raised.

[madsmtm]

To any interested parties: we're reopening the clipboard discussion of this again in #2156, feel free to drop by and give input!

[Laocoon7]

Is #584 on anyones radar? Interest has been expressed in #850 about a month ago, so I'm not the only person looking for this feature. I would be willing for it to work under the same conditions as glfwSetWindowAspectRatio(); specifically "If the window is full screen, the aspect ratio only takes effect once it is made windowed. If the window is not resizable, this function does nothing."

[John-Nagle]

I gather nothing was ever done about double-clicking.

I'd argue that should be in Winit's scope, because there's much platform-specific stuff involved in double-click recognition, especially where that interacts with "accessibility". Winit should provide the same double-click recognition rules as other programs on the same platform, using the current system settings. Otherwise, the Winit user has to have per-platform code, which is rarely needed for Winit. That's the whole point.

Game controllers, etc. probably don't need to be handled by Winit. They're usually handled through the "gilrs" crate, which can handle multiple joysticks, force feedback, and other devices way beyond Winit's department. That crate provides a consistent interface for HID devices across platforms.

[kchibisov]

Winit should provide the same double-click recognition rules as other programs on the same platform, using the current system settings

there's no such rules on X11 and it's all toolkit dependant. Getting that through e.g. gtk is not an option, because e.g. QT is different.

Maybe on macOS/Windows the threshold can be somehow exposed, but it's also, UI toolkit specific most of the time.

[John-Nagle]

MacOS and Windows have documented APIs for double click time. Working on figuring out where the Gnome "Settings" program and KDE settings programs for Linux store that info.

No idea how this works in browser WASM.

Egui uses this:

const MAX_DOUBLE_CLICK_DELAY: f64 = 0.3; // TODO(emilk): move to settings

More info to follow.

[kchibisov]

macOS and Windows are GUI toolkits, more or less. X11/Wayland are protocols, protocols don't have settings in general, and you can change settings on the fly.

I'm pretty sure gtk uses dbus, kde probably something else. The also hardcode it sometimes or change the value based on the UI element (I have a feeling that it's different on decorations vs text).

I remember that it's was 300, but I think it's like 400 now in gtk.

[John-Nagle]

I'm collecting data on where to get that value. Currently, I've found references for Windows, MacOS, and KDE. Working on finding out where Gnome Settings puts that info.

doubleclickapi.txt

[daxpedda]

Browsers have a dblclick event, so that should be fairly easy to implement.

It would probably also help to dig into Chrome's and Firefox's source code to see how they handle all the different backends.

@John-Nagle if you could open a new issue with your findings it would be greatly appreciated.