Add true multitouch trackpad support.

[wusticality]

What problem does this solve or what need does it fill?

At present, trackpad events are captured as if the trackpad were a mouse. For example, on a MacBook Pro, events come in as a left mouse press / release event. The MacBook Pro trackpad is actually a multitouch display, however. Bevy should emit touches as if it were any other multitouch display, along with press / release events.

What solution would you like?

After talking to @alice-i-cecile, it seems the solution is to have Bevy emit touches (as if it were a multitouch display) and emit the mouse behavior if someone wants to opt into that default behavior. An example of what I'd like to accomplish using the trackpad is:

• Pan the camera by dragging two fingers on the trackpad.
• Zoom in and out by detecting pinch to zoom gestures.
• Other multitouch gestures as things come up.

One potential issue is that the MacBook Pro trackpad touch events should support press / release states when your finger touches and releases from the screen, and also trackpad click events as the trackpad itself can be fully depressed. Both types of touches should be supported which might be an issue with the current Touches implementation:

pub struct Touches {
    /// A collection of every [`Touch`] that is currently being pressed.
    pressed: HashMap<u64, Touch>,
    /// A collection of every [`Touch`] that just got pressed.
    just_pressed: HashMap<u64, Touch>,
    /// A collection of every [`Touch`] that just got released.
    just_released: HashMap<u64, Touch>,
    /// A collection of every [`Touch`] that just got cancelled.
    just_cancelled: HashMap<u64, Touch>,
}

It may be that Bevy should have a Trackpad input abstraction for this case that supports both touch events and button-click events in one interface but I'm not sure. There may even be trackpads that support multitouch but also have separate physical buttons.

What alternative(s) have you considered?

For now, I'm getting around this by using keyboard shortcuts to zoom the world and holding a modifier key while dragging my finger on the trackpad for drag events which is not ideal.

[bjorn3]

I am pretty sure that macOS itself does the translation to mouse events. There is probably an api for accessing gestures that don't map to a regular mouse and one for raw input. Adding support for gestures makes sense. I don't think we should treat it as a touch screen without the game explicitly requesting this though, as it will break any action where the user expects it to behave as a trackpad rather than a touch screen unless the game itself adds support for interpreting it, which is likely not going to happen for most games that don't target phones or tablets.

[mockersf]

Complete support for that needs to be done in winit: rust-windowing/winit#2379. It's not really possible in Bevy without winit support.

Next version of winit will expose TouchpadMagnify and TouchpadRotate events (rust-windowing/winit#2157). 2 fingers dragging already works and is detected as mouse wheel.

[wusticality]

Yeah, I wasn't sure if Bevy should directly support gestures as they're totally up to the user, but I guess there are some common ones (pinch to zoom, drag with two fingers, etc.).

[wusticality]

Complete support for that needs to be done in winit: rust-windowing/winit#2379. It's not really possible in Bevy without winit support.

Next version of winit will expose TouchpadMagnify and TouchpadRotate events (rust-windowing/winit#2157). 2 fingers dragging already works and is detected as mouse wheel.

Huh, I don't get any MouseWheel events when dragging with two fingers, at least on my MacBook Pro (or are you saying in winit, not Bevy)?

[mockersf]

I get them with Bevy on my Mac, with example mouse_input_events

[xgbwei]

Huh, I don't get any MouseWheel events when dragging with two fingers, at least on my MacBook Pro (or are you saying in winit, not Bevy)?

Also works fine on my MBP. I don't see anything in system preferences that might break it, so this seems like it could be caused by another app?

[james7132]

Marking this as blocked until winit supports this.

[megaboy101]

Complete support for that needs to be done in winit: rust-windowing/winit#2379. It's not really possible in Bevy without winit support.

Next version of winit will expose TouchpadMagnify and TouchpadRotate events (rust-windowing/winit#2157). 2 fingers dragging already works and is detected as mouse wheel.

Hey, just stumbled upon this issue and it looks like winit now supports TouchpadMagnify and TouchpadRotate as of v0.28.0?

I'd be down to make a PR to support this in Bevy, although I'm not very familiar with the engine's internal's atm (I only started playing around with Bevy a few weeks ago 😅)

[alice-i-cecile]

I believe those two events were added in #8791 :)

[megaboy101]

Ah wonderful, thank you @alice-i-cecile !

[simbleau]

@james7132 @alice-i-cecile This is no longer blocked

[JeanMertz]

@james7132 @alice-i-cecile This is no longer blocked

Is it not? I was looking for a way to support two-finger drag, but Winit's features page still mentions multitouch as not implemented for macOS.

[simbleau]

It seems like they maybe just forgot to update the feature matrix?
rust-windowing/winit#2157

But it may actually still be blocked -
rust-windowing/winit#2379 is tracking true "multitouch" support -

Discussion here. It seems there's no one driving the feature on winit anymore, but PRs are welcome. Call to action!