Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve keyboard input with Input<ScanCode> #2052

Closed
TheRawMeatball opened this issue Apr 29, 2021 · 17 comments
Closed

Improve keyboard input with Input<ScanCode> #2052

TheRawMeatball opened this issue Apr 29, 2021 · 17 comments
Labels
A-Input Player input via keyboard, mouse, gamepad, and more C-Feature A new feature, making something new possible

Comments

@TheRawMeatball
Copy link
Member

What problem does this solve or what need does it fill?

Right now, getting the status of raw keys is difficult. An Input<ScanCode> would make this simpler.

What solution would you like?

Add a Input<ScanCode> resource by default in bevy_input, and change the ScanCode type to be an enum where the keys are represented as the key they correspond to on a standard QWERTY keyboard.

What alternative(s) have you considered?

Do nothing, or do this partially.

Additional context

Started discussion in discord

@TheRawMeatball TheRawMeatball added C-Feature A new feature, making something new possible D-Trivial Nice and easy! A great choice to get started with Bevy A-Input Player input via keyboard, mouse, gamepad, and more labels Apr 29, 2021
@inodentry
Copy link
Contributor

inodentry commented Apr 29, 2021

Key Codes are layout-dependent. Scan Codes are layout-independent. I think we should improve our keyboard APIs to encourage good practices:

  • Scan Codes should be used for game input.
  • Key Codes should be used for text input (and maybe display).

So I would suggest the following:

  • Create a new ScanCode enum, that is similar to the current KeyCode enum.
    • It should contain enum variants for every key commonly found on typical keyboards.
    • Alphanumeric keys could be named based on QWERTY (as that is what most people are familiar with).
    • It should be trivially convertable to/from an integer.
    • An Other variant could be included for extra keys on weird keyboards, when not covered by any enum variant.
  • Change the KeyCode enum to be better suited for text input and more indicative of its layout-centric nature.
    • Keep the variants for for non-alphanumeric keys like Esc, Delete, Return, ...
    • Replace all alphanumeric keys with a single Char(char) variant, containing the unicode character value, as per the keyboard layout of the user.
      • This should make it trivial to implement text input.
      • It would make it easy for users to support non-latin scripts for text input.
      • It would also encourage people to use ScanCode for game inputs, instead of KeyCode.
  • Both Input<KeyCode> and Input<ScanCode> resources should be available.
  • Replace the scan_code field in the KeyboardInput event type with a ScanCode (it is currently a bare u32).

These changes should result in a much cleaner keyboard input API that encourages good practices and is easy to use correctly.

@cart
Copy link
Member

cart commented Apr 29, 2021

To me this isn't necessarily clear cut. Game input generally cares about the "shape" of the keyboard (ex: WASD movement or the position of the arrow keys). ScanCode is closer to the ground truth of the "shape". But it also isnt the ground truth because we live in a world with different physical keyboard layouts.

By adopting ScanCode as our standard for game inputs, we are essentially saying that non-qwerty-scancode-aligned keyboard owners need to remap all of the defaults a game provides (or have the game provide alternative mappings for each layout they want to support). This feels pretty english-centric to me.

Using ScanCode for input will likely result in better outcomes, but it isn't a complete solution:

Semi-related Bevy conversation: #782

@mrec
Copy link

mrec commented Apr 29, 2021

As part of this, would Bevy need a portable mechanism to translate a ScanCode to a KeyCode for the user's keyboard and locale? Otherwise I don't know how a game's instructions are supposed to tell the user what keys to press.

EDIT: hmm, this is the same use case as #1995 which is now closed, but as far as I can see #805 only added a ReceivedCharacter event for text input. Maybe I'm missing something, but I don't see how this helps describe a key which the user isn't pressing.

@inodentry
Copy link
Contributor

inodentry commented Apr 29, 2021

By adopting ScanCode as our standard for game inputs, we are essentially saying that non-qwerty-scancode-aligned keyboard owners need to remap all of the defaults a game provides (or have the game provide alternative mappings for each layout they want to support). This feels pretty english-centric to me.

Hmm ... what? This seems backwards to me? What is a "non-qwerty-scancode-aligned keyboard"? I thought the scan codes represent the physical keys and are the same on every keyboard?

If you are referring to my suggestion about naming the enum variants after QWERTY, I can see the point, but those are just internal enum values. I suggest using the matching KeyCode for text display to users / as part of UI.

Game inputs should use scan codes, precisely so that they are not "english-centric". That way they will work regardless of whether i switch my keyboard to QWERTY, Dvorak, or Russian. The current KeyCode-based API is english-centric, because it makes it inconvenient to use non-qwerty (or worse, non-latin) layouts.

But it also isnt the ground truth because we live in a world with different physical keyboard layouts.

Of course some people have special nonstandard keyboards (like the various ergonomic keyboards with weird physical arrangement of keys), but those people are going to have to customize/remap their game inputs anyway.

@cart
Copy link
Member

cart commented Apr 30, 2021

Hmm ... what? This seems backwards to me? What is a "non-qwerty-scancode-aligned keyboard"? I thought the scan codes represent the physical keys and are the same on every keyboard?

This is what I was contesting. I understand that scan code is the raw event sent from the keyboard and doesn't directly correlate to a letter (this is the OS's job, which is what produces KeyCode and is virtual-keyboard-layout dependent). I have only ever used "qwerty" keyboards, but I am currently under the impression that international keyboards exist (such as this azerty keyboard) with scan codes that don't line up with the physical positions of qwerty keyboard scancodes. This is confounded by the fact that other azerty keyboards apparently do use qwerty scan codes (and rely on the OS to configure an azerty layout).

I was under the impression that the W3C KeyboardEvent.code spec was designed to combat this by providing abstract non-scancode values. But maybe they also assume scancode positions are constant and they just 1:1 map scancode to new names.

Of course some people have special nonstandard keyboards (like the various ergonomic keyboards with weird physical arrangement of keys), but those people are going to have to customize/remap their game inputs anyway.

If these are truly "non-standard" in terms of "an overwhelming majority of the world uses the same scan code positions", then I'm totally cool with using scancodes because we can't support every fringe thing that exists. I just want to make sure we aren't alienating some significant portion of the world's population.

I'm definitely not an expert here, but I'd like some way to quantify the actual situation. Something like "windows / mac / linux all assume physical scancode positions are stable as defined by some standard X and only 0.1% of the population uses standard Y".

edit: adjusted qwerty scancode link to something that uses hex values for parity with the other example

@Tsudico
Copy link

Tsudico commented May 2, 2021

The USB HID spec for keyboards is based off the physical layout of a QWERTY keyboard so I think any enumeration should use the spec's defined values where applicable. [See Keyboard / Keypad page (0x07) on pdf page 82]:
https://usb.org/document-library/hid-usage-tables-122

@mockersf
Copy link
Member

mockersf commented May 2, 2021

Fun news! on macOS, the scan codes provided by winit don't seem to be the real scan codes, but the virtual key codes macOS is using... https://developer.apple.com/documentation/coregraphics/cgkeycode

my "q" key have a scan code of... 0

@inodentry
Copy link
Contributor

Ugh, why does macOS always ... uhm ... "think different"

@mockersf
Copy link
Member

mockersf commented May 2, 2021

I tried running Bevy with rust-windowing/winit#1890 (macOS impl for rust-windowing/winit#753) and events are much better for keyboard 👍

example event:

KeyEvent {
    physical_key: KeyA,
    logical_key: Character(
        "q",
    ),
    text: Some(
        "q",
    ),
    location: Standard,
    state: Pressed,
    repeat: false,
    platform_specific: KeyEventExtra {
        text_with_all_modifiers: Some(
            "q",
        ),
        key_without_modifiers: Character(
            "q",
        ),
    },
}

It seems done for Windows (rust-windowing/winit#1788), in review for macOS (rust-windowing/winit#1890), draft for web (rust-windowing/winit#1888) and under work for Linux.

Unless we are ready to implement something temporary, I think we should wait for this to be done on winit side.

@mockersf mockersf removed the D-Trivial Nice and easy! A great choice to get started with Bevy label May 2, 2021
@cart
Copy link
Member

cart commented May 3, 2021

Wow that api is so much better. And yeah the scancodes for mac (even in the new impl) are completely out of sync with what I expect them to be. Sounds like the "physical_key" api is the way to go.

Relevant bit from the api redesign issue:
image

Which is basically what the W3C decided to roll with for their new keyboard api for the web. Unfortunately this is the sort of thing that could drag on for a long time (iirc winit's eventloop 2.0 started midway through 2018 and winit 20.0 dropped at the start of 2020). We should try to help them if possible.

Bevy contributors motivated to make input better (faster) should consider reaching out in this issue: rust-windowing/winit#1806

@maroider
Copy link

maroider commented May 5, 2021

Unfortunately this is the sort of thing that could drag on for a long time (iirc winit's eventloop 2.0 started midway through 2018 and winit 20.0 dropped at the start of 2020).

Heh, rust-windowing/winit#753 has been open for well over two years now, so I hope we don't end up spending too much time getting this across the finish line.

Any help on this matter would be appreciated. The Linux impl should have a PR up later this week (hopefully), so what would be most helpful is help with code review, preferably from someone familiar with X11 and/or Wayland, as both these platforms (effectively) lack maintainers. I'm sure @ArturKovacs would also appreciate another pair of eyes (and more testing) on the macOS impl.

@mockersf
Copy link
Member

mockersf commented May 7, 2021

and now rust-windowing/winit#1932 is up for anyone on Linux to test 🎉

@janhohenheim
Copy link
Member

Are there any updates on this?

@mlp1802
Copy link

mlp1802 commented Apr 10, 2022

I'm not sure I understand this discussion fully, I too also need a way to show the user which keys he has selected, in a controls panel. I did this simple thing. Is this bad?

pub fn create_keycode_map() -> HashMap<KeyCode, String> {
    let mut map = HashMap::default();
    map.insert(KeyCode::A, "A");
    map.insert(KeyCode::B, "B");
    map.insert(KeyCode::C, "C");
    map.insert(KeyCode::D, "D");
    map.insert(KeyCode::E, "E");
    map.insert(KeyCode::F, "F");
    map.insert(KeyCode::G, "G");
    map.insert(KeyCode::H, "H");
    map.insert(KeyCode::I, "I");
    map.insert(KeyCode::J, "J");
    map.insert(KeyCode::K, "K");
    map.insert(KeyCode::L, "L");
    map.insert(KeyCode::M, "M");
    map.insert(KeyCode::N, "N");
    map.insert(KeyCode::O, "O");
    map.insert(KeyCode::P, "P");
    map.insert(KeyCode::Q, "Q");
    map.insert(KeyCode::R, "R");
    map.insert(KeyCode::S, "S");
    map.insert(KeyCode::T, "T");
    map.insert(KeyCode::U, "Y");
    map.insert(KeyCode::V, "V");
    map.insert(KeyCode::X, "X");
    map.insert(KeyCode::Y, "Y");
    map.insert(KeyCode::Z, "X");
    map.insert(KeyCode::Key1, "1");
    map.insert(KeyCode::Key2, "2");
    map.insert(KeyCode::Key3, "3");
    map.insert(KeyCode::Key4, "4");
    map.insert(KeyCode::Key5, "5");
    map.insert(KeyCode::Key6, "6");
    map.insert(KeyCode::Key7, "7");
    map.insert(KeyCode::Key8, "8");
    map.insert(KeyCode::Key9, "9");
    map.insert(KeyCode::Numpad0, "Numpad0");
    map.insert(KeyCode::Numpad1, "Numpad1");
    map.insert(KeyCode::Numpad2, "Numpad2");
    map.insert(KeyCode::Numpad3, "Numpad3");
    map.insert(KeyCode::Numpad4, "Numpad4");
    map.insert(KeyCode::Numpad5, "Numpad5");
    map.insert(KeyCode::Numpad6, "Numpad6");
    map.insert(KeyCode::Numpad7, "Numpad7");
    map.insert(KeyCode::Numpad8, "Numpad8");
    map.insert(KeyCode::Numpad9, "Numpad9");
    map.insert(KeyCode::Home, "Home");
    map.insert(KeyCode::Delete, "Delete");
    map.insert(KeyCode::End, "End");
    map.insert(KeyCode::PageDown, "PgDown");
    map.insert(KeyCode::PageUp, "PgUp");
    map
}

@inodentry
Copy link
Contributor

This issue is not about that. This issue is about being able to handle keyboard input in a way that does not break for users with multiple keyboard layouts set in the OS.

For instance, I have a Workman (previously Dvorak), QWERTY (because i need it to test stuff that assumes everyone is using qwerty, like Bevy currently), and a Cyrillic layout (for Bulgarian/Ukrainian/Russian). French users use AZERTY. Etc...

Currently, Bevy does not work properly, because key codes come from the currently active layout. If a game uses WASD for movement, and I play the game, I have to switch to qwerty, or the keys will be in the wrong places (physically). If I have a keybinding (like Alt-Shift or something) for switching layouts, there is a risk I might accidentally press it while in game, causing the keyboard input to get messed up in the middle of gameplay.

The solution to this are "scan codes". Scan codes identify the physical key on the keyboard, regardless of the software layout set in the OS.

In Bevy, scan codes are only available when using keyboard events, not when using the Input abstraction, which makes them annoying to use. This could be fixed by providing a Input<ScanCode> type.

Further, scan codes are only represented as opaque integers, not a friendly enum like KeyCode. Due to the platform differences in the exact values of the scan codes, there isn't much Bevy can do about this, without the assistance of winit.

@Ryder17z
Copy link

For reference, on Scandinavian keyboards, the keys painted black here are not detected currently:
image

bors bot pushed a commit that referenced this issue Aug 5, 2022
# Objective

- I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that

## Solution

- I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>`
- closes both #2052 and #862

Co-authored-by: Bleb1k <[email protected]>
inodentry pushed a commit to IyesGames/bevy that referenced this issue Aug 8, 2022
# Objective

- I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that

## Solution

- I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>`
- closes both bevyengine#2052 and bevyengine#862

Co-authored-by: Bleb1k <[email protected]>
maccesch pushed a commit to Synphonyte/bevy that referenced this issue Sep 28, 2022
# Objective

- I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that

## Solution

- I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>`
- closes both bevyengine#2052 and bevyengine#862

Co-authored-by: Bleb1k <[email protected]>
james7132 pushed a commit to james7132/bevy that referenced this issue Oct 28, 2022
# Objective

- I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that

## Solution

- I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>`
- closes both bevyengine#2052 and bevyengine#862

Co-authored-by: Bleb1k <[email protected]>
@james7132
Copy link
Member

james7132 commented Dec 4, 2022

This was added in #5495. Closing.

ItsDoot pushed a commit to ItsDoot/bevy that referenced this issue Feb 1, 2023
# Objective

- I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that

## Solution

- I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>`
- closes both bevyengine#2052 and bevyengine#862

Co-authored-by: Bleb1k <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Input Player input via keyboard, mouse, gamepad, and more C-Feature A new feature, making something new possible
Projects
None yet
Development

No branches or pull requests