Naming problems: KeyboardInput Scancodes/Virtual Scancodes

[ickk]

Edit: RFC was created HERE

---

This is perhaps a pedantic issue to raise, but it is my opinion that it is important to get right early, before Bevy has enough users that a change will become very difficult.

This issue is not about the general solution of how the Bevy engine or Bevy users should handle scan codes or key symbols.

This issue is not solved by winit's upcoming API changes.

This issue concerns only a trivial but largely breaking change to correct Bevy's problematic naming of KeyboardInput field names.

Background

Some data is sent to the computer depending on which key we press on the keyboard. This data is normally called a scancode and gets its name from the fact that the keyboard usually scans an electrical matrix to figure out which switches are in contact (using a matrix is a cost saving measure to minimise the number of microcontroller pins required).

The value of the data for each key is relatively arbitrary and can vary from keyboard to keyboard. This is largely due to

• a lack of standards when computers were still new technology,
• a cost cutting measure (you can save diodes and a mess of electrical traces), and
• as a way for manufacturers of early computer systems to add an element of proprietary lock-in.

Modern keyboards have since largely standardised around a handful of scancode sets (mostly XT, AT, PC, and of course USB-HID ), but there is still quite a bit of variety out there.

This document published by Microsoft shows some of the different scancode sets available. Note the passage on page 29 regarding USB-keyboard scancodes (emphasis mine):

The closest equivalent key position should be used, so that a keyboard may be modified for a different language by simply printing different keycaps. One example is the Y key on a North American keyboard. In Germany, this is typically Z. Rather than changing the keyboard firmware to put the Z Usage into that place in the descriptor list, the vendor should use the Y Usage on both the North American and German keyboards. This continues to be the existing practice in the industry, in order to minimize the number of changes to the electronics to accommodate other languages.

That is to say USB-keyboards should normally emit US QWERTY scancodes regardless of the actual printed character, then the operating system can translate the actual scancode to a virtual scancode representing the intended character based on the user's language settings.

This advice is not always followed, as noted by cart in #2052 (comment)

Depending on the current context or application state, games and other software sometimes cares more about the physical position of keys than their semantic meaning, as when using the keyboard as non-textual controller i.e. WASD and Hotkeys. This is why Bevy needs provide both a way to get the "physical key location" as well as the "symbolic/logical key meaning".

Definitions

Definitions so the rest of this discussion can be understood without ambiguity.

scancode: The raw value of the key given by the operating system. Doesn't necessarily have a consistent semantic meaning, nor is it necessarily consistent layout-wise from platform-to-platform and keyboard-to-keyboard. Somewhat useful for physical key positions (though not totally reliable).

virtual-scancode: The semantic value of a key press. Translated from the scancode using the user's language settings by the operating system. Handles international keyboard layouts and non-english characters. Consistent as far as user expectations go when writing text.

Naming: How do different platforms refer to scancodes and virtual-scancodes?

• Windows
  Windows Developer: About Keyboard Input

  Assigned to each key on a keyboard is a unique value called a scan code, a device-dependent identifier for the key on the keyboard.

  The keyboard device driver interprets a scan code and translates (maps) it to a virtual-key code, a device-independent value defined by the system that identifies the purpose of a key.

  scancode is called "scan code" on Windows virtual-scancode is called "virtual-key code" on Windows

• Linux / X11 / Wayland / xmodmap / xkb
  Man: Xorg docs

  The X keyboard model is broken into two layers: server-specific codes (called keycodes) which represent the physical keys, and server-independent symbols (called keysyms) which represent the letters or words that appear on the keys.

  scancode is called "keycode" on X virtual-scancode is called "keysym" on X

  Note that since X uses this terminology, both xmodmap and xkb do too. Wayland normally uses xkb for keyboard input.

  evdev on linux also talks about "scancodes" and "keycodes" with different and more nuanced semantics still. Where as far as I understand, "scancodes" are the actual bits sent by the keyboard hardware, and "keycodes" are how the kernel should interpret those bits as far as some "standard keyboard" is concerned (but not necessarily with a logical/semantic key meaning, not internationalised), and then X11/xkb reinterprets those "keycodes" as "keysyms" (therefore in this context the user might interact with "keycodes" and "keysyms" but probably never directly with "scancodes").

• Web
  W3C: UI Events

  code holds a string that identifies the physical key being pressed.

  key holds a key attribute value corresponding to the key pressed.

  where a key attribute value is

  A key string that corresponds to the character typed by the user, taking into account the user’s current locale setting, modifier state, and any system-level keyboard mapping overrides that are in effect.

  scancode is called "KeyboardEvent.code" on Web virtual-scancode is called "KeyboardEvent.key" on Web

Naming: How do other game engines and UI toolkits refer to scancodes and virtual-scancodes?

• TK
  Tcler's Wiki: KeySyms on platforms other than X11

  uses X names,

  scancode is called "keycode" in TK virtual-scancode is called "keysym" in TK

• Qt
  Qt Documentation: QKeyEvent Class

  Provides the following methods on the QKeyEvent Class:
  nativeScanCode() and nativeVirtualKey()

  scancode is called "scan code" in Qt virtual-scancode is called "native virtual key" (or "key sym" in documentation) in Qt

• Unity
  Unity Documentation: KeyCode

  Key codes returned by Event.keyCode. These map directly to a physical key on the keyboard.

  Doesn't provide any kind of virtual-scancode as far as I can tell.

  Note: Unity is weird and maps keyCode to ASCII values.. where they can anyway.

  scancode is called "KeyCode" in Unity

• Godot
  Godot API: InputEventKey

  • int scancode
    The key scancode, which corresponds to one of the KeyList constants

  • int unicode
    The key Unicode identifier (when relevant). Unicode identifiers for the composite characters and complex scripts may not be available unless IME input mode is active.

  scancode is called "scancode" in Godot virtual-scancode closely maps to "unicode" in Godot

The Current Implementation

Currently bevy_input has a KeyboardInput type that exposes both a scan_code and key_code fields

pub struct KeyboardInput {
    pub scan_code: u32,
    pub key_code: Option<KeyCode>,
    pub state: ElementState,
}

and bevy_winit creates KeyboardInput instances via the convert_keyboard_input function

pub fn convert_keyboard_input(keyboard_input: &winit::event::KeyboardInput) -> KeyboardInput {
    KeyboardInput {
        scan_code: keyboard_input.scancode,
        state: convert_element_state(keyboard_input.state),
        key_code: keyboard_input.virtual_keycode.map(convert_virtual_key_code),
    }
}

Where

• the scan_code field contains the scancode which should be the raw value produced by the keyboard itself, and might provide a somewhat flakey way to get the physical key location, and
• the key_code field contains the virtual-scancode which should represent the symbolic key meaning.

The Problem

Bevy's name for the virtual-scancode is key_code/KeyCode which directly collides with the names that X11/Wayland(xmodmap, xkb) give to scancodes. i.e. Bevy has the opposite meaning compared to *nix platforms.

Bevy's name for virtual-scancode also collides with the name Unity uses for scancodes.

Bevy's names for scancode & virtual-scancode aren't really consistent with anything that I found.

This will cause a lot of confusion for users of Bevy on Linux and possibly users coming from Unity.

Potential Solutions

• At the very least Bevy should rename the key_code field on KeyboardInput and the KeyCode type to something else.

  Some suggestions are:

  • key_symbol/KeySymbol, keysym/KeySym to bring it in line with the naming from X11/Wayland.
  • virtual_keycode/VirtualKeyCode to bring it in line with the winit project as it currently exists.
  • logical_key/LogicalKey to bring it in line with winit's new/proposed API

• Bevy could rename both the scancode and virtual-scancode fields and types.

  Some suggestions are:

  • physical_key/PhysicalKey & logical_key/LogicalKey to bring it inline with winit's new/proposed API

Notes

I like the idea of renaming both the scancode and virtual_scancode fields and types to "physical"/"logical" (or "physical"/"symbol"), since it's arguably a lot more descriptive about the intended use, and helps to fall in line with winit's proposed new API.

However, naming the scan_code physical does not solve the issues outlined by cart in #2052 (comment) and so that naming choice could lead to some confusion.

Tangent regarding actual key positions
It may be possible to get the actual key position, irrespective of keyboard/platform/language by working backwards from the native virtual-scancode and using the user's keyboard-language configuration settings, locale, etc.

This can potentially even address non-standard keyboard layouts such as Dvorak and Colemak and might be quite reliable. This would allow Bevy to potentially expose all 3 of scancode, virtual-scancode, "actual-key-position".

TL;DR

Bevy's current naming of "scan code" & "key code" is consistent with nothing, and wrong compared to some platforms, UI toolkits & other game engines.