[Winink] Check pointer button states.

[bruvzg]

Should fix #93951 (apparently driver for my ancient Hunion table is no longer working on current version of Windows 11, so won't be able to test it until Friday).

[alvinhochun]

Tested on Windows 10 with both Wacom table and Surface Pro 2017 with pen, this doesn't fix the issue.

The exact behaviour depends on whether the "Allow my pen to act as mouse in legacy applications" option is enabled in Windows Settings. I tested with the following code:

func _unhandled_input(event):
	if event is InputEventScreenTouch:
		prints("touch", event)
	elif event is InputEventScreenDrag:
		prints("touch", event)
	elif event is InputEventMouse:
		prints("mouse", event)

If "Allow my pen to act as mouse in legacy applications" is off, this is the output (trimmed):

mouse InputEventMouseMotion: button_mask=0, position=((62.33334, 137.6667)), relative=((6, -7.333333)), velocity=((3.304485, 85.91661)), pressure=0.00, tilt=((0, 0)), pen_inverted=(false)
mouse InputEventMouseButton: button_index=1, mods=none, pressed=true, canceled=false, position=((65.33334, 133.6667)), button_mask=1, double_click=false
touch InputEventScreenTouch: index=552, pressed=true, canceled=false, position=((65.33334, 133.6667)), double_tap=false
mouse InputEventMouseMotion: button_mask=1 (Left Mouse Button), position=((68, 129.6667)), relative=((5.666667, -8)), velocity=((172.2283, -295.7128)), pressure=0.49, tilt=((0, 0)), pen_inverted=(false)
[...]
mouse InputEventMouseMotion: button_mask=1 (Left Mouse Button), position=((88.66667, 128.3333)), relative=((0.333333, -11.33333)), velocity=((-345.8853, 191.5232)), pressure=0.15, tilt=((0, 0)), pen_inverted=(false)
mouse InputEventMouseButton: button_index=1, mods=none, pressed=false, canceled=false, position=((88.66667, 128.3333)), button_mask=0, double_click=false
touch InputEventScreenTouch: index=552, pressed=false, canceled=false, position=((88.66667, 128.3333)), double_tap=false
mouse InputEventMouseMotion: button_mask=0, position=((95.66667, 121.6667)), relative=((7, -6.666667)), velocity=((-345.8853, 191.5232)), pressure=0.00, tilt=((0, 0)), pen_inverted=(false)

The above shows Godot receiving WM_TOUCH through DefWindowProc from not handling WM_POINTERDOWN and WM_POINTERUP.

In the editor, this does make the tree nodes draggable, but code cannot be selected in the script editor.

---

If "Allow my pen to act as mouse in legacy applications" is on, this is the output (trimmed):

mouse InputEventMouseMotion: button_mask=0, position=((225.3333, 77.66667)), relative=((0, -2.666667)), velocity=((-57.81016, -23.12406)), pressure=0.00, tilt=((0, 0)), pen_inverted=(false)
mouse InputEventMouseMotion: button_mask=1 (Left Mouse Button), position=((224.6667, 74)), relative=((-0.666667, -3.666667)), velocity=((-57.81016, -23.12406)), pressure=0.30, tilt=((0, 0)), pen_inverted=(false)
[...]
mouse InputEventMouseMotion: button_mask=1 (Left Mouse Button), position=((210.3333, 101.3333)), relative=((-5, -0.666667)), velocity=((227.9962, -37.51836)), pressure=0.33, tilt=((0, 0)), pen_inverted=(false)
mouse InputEventMouseMotion: button_mask=0, position=((201, 96.66667)), relative=((-9.333334, -4.666667)), velocity=((227.9962, -37.51836)), pressure=0.00, tilt=((0, 0)), pen_inverted=(false)
mouse InputEventMouseButton: button_index=1, mods=none, pressed=true, canceled=false, position=((225, 76.33334)), button_mask=1, double_click=false
mouse InputEventMouseButton: button_index=1, mods=none, pressed=false, canceled=false, position=((225, 76.33334)), button_mask=0, double_click=false
mouse InputEventMouseMotion: button_mask=0, position=((194, 93.66667)), relative=((-7, -3)), velocity=((227.9962, -37.51836)), pressure=0.00, tilt=((0, 0)), pen_inverted=(false)

The above shows the button press event being delayed until the pen was released. This might be the doing of DefWindowProc but this doesn't seem to make any sense to me.
Edit: Just checked with Spy++, apparently the window does not receive WM_LBUTTONDOWN and WM_LBUTTONUP at all? This is weird.
Edit again: Only sometimes does the window not receive WM_LBUTTONDOWN and WM_LBUTTONUP.

In the editor, dragging doesn't work (obviously).

(I don't mean to say that Godot should respect the "Allow my pen to act as mouse in legacy applications" setting -- I think Godot should handle pen events consistently, that is to not let DefWindowProc decide what to synthesize. In Krita I resorted to calling SetCursorPos to move the cursor (because some of our code relies on GetCursorPos) and calling SendInput with a certain dwExtraInfo to send mouse down/up events.)

• Related: Pen Tablet: can't click on menu items #91205

[bruvzg]

Added WM_POINTERDOWN and WM_POINTERUP handlers and tested with Wacom tablet, it should work now.

[alvinhochun]

Thanks for the update, but sadly there are still a few issues when testing with the GHA build:

• Double-clicking doesn't seem to work with pen. On the Project Manager I cannot double click on a project item to open it.
  • The same thing happens when using touchscreen, though this can be considered a separate issue.
• The pen cannot be used to select text in the script editor. This may be a problem specific to the script editor though, because regular line edits seems to work fine, as well as the output message box.
  • It seems to me it may be using the system cursor position? For example if I move the mouse pointer to line 3, then try to select line 6 and below with the pen, the caret jumps to line 3 and doesn't move from there.

Dragging nodes and tabs now work with pen, so at least that's part of the issue solved. (Though apparently it doesn't work with touchscreen input.)

[bruvzg]

It seems to me it may be using the system cursor position?

Probably, but why it would be an issue, system cursor is following pen input (or is there some mode where it is not)?

[bruvzg]

Though apparently it doesn't work with touchscreen input.

Touchscreen should generate InputEventScreenDrag instead of mouse events, so it probably should be fixed in canvas editor (likely not handling these events, or using only to pan).

[alvinhochun]

It seems to me it may be using the system cursor position?

Probably, but why it would be an issue, system cursor is following pen input (or is there some mode where it is not)?

The default behaviour under Windows Ink is to allow each pointer to be operated separately. By default Windows does not move the cursor which represents the mouse (the one you get from GetCursorPos()). You can move the mouse to some position, use the pen or touchscreen to do stuff on another area, then move the mouse again and find the cursor at it's original position. I can observe this with Firefox, as well as most UWP apps.

There are cases when Windows does move the cursor, when an application does not handle WM_POINTER and/or WM_TOUCH messages. DefWindowsProc will generate mouse event messages as well as physically move the mouse cursor to the primary pointer for you. This can be observed in Notepad, for example.

At least that's what I gathered 7 years ago. I don't think much has changed now but I may be wrong.

[bruvzg]

The default behaviour under Windows Ink is to allow each pointer to be operated separately. By default Windows does not move the cursor which represents the mouse (the one you get from GetCursorPos()). You can move the mouse to some position, use the pen or touchscreen to do stuff on another area, then move the mouse again and find the cursor at it's original position. I can observe this with Firefox, as well as most UWP apps.

This seems to be the case, but everything in Godot assumes there's a single pointer position for all devices (ant this is how it's handled on other OSes).

[bruvzg]

Added double click simulation. For the rest, we'll probably need to support multiple pointer devices across the engine (input events have device property, but most of the UI will ignore it if it's not set to DEVICE_ID_INTERNAL).

[alvinhochun]

Okay, double click is fixed.

Can we change DisplayServerWindows::mouse_get_position to return the primary pen (or touch point) position that we got if there is a pen move happening, instead of calling GetCursorPos? There is also just one other call of GetCursorPos in DisplayServerWindows::_update_real_mouse_position that would need the same treatment. (The wintab one can probably be left alone.)

[akien-mga]

Thanks!