Mouse and Keyboard doesn't work on Mobile Engine (Samsung Dex)

[GuiRafaCL]

Godot version

4.1.1 PlayStore

System information

Samsung Galaxy S20 FE (Mobile Renderer)

Issue description

I installed Godot Engine 4.1.1, and in the Phone it works fine, but when I try to use in Dex mode with my mouse and keyboard, I can't event use WASD and Mouse to move in Editor. Also, there is some input errors on the Output.

Steps to reproduce

1. Open Godot Engine 4.1.1 on Android connected on Samsung Dex mode
2. Open a Project
3. Can't use mouse and keyboard

Minimal reproduction project

This looks like a generic error, isn't on a specific project.

[VecterraSoft]

Any updates ? Keyboard still does not work in DEX

[Calinou]

@VecterraSoft Does hardware keyboard/mouse work while using the internal display (i.e. when not using DeX)?

[Falor]

Since I'm having the same issue, I checked some things.
So the mouse works just fine for me, (I'm using a standard 3 button mouse and a standard cherry kc1000 keyboard)
The output windows says this when I'm typing a on my keyboard.

core/input/input.cpp:1039 - Index (int)p_button = -139 is out of bounds ((int)JoyButton::MAX = 128).

This happens regardless of the focus of my application. I can type in the 2D/3D view or in a textbox, the output window will show the error message.

I should also note that this is thrown twice. Once on key down and once on key up.
I have tested this behaviour with mobile, compability mode.
Switching modes also does not work with DeX but thats a separate issue in itself

Using it without dex works just fine.
I have not been able to confirm Bluetooth hardware though.
I can also confirm the S-Pen working just fine on a Samsung Galaxy Note S10+

I can also confirm that the issue is not present on a fresh install of version 4.1. The playstore version (is v4.2.stable.official [46dc277] atm) does contain the issue.

[GuiRafaCL]

@Falor I don't know if there is any relationship, but I noticed a difference between branch 4.1 and branch 4.2. In 4.1, in script input.cpp, the part of the code that checks if a button has been pressed (line 1007) is like this:

for (const KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed) {
			action_release(E.key);
		}
	}
}

However, in branch 4.2 (line 1022), an extra check was added:

	for (KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed > 0) {
			// Make sure the action is really released.
			E.value.pressed = 1;
			action_release(E.key);
		}
	}
}

I wonder if this is the change that caused the issue. I can't test it because I'm without Dex at the moment.

[Falor]

@Falor I don't know if there is any relationship, but I noticed a difference between branch 4.1 and branch 4.2. In 4.1, in script input.cpp, the part of the code that checks if a button has been pressed (line 1007) is like this:

for (const KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed) {
			action_release(E.key);
		}
	}
}

However, in branch 4.2 (line 1022), an extra check was added:

	for (KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed > 0) {
			// Make sure the action is really released.
			E.value.pressed = 1;
			action_release(E.key);
		}
	}
}

I wonder if this is the change that caused the issue. I can't test it because I'm without Dex at the moment.

I will look into it. The question I'd have for that, if this is actually the issue would be: Why would the keyboard input be negative or 0 when its coming through dex? That being said, I'll first need to be able to build Godot, then build it on Android to verify if that change is the cause for it. I'll post an update by the end of the week

[VecterraSoft]

@VecterraSoft Does hardware keyboard/mouse work while using the internal display (i.e. when not using DeX)?

Yes. I plug my keyboard into my computer and use its that way and works. Then I plug Dex and on Dex it works on every app except Godot, which is why I thought it might be a godot issue.
Edit: So just tried using dex on a laptop and even with the laptop keyboard godot does not register anything when I type. But all other apps do.

[GuiRafaCL]

I will look into it. The question I'd have for that, if this is actually the issue would be: Why would the keyboard input be negative or 0 when its coming through dex? That being said, I'll first need to be able to build Godot, then build it on Android to verify if that change is the cause for it. I'll post an update by the end of the week

This is really strange and I can't imagine why, but to me, apparently Godot can identify negative inputs, and, apparently, this line is causing Godot to not identify the Dex input. It remains a strange situation.

[Calinou]

-1 and -2 device IDs in Godot are specially handled, as they can refer to mouse-touch emulation. In the context of gamepad input actions, -1 means "all devices".

[VecterraSoft]

-1 and -2 device IDs in Godot are specially handled, as they can refer to mouse-touch emulation. In the context of gamepad input actions, -1 means "all devices".

Does this mean a fix can be had? Im not too versed in godot yet, just jumped ship from LibGDX after 9 years and only been using Godot for about a month and half. So not sure what that means.

[Calinou]

I wonder if this is the change that caused the issue. I can't test it because I'm without Dex at the moment.

@GuiRafaCL I can't find the code you're referring to in the master branch. Could you link to it (select a line number on GitHub's web viewer then press Y)?

[GuiRafaCL]

@Calinou For some reason, the 4.2 branch is different from the master branch. In the master branch, what I mentioned is found on line 1050 in the "godot/core/input/input.cpp" directory, and is different from what I mentioned above:

	for (KeyValue<StringName, Input::ActionState> &E : action_states) {
		if (E.value.cache.pressed) {
			action_release(E.key);
		}
	}
}

I don't know what to do about this.

[Calinou]

This was changed in #84943, which is only merged in master and not 4.2 by design.

cc @groud

[groud]

This was changed in #84943, which is only merged in master and not 4.2 by design.

If the issue was already there in 4.1.1, I doubt this changes introduced the issue. I can't really help for why it does not work, but I guess the error might come from this line:

void Input::joy_button(int p_device, JoyButton p_button, bool p_pressed) {
   ...
	ERR_FAIL_INDEX((int)p_button, (int)JoyButton::MAX);

I suspect there might be an issue with the DeX mode, where the keyboard is considered as a joypad and not a keyboard, something like that.

[Falor]

Alrighty, so I set up the project and can fully test just about everything. I'm currently on the 4.3 master branch. As expected, the keyboard does not work in that version either. 4.1, I can confirm does work with keyboard and mouse. As for testing out a revert of line 1050, there is no change in the keypress result.

@Falor I don't know if there is any relationship, but I noticed a difference between branch 4.1 and branch 4.2. In 4.1, in script input.cpp, the part of the code that checks if a button has been pressed (line 1007) is like this:

for (const KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed) {
			action_release(E.key);
		}
	}
}

However, in branch 4.2 (line 1022), an extra check was added:

	for (KeyValue<StringName, Input::Action> &E : action_state) {
		if (E.value.pressed > 0) {
			// Make sure the action is really released.
			E.value.pressed = 1;
			action_release(E.key);
		}
	}
}

I wonder if this is the change that caused the issue. I can't test it because I'm without Dex at the moment.

I am not well versed in Godot, nor fluent in C++. If you have any changes you want me to try out, I'll be able to make the changes and test them on a dex system.

[GuiRafaCL]

@Falor It's a shame it didn't work. Above your answer, @groud wrote his hypothesis.

void Input::joy_button(int p_device, JoyButton p_button, bool p_pressed) {
   ...
	ERR_FAIL_INDEX((int)p_button, (int)JoyButton::MAX);

I'm also not fluent in C++, so I don't know what possible change he was referring to. It's a generic error, and I have no idea :/. Anyway, thanks for building and testing what I thought.

[Calinou]

I suspect there might be an issue with the DeX mode, where the keyboard is considered as a joypad and not a keyboard, something like that.

This reminds me that in DeX mode, gamepad input will go to the last window that was touched on the device. You need to press the virtual touchpad once for input to go to DeX, as opposed to going on your local device. I don't know if this applies to keyboard and mouse input as well.

I have a Fold4 and therefore have access to DeX mode, and often run into this when trying to use a gamepad with it.

[Falor]

I suspect there might be an issue with the DeX mode, where the keyboard is considered as a joypad and not a keyboard, something like that.

This reminds me that in DeX mode, gamepad input will go to the last window that was touched on the device. You need to press the virtual touchpad once for input to go to DeX, as opposed to going on your local device. I don't know if this applies to keyboard and mouse input as well.

I have a Fold4 and therefore have access to DeX mode, and often run into this when trying to use a gamepad with it.

Well, even if it were the case, which I have not confirmed. The fact that the issue only came up with versions past 4.1 is the problem. Something has changed. I can run both 4.1 and 4.3, however, one has a working keyboard and the other does not. So even if this workaround were to work, this is not what this thread would be about.

Exaggerated example but:
If a "normal" user were to interact for the first time with the app via DeX, they'd not bother to look up how to fix things. Little Timmy's dad is gonna look into alternatives like Greenfoot for his kid. If we want Godot to succeed, workarounds will not help with that sadly. Well, that is, unless we make the workaround part of opening Godot somehow which I'd also consider a "fix". An "ugly" one from my own C# programming experience, but a fix nonetheless.

Back on topic, As deducted from OP and my testing, between version 4.1 and 4.1.1, something was changed about the way, keyboard input is handled. This broke the DeX Keyboard input.

[VecterraSoft]

Does that mean this fix is dead in the water?

[Falor]

I'm still open for any code change suggestions. But as I mentioned before: I'll need a code snippet to insert and the location. I'll test everything, but I can not investigate something in a language that I don't personally use, so please provide any suggestion, I'll test it out.

[GuiRafaCL]

Sorry, I have no idea what to do. I read the lines related to input and couldn't find any possible code problems for any solution :/

[ActuallyTheOwner]

I encountered an issue very similar with the 8bitdo Gbros adapter where if I press the heart button while using bluetooth mode, even on empty projects. No controller is needed to trigger the issue. I needed 32 bit support for intercepting the GBA trade sequence that the GBA uses with the GameCube....

core/input/input.cpp:1039 - Index (int)p_button = -180602665 is out of bounds ((int)JoyButton::MAX = 128).