Gui in 3D issues

[BastiaanOlij]

Godot version

4.0.0.stable

System information

N/a

Issue description

I couldn't find the issue where this was previously discussed however after doing some testing and creating an MRP I figured a new issue that highlights just two of the issues specifically would help.

When we're using the "GUI in 3D" approach as demo'ed in https://github.com/godotengine/godot-demo-projects/tree/master/viewport/gui_in_3d we run into the problem that a number of controls now assume a subviewport is used in conjunction with a subviewport container and is always a normal 2D UI where coordinates within the subviewport map 1:1 to screen coordinates through simple translation.

This assumption is false.

The approach showcased in the "GUI in 3D" demo is prevalent among 3D games where UIs are shown on elements within the scene.

In the use case of XR this approach is exclusively used as this is the only method of displaying and interacting with a 2D UI. The input device is no longer the mouse connected to the PC (if there even is one) but often is implemented as a laser pointer attached to one of the users hands, where the laser pointer interacting with the 3D element results in mouse events being sent to the subviewport.

It is thus important that UI controls properly react to mouse events and the mouse coordinates presented within, instead of attempting to use system mouse coordinates transformed into local space as the latter may not exist.

Steps to reproduce

Run the attached MRP, this is a copy of the "GUI in 3D" example from our demo repository but with a larger display and two controls embedded that show the issue.

First we immediately see this warning in the debugger:

This warning indicates we're using the screen transform function from within a subviewport which isn't used in conjunction with a subviewport container. It is only presented the first time it's encountered but triggered from a bunch of controls that are attempting to obtain the mouse position.

In the UI the first failure, which is a minor one, is our tab bar. This contains logic to detect hovering over a tab to highlight it and show a preview. This does not work in the UI:

Then switch to the second tab, this has a color picker button on it. Pressing this button will overlay a color picker. This would normally close if you click away from it but this doesn't work.

There are other controls that exhibit similar issues. They can be easily replicated within the "Gui in 3D" demo project.

Minimal reproduction project

gui_in_3d.zip

[dhoverml]

EDIT: fixed in #77923 (tested with v4.1.master.custom_build [300748e]). This is still broken in Godot 3.5.

Adding to the list of issues, selecting text in a TextEdit selects everything from the caret to the end of the text. Placing the caret works okay.
This is reproducible in Godot 3.5 too.

godot4guiin3dselectionbug.mp4

[Sauermann]

#77923 didn't fix this completely, because the ColorPicker still doesn't close, when clicked away from it.

[Sauermann]

With #78472 I want to fix several of these Control-problems for Viewport-in-3D, that are related to mouse_get_position calls. I would appreciate feedback on that PR.