Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.x] Editor Android Port #57747

Merged
merged 2 commits into from
Mar 28, 2022
Merged

[3.x] Editor Android Port #57747

merged 2 commits into from
Mar 28, 2022

Conversation

m4gr3d
Copy link
Contributor

@m4gr3d m4gr3d commented Feb 7, 2022

3.x version of the Godot Editor Android port based on the work done by @thebestnom in #36776!

This PR cleans a lot of the logic in #36776 and adapt the code for the 3.x branch. Given the stability of the 3.x branch, this should help accelerate testing and feedback on this implementation.

There are still a lot of tweaks, UI cleanup and touch-ups needed to be done before the Editor Android port can be considered suitable for public release and usage. Nonetheless, the current logic is stable enough and partitioned away that we can start reviewing and landing this PR and then iterate on top of it.

Addresses godotengine/godot-proposals#3931

godot_android_editor_on_device_run_through.mp4

Apk Download (only meant for testing and feedback!)

Latest 3.x Godot Editor Android build (all architectures)

Generate apk from sources

  • Using the regular Godot build process, generate Godot tools android shared libraries for each platform you want the apk to support:
    • e.g: scons tools=true platform=android target=<build_target> android_arch=<android_arch>
    • release_debug is recommended for the target option
    • android_arch should be one of arm64v8, armv7, x86 or x86_64
  • Navigate to the platform/android/java directory
  • Run ./gradlew generateGodotEditor
    • The command will generate one apk for each available target
    • The generated apk(s) can be found in the bin directory.

@m4gr3d m4gr3d added this to the 3.5 milestone Feb 7, 2022
@m4gr3d m4gr3d requested a review from a team February 7, 2022 10:47
@m4gr3d m4gr3d requested a review from a team as a code owner February 7, 2022 10:47
@ElhamAryanpur
Copy link

This is AWESOME!

@Calinou
Copy link
Member

Calinou commented Feb 8, 2022

I gave the APK a try on a Samsung Galaxy Z Fold2.

Screenshots

Screenshot_20220208-014212_Godot Editor (release-debug)

Screenshot_20220208-014321_Godot Editor (release-debug)

Screenshot_20220208-014342_Godot Editor (release-debug)

Screenshot_20220208-014434_Godot Editor (release-debug)

Screenshot_20220208-014516_Godot Editor (release-debug)

Screenshot_20220208-015457_Godot Editor (release-debug)

Screenshot_20220208-015558_Godot Editor (release-debug)

Screenshot_20220208-015613_Godot Editor (release-debug)

Some comments

  • A FileSystem access permission prompt is displayed on the first startup, but only for a short duration. If you don't press Allow in time, the project manager won't be able to create any folders (or display existing folders). In this case, you can kill and restart the app and the permission prompt will appear again.
  • Empty 3D editor runs at 120 FPS (locked to V-Sync). Godot can make use of high refresh rate displays 🙂
  • The device heats up a lot, even when the editor is idle. This is likely because low-processor mode isn't kicking in (Fix android low processor usage mode flicker 3.x #55604).
  • Buttons are quite small, but it's usable given the large screen size (on the Z Fold2's main display).
  • The splash screen doesn't appear when I open a project in the editor – I just get a black screen for a few seconds.
  • The "Save and Restart" prompt that appears when changing the language in the Editor Settings does not work. The editor will never restart after clicking this button. This can also apply to the project manager's language selection dropdown (the globe icon).
  • I think the default orientation should allow both landscape modes (standard and reverse), rather than being locked to a single landscape mode.
  • There could be an editor setting to allow using any orientation, including portrait mode. On tablets, portrait mode should still be wide enough to allow using the editor.
  • There could be an editor setting to make the editor use fullscreen mode to maximize the available screen real estate. In fact, we may want to make this the default given most mobile displays are quite small.
  • It's easy to make the editor (or project manager) overflow due to long translations, or when increasing the editor scale above the default (I tried 2.5 on the last 2 screenshots above). This isn't related to your PR, it's just something we need to improve on the editor side in general.

Touch UX improvements that could be implemented in the future

  • Dragging a finger on a scrollable area should scroll it. This needs to be implemented in ScrollContainer, Tree and CodeEdit separately.
  • Dragging a finger on the 2D/3D viewport should pan it. Pinching with two fingers should zoom. (What would be used to orbit in the 3D viewport?)
  • Double-tapping in the project list and FileDialogs should work as it does on PC.
  • Long-pressing a node in the Scene tree dock should pop up the context menu. This should also be done for LineEdit, TextEdit, etc.
  • The back button currently doesn't do anything when you are in the editor or project manager. We should probably make it do something, such as exiting to the project list (with the usual confirmation).
    • To improve productivity on the go, there could be an editor setting that allows customizing the back button action while in the editor:
      • Simulate an Undo press (Ctrl + Z), or Redo (Ctrl + Shift + Z) if held for 1 second or more.
      • Switch between the 2D/3D and Script tabs (depending on the last tab), to avoid having to tap a small icon at the top every so often.
  • Implementing Add an OLED-friendly dark theme preset to the editor godot-proposals#3368 could help improve battery life.
  • (Low priority) Freelook mode in the 3D editor:
    • Add a button on the 3D viewport to enter/exit freelook mode.
    • Implement twin-stick controls for freelook mode (with additional Up, Down and Speed buttons).
      • If we add gamepad support to the 3D editor freelook, we could effectively emulate gamepad controls on a touch screen.

Very impressive work overall – it should make for a much better editor-on-Android experience compared to the web editor.

@AaronRecord
Copy link
Contributor

Agree with everything Calinou said. First thing I did was change the editor scale to 3 and soft lock myself out of the editor (can't open any projects):
Screenshot_20220208-132200

@m4gr3d
Copy link
Contributor Author

m4gr3d commented Feb 8, 2022

@Calinou @LightningAA Thanks for the feedback!!

I'll create a set of issues to track the feedbacks so we can start addressing it.

@akien-mga Given the changes in this PR are partitioned away and should not affect the stable code/logic, we can start reviewing and merging it.
Then we'll address the reported feedback in separate PRs. These would prevent this one from growing too large.

@thebestnom
Copy link
Contributor

@Calinou @LightningAA Thanks for the feedback!!

I'll create a set of issues to track the feedbacks so we can start addressing it.

@akien-mga Given the changes in this PR are partitioned away and should not affect the stable code/logic, we can start reviewing and merging it.
Then we'll address the reported feedback in separate PRs. These would prevent this one from growing too large.

Also any changes you did here I would glad you add back (directly without PR) to 4.0 and merge after

@m4gr3d
Copy link
Contributor Author

m4gr3d commented Feb 9, 2022

Also any changes you did here I would glad you add back (directly without PR) to 4.0 and merge after

@thebestnom Sounds good. I'll push the changes for the master branch as well once this PR is reviewed and merged.

@ghost
Copy link

ghost commented Feb 9, 2022

Some of the issues Calinou raised could be fixed by

  • Disabling features from Editor->Manage Editor Features... so you have less components at the same time.
  • Using an editor theme specifically designed for Android that have larger and more square controls that are easier to click on
  • Also this PR Decrease margins around the editor and project manager windows #40287 could prove useful for an Android port
  • (Offtopic for this PR but deserves mention) Editor should remove all hardcoded minimum panel sizes to prevent overflows on smaller screens.
  • Also worth mentioning there are addons that allow touch gesture which could be ported to C++.

@fire-forge
Copy link
Contributor

This is great! To add on to what everyone else has said, I have a few suggestions for this:

  • Use different theme settings by default, with the editor scale increased for easier use on small screens and Additional Spacing increased to make buttons easier to touch.
  • Make the area behind the navigation bar use the editor theme color instead of solid black.
  • Add new editor settings for showing/hiding navigation bar and status bar (those with phones would want them disabled for more screen space, those with larger tablets might want them enabled).

@FancyTextLabel
Copy link

I love it, it's so lightweight even on my not so powerful phone, with a 720x1520 screen and 5.9" size, using 125% proportion is perfect!

Touching using fingers isn't much fun, so I fixed it with a touchpen. 😁

My suggestions are everything above, and a creation of a built-in keyboard!
Why? Android's built-in keyboard have so much useless stuff (Useless in Godot), occupy to much space and every time it's called it waste memory (150mb).

I only have a few crashes, and my phone doesn't heat much!

And I have a question, how can I delete a scene?

Screenshot_20220209-112348

@fire-forge
Copy link
Contributor

fire-forge commented Feb 9, 2022

I tried it out on my phone, and I was able to run the project manager, but after making a project and pressing edit I got this screen for a few seconds and then it crashed. Not sure if that's the fault of this app or if it's because my phone is a potato.

Screenshot_20220209-090712

My phone is a Moto g7 with Android 10.

Screenshot_20220209-091331

@Calinou
Copy link
Member

Calinou commented Feb 9, 2022

Use different theme settings by default, with the editor scale increased for easier use on small screens and Additional Spacing increased to make buttons easier to touch.

The best approach would be to rely on the OS-provided scaling factor. However, this OS-provided scaling would often be too large to be usable, so it would have to be clamped depending on the screen's lowest dimension (width or height) or divided by a constant factor.

My suggestions are everything above, and a creation of a built-in keyboard!
Why? Android's built-in keyboard have so much useless stuff (Useless in Godot), occupy to much space and every time it's called it waste memory (150mb).

Using a custom virtual keyboard is a bad idea for accessibility reasons. It will no longer follow the user's keyboard layout, theming options and various typing aids.
We should however figure out if suggestions can be hidden on the virtual keyboard by default, as they're often more trouble than they're worth when using the editor.

@ghost
Copy link

ghost commented Feb 9, 2022

An interesting approach would be for Godot to provide it's own basic built in keyboard that goes in the bottom panel since most of those panels are read only with exception for the Shader Script. Obviously it won't be as effective as the system touch keyboard but it would be an integrated alternative that embeds with the interface. There are some mobile addons that provide custom keyboard

@FancyTextLabel
Copy link

Obviously it won't be as effective as the system touch keyboard but it would be an integrated alternative that embeds with the interface.

A more compact, simple and independent keyboard, with a switch 🔄 to change to normal keyboard!

Half screen.
Screenshot_20220209-162753

@Calinou
Copy link
Member

Calinou commented Feb 9, 2022

Obviously it won't be as effective as the system touch keyboard but it would be an integrated alternative that embeds with the interface.

We can't resize the editor window to let the virtual keyboard display, as the screen is too small to do that. If the built-in keyboard was designed to be less tall, it would be pretty much unusable with touch input too.

PS: Most virtual keyboards offer settings to let you adjust their width, height and displayed elements.

@Deozaan
Copy link

Deozaan commented Feb 11, 2022

I just tried this on a 10 inch tablet. It was pretty amazing! I also got stuck indefinitely on a black screen when trying to open a GLES3 project, but GLES2 worked after a few seconds of black screen, which I now realize was probably the duration of the splash screen.

+1 to all of Calinou's feedback, except:

Dragging a finger on the 2D/3D viewport should pan it. Pinching with two fingers should zoom. (What would be used to orbit in the 3D viewport?)

My instinct was to drag a finger to "look around" (orbit?) in the 3D viewport. I suggest two-finger dragging for panning and pinching for zooming, as those two motions feel more closely related. It seems more intuitive to me. That said, I don't feel strongly about this and would be willing to do it Calinou's suggested way.

@m4gr3d
Copy link
Contributor Author

m4gr3d commented Feb 15, 2022

I've added a master (vulkan) version of the editor Android port in #58160 for those brave enough to test!

Note that the two versions can be installed side by side on the same device.

@akien-mga akien-mga merged commit 11d40d7 into godotengine:3.x Mar 28, 2022
@akien-mga
Copy link
Member

Thanks @thebestnom for an amazing port and @m4gr3d for the second pass!

@AaronRecord
Copy link
Contributor

Maybe after a few more of the issues are worked out, but what would it take to get this on the Google Play store (and possibly other app stores)?

@thebestnom
Copy link
Contributor

And thanks again for @fahrstuhl for first trying to do it 😄

@akien-mga
Copy link
Member

Maybe after a few more of the issues are worked out, but what would it take to get this on the Google Play store (and possibly other app stores)?

That's planned further down the line :)

@m4gr3d m4gr3d deleted the android_editor_3x branch March 28, 2022 21:08
@dugramen
Copy link

dugramen commented Mar 30, 2022

This is amazing! I have suggestion for touch use though. It's currently impossible to right click anything, I think holding down and releasing something should be considered a right click, as that's how it's usually done on mobile.

Also dragging should only occur if the Control was held for a short period before moving. Otherwise it's difficult to scroll.

Basically:
Hold down and release -> Right Click
Hold down and move -> Drag
Just release -> Left Click
Just move -> Scroll

@dugramen
Copy link

Another thing I noticed, which may not be related to this but to text fields on mobile in general. Usually if you tap the text field, a handle appears, and if you click it you get a menu for copy, paste etc. That's not here for now. Maybe there's a way to let Android itself handle the text field?

@Calinou
Copy link
Member

Calinou commented Mar 30, 2022

Maybe there's a way to let Android itself handle the text field?

This already happens with an invisible text field control, but we can't replace all LineEdits/TextEdits with native Android controls as they don't support all the features that LineEdit and TextEdit provide.

@dugramen
Copy link

Maybe there's a way to let Android itself handle the text field?

This already happens with an invisible text field control, but we can't replace all LineEdits/TextEdits with native Android controls as they don't support all the features that LineEdit and TextEdit provide.

I see. How about just for the script editor then, as I feel copy and paste is pretty important there (As well as precise cursor movement when dragging the space bar)

@Calinou
Copy link
Member

Calinou commented Mar 30, 2022

How about just for the script editor then, as I feel copy and paste is pretty important there (As well as precise cursor movement when dragging the space bar)

The script editor is literally the worst candidate for using a native Android control, since it's so advanced 🙂

The only way to implement this is to implement long-tapping selection in LineEdit and TextEdit. This is a lot of work to do correctly, and I'm not sure if it can be done at all. The same applies to dragging the spacebar, but I don't know how virtual keyboards actually translate this to the application. Do they send keyboard "left arrow" and "right arrow" events, or do something special? If this is something special, can C++ applications intercept it somehow?

@akien-mga
Copy link
Member

We'll be gathering feedback on the Android editor in this proposal: godotengine/godot-proposals#3931

Feel free to add your suggestions there. If you find actual bugs, you can of course open bug reports like usual.

@dugramen
Copy link

The script editor is literally the worst candidate for using a native Android control, since it's so advanced 🙂

The only way to implement this is to implement long-tapping selection in LineEdit and TextEdit. This is a lot of work to do correctly, and I'm not sure if it can be done at all. The same applies to dragging the spacebar, but I don't know how virtual keyboards actually translate this to the application. Do they send keyboard "left arrow" and "right arrow" events, or do something special? If this is something special, can C++ applications intercept it somehow?

Oh I assumed Android controls could be added on top of everything else, but I guess not 😅. I remembered seeing code editors in Android before, but after looking at how they do it, it seems like a lot of these things they had to implement themselves. But they WERE able to get rid of the suggestion bar at least

https://imgur.com/a/AdYDPVG
Top bar: Select all, cut, copy, paste
Bottom bar: Tab, undo, redo, search, left_key, right_key, save

@alex9310
Copy link

I just didn’t understand where can I get android sdk to build apk and aab?

@akien-mga
Copy link
Member

I just didn’t understand where can I get android sdk to build apk and aab?

See https://docs.godotengine.org/en/stable/development/compiling/compiling_for_android.html for documentation.

@Starkium
Copy link

this runs on an oculus quest for those who were wondering :)

I'd like to get a way to launch into vr mode from it or even start the editor in a vr mode. This is going to be crazy for vr development.

@alex9310
Copy link

I just didn’t understand where can I get android sdk to build apk and aab?

See https://docs.godotengine.org/en/stable/development/compiling/compiling_for_android.html for documentation.

as I understand it, I collect it on the pc as usual, did I understand correctly?

@2439905184
Copy link

2439905184 commented Apr 4, 2022

mouse right click doesn't work.
Device: Android8.0 huawei with bluetooth mouse and keyboard

@thebestnom
Copy link
Contributor

mouse right click doesn't work.\nDevice

In most stock rom Android recognize right click as back button in a non configureable way... Only fix is to replace rom 😅 (also middle click as home which is so frustrating)

@2439905184
Copy link

maybe just add button "del node" and then just click it to remove nodes in editor.
Because right click doesn't work .

@badduck32
Copy link

Editing scripts is very hard, I propose that there should be a way to edit scripts in portrait mode. I used an app to force portrait mode on the Godot editor, and it looks very good, apart from the left panel which I don't know how to turn off.
If this were full-screen I feel like it would be very usable. Maybe there should be a setting to make the entire editor portrait mode?

Screenshot_20220430-164842

The app looks and feels great in general, it hasn't crashed for me and it runs my project well, I'm excited for the future!!!

@Starkium
Copy link

Starkium commented Apr 30, 2022 via email

@Calinou
Copy link
Member

Calinou commented Apr 30, 2022

I used an app to force portrait mode on the Godot editor, and it looks very good, apart from the left panel which I don't know how to turn off.

You can enable distraction-free mode by clicking the "4 outward arrows" icon in the top-right corner of the 2D/3D/script tab:

image

If you have a hardware keyboard, you can also press Ctrl + Shift + F11 to toggle it.

Hiding the script list is done by clicking the leftwards arrow at the bottom of the script editor (it's obstructed by the virtual keyboard on your screenshot):

image

The Ctrl + \ keyboard shortcut can be used to toggle the script list from a hardware keyboard. (It's probably a good idea to change that shortcut, as it may not be triggerable depending on your keyboard layout.)

I agree it makes sense to toggle distraction-free mode automatically if the screen width isn't sufficient for editor usage with a dock on each side. This can probably be done on desktop as well.

@badduck32
Copy link

Hiding the script list is done by clicking the leftwards arrow at the bottom of the script editor (it's obstructed by the virtual keyboard on your screenshot):

That works like a charm! unfortunately you can't scroll using only touch input as of yet, and the scroll bar is off-screen when I force portrait mode, so there will have to be some slight changes but it's looking very promising!

@Pedrokkkkkk
Copy link

Hello, i made a plugin that helps using it on android, i think its not relevant to show here, btw the plugin handles touch inputs and converts into mouse inputs, a keyboard inside it is also included, link to the project at github there:

Godot Touch Acessibility Plugin

@Hanprogramer
Copy link

Screenshot_2022-05-16-00-00-13-788_org godotengine editor v3
I cant access some parts on the bottom of the screen probably because my phone has a camera notch on the screen (Redmi note 10 pro)

@Calinou
Copy link
Member

Calinou commented May 15, 2022

I cant access some parts on the bottom of the screen probably because my phone has a camera notch on the screen (Redmi note 10 pro)

This is not related to your device having notch. It's just that there isn't enough vertical space to fit the Animation bottom panel (which has a minimum height).

@Jake012345
Copy link

Hello, i made a plugin that helps using it on android, i think its not relevant to show here, btw the plugin handles touch inputs and converts into mouse inputs, a keyboard inside it is also included, link to the project at github there:

Godot Touch Acessibility Plugin

Is nice but how do i use it? 😬

@Jake012345
Copy link

Hello, i made a plugin that helps using it on android, i think its not relevant to show here, btw the plugin handles touch inputs and converts into mouse inputs, a keyboard inside it is also included, link to the project at github there:

Godot Touch Acessibility Plugin

Is nice but how do i use it? 😬

I see. Thats pretty helpful. Thanks for your work 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.