Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose Rust API for input method handling via the GameTextInput library #18

Closed
rib opened this issue Aug 21, 2022 · 6 comments
Closed

Expose Rust API for input method handling via the GameTextInput library #18

rib opened this issue Aug 21, 2022 · 6 comments

Comments

@rib
Copy link
Collaborator

rib commented Aug 21, 2022

For Android applications to be able to utilize soft keyboard UIs they need to be able to support the Android input methods protocol which the GameTextInput library helps with.

For reference this is the Winit IME API that was recently added that we should aim to be able to support: rust-windowing/winit#2243 (and the original tracking issue: rust-windowing/winit#1497)

Also see this Winit issue about supporting soft keyboards on Android: rust-windowing/winit#2305

And here's an initial summary of what I found when considering supporting Winit's IME API on Android: rust-windowing/winit#1497 (comment)
@rib rib mentioned this issue Aug 21, 2022
Closed
@Zoxc
Copy link
Contributor

Zoxc commented Aug 25, 2022

I've read up on things a bit now and it doesn't look like GameTextInput is compatible with Winit's IME model. It expects the application's text buffer state to be accessible to read and write while in Winit's IME model text flows from the IME into the application.

A temporary hack would be to drain GameTextInput's state for characters and generate ReceivedCharacter from these to get some basic support.

@rib
Copy link
Collaborator Author

rib commented Aug 25, 2022

aah, okey, yeah interesting point.

Winit's IME abstraction is quite new, and was initially just aimed at supporting traditional desktop IME interfaces so it's not that surprising that it's not a good fit - though I hadn't realised that aspect about needing to be able to read back text entry state.

I would guess that we would be able to make additions to the Winit IME abstraction - so maybe it's something that we should first just get working in a branch - building on the android-activity-0.27 backend branch and hopefully it's possible to figure out a minimal addition that could support Android - and then whatever we add can probably just be stubbed out / ignored for platforms that don't need it.

I am a bit curious about this though - I would have thought other IMEs would also want to be able to sync with the contents of a text entry - unless they maybe only support appending text. I wonder if other platforms could maybe benefit from a way to access application text buffer state but they maybe didn't strictly need it.

@rib
Copy link
Collaborator Author

rib commented Aug 25, 2022

For example with Wayland there is a set_surrounding_text protocol that can inform an IME about the existing contents of a text entry and that also ties into a related delete_surrounding_text event that the IME can send for being able to delete characters from the original text.

@rib
Copy link
Collaborator Author

rib commented Aug 25, 2022 •
edited
Loading

Also taking a look at the GTK API for IME handling they also have general support for sending surrounding text to an IME and also for an IME to be able to delete some of the surrounding text.

One initial proposal could just be to add a window API like set_ime_surrounding_text().

@rib
Copy link
Collaborator Author

rib commented Aug 25, 2022

Just linking another Winit issue that looks related to this: rust-windowing/winit#1823

rib added a commit that referenced this issue Sep 12, 2022
@rib
Expose input method APIs (GameActivity only)
b4248d6 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard.

This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state.

Fixes: #18
@rib rib mentioned this issue Sep 12, 2022
Merged
rib added a commit that referenced this issue Sep 12, 2022
@rib
Expose input method APIs (GameActivity only)
4ace584 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard.

This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state.

Fixes: #18
rib added a commit that referenced this issue Sep 12, 2022
@rib
Expose input method APIs
5a91699 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard. (supported for NativeActivity
and GameActivity)

This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state. (only supported with GameActivity)

Fixes: #18
rib added a commit that referenced this issue Sep 12, 2022
@rib
Expose input method APIs
513dc7d 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard. (supported for NativeActivity
and GameActivity)

This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state. (only supported with GameActivity)

Fixes: #18
rib added a commit that referenced this issue Sep 15, 2022
@rib
Add show/hide_soft_input methods
fd4b126 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard. (supported for NativeActivity
and GameActivity)

Addresses: #18
rib added a commit that referenced this issue Sep 15, 2022
@rib
native-activity: InputEvent forward compatibility
86c5dd3 
When building with native-activity then we no longer directly use
the `ndk::InputEvent` type and instead have our own extensible,
`#[non_exhaustive]` enum that ensures we will be able to add an
event for Ime state changes without necessarily needing a semver
bump.

Addresses: #18
@rib rib mentioned this issue Sep 15, 2022
Merged
rib added a commit that referenced this issue Sep 15, 2022
@rib
Add show/hide_soft_input methods
feff63a 
This adds `AndroidApp::show/hide_soft_input` APIs for showing or
hiding the user's on-screen keyboard. (supported for NativeActivity
and GameActivity)

Addresses: #18
rib added a commit that referenced this issue Sep 15, 2022
@rib
native-activity: InputEvent forward compatibility
8d30454 
When building with native-activity then we no longer directly use
the `ndk::InputEvent` type and instead have our own extensible,
`#[non_exhaustive]` enum that ensures we will be able to add an
event for Ime state changes without necessarily needing a semver
bump.

Addresses: #18
rib added a commit that referenced this issue Sep 15, 2022
@rib
Expose TextEvent and input method state
96df78d 
This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state. (only supported with GameActivity)

Fixes: #18
@rib
Copy link
Collaborator Author

rib commented Sep 15, 2022

Just for reference here; there has been progress in egui-winit with adding IME support - so hopefully if the issues with https://github.com/rib/android-activity/pull/24 can be resolved (and the winit android-activity backend updated to use this API) then we will hopefully be able to test with the latest egui-winit

ref: emilk/egui#2046

rib added a commit that referenced this issue Jul 30, 2023
@rib
Expose TextEvent and input method state
aa63494 
This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state. (only supported with GameActivity)

Fixes: #18
rib added a commit that referenced this issue Jul 31, 2023
@rib
Expose TextEvent and input method state
c3328a4 
This also adds `InputEvent::TextEvent` for notifying applications of IME
state changes as well as explicit getter/setter APIs for tracking IME
selection + compose region state. (only supported with GameActivity)

Fixes: #18
@rib rib closed this as completed in 41f30c3 Aug 1, 2023
@jb55 jb55 mentioned this issue Jul 22, 2024
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Zoxc @rib