[rcore] Draft: IME support #2809

daipom · 2022-11-25T09:07:17Z

This is a fix for issue #1945.
We are mainly assuming Japanese input, but this feature can be used for other languages using IME as well.

We are implementing IME support features in GLFW now:

Add IME support for each platform glfw/glfw#2130

This fix adds the corresponding APIs to raylib.

It may take a while to be merged in GLFW, so we have created this PR as a draft ahead of time.

APIs to be added

void SetPreeditCallback(PreeditCallback callback)
- PreeditCallback: void (*PreeditCallback)(int preeditLength, int *preeditString, int blockCount, int *blockSizes, int focusedBlock, int caret)
- Preedit version of CharCallback.
- An app gets preedit info (preedit text, blocks, and caret position) by this callback and draws preedit.
- The reason why we have an app register the callback directly, which is a different from CharCallback, is that this is not like CharCallback, where the input is resolved for each character, but rather requires managing the entire current preedit info at all times. Therefore, it is inefficient for an app to ask for preedit info every frame, so we use a callback to notify the change of preedit info.
void SetPreeditCursorRectangle(int x, int y, int w, int h)
- Set the area of the preedit cursor. This is used to decide the candidate window position that displays a list of conversion candidates. (For example, converting hiragana into kanji in Japanese).
- In general, the position of this window should be managed by an app side by using this API.
void GetPreeditCursorRectangle(int *x, int *y, int *w, int *h)
- The getter version of SetPreeditCursorRectangle.
void ResetPreedit(void)
- Clear the current preedit.
bool IsImeOn(void)
- Get if the IME is ON.
void SetImeStatus(bool on)
- Set IME ON/OFF.

Feature to take candidate list (Win32 ONLY)

For a detailed explanation, please see below.

void SetPreeditCandidateCallback(PreeditCandidateCallback callback)
- PreeditCandidateCallback: void (*PreeditCandidateCallback)(int candidatesCount, int selectedIndex, int pageStart, int pageSize)
- The app takes the current candidate status by this callback.
- To get the candidate text for each index, use GetPreeditCandidate.
int *GetPreeditCandidate(int index, int *textCount)
- Get the current candiadate text of the specified index.

Specifications for each platform

Details are in the GLFW PR glfw/glfw#2130.

Win32

All APIs work fine.

macOS

All APIs work fine.

X11

Use over-the-spot style, so an app side doesn't need to draw preedit.
- There are 2 main styles: on-the-spot and over-the-spot.
- In on-the-spot style, an app side draws preedit, and this allows suitable drawing for each app. The other platforms use this style.
- In over-the-spot style, the IME side draws preedit. This reduces what an app has to do, but it does not allow flexible display.
- On X11, the behavior of on-the-spot style is not stable, so we use over-the-spot style.
Only SetPreeditWindowPosition and GetPreeditWindowPosition work, and other APIs don't work.
This is OK for this style because the IME side manages the drawing of preedit, so all an app side needs to do is managing the candidate window position.

Wayland

Currently, we support the minimum required APIs: SetPreeditCallback, SetPreeditWindowPosition, and GetPreeditWindowPosition.

Sample Application

We are implementing the sample application RaylibIMEInputSampleApp.

https://github.com/clear-code/RaylibIMEInputSampleApp

We can check all newly added APIs in this application.

Known issues

X11: Some keys (arrow, Enter, BackSpace, ...) are passed to an app during preediting.
- This will be fixed by Properly discard filtered events by XFilterEvent() glfw/glfw#1972

Additional planned fixes

Since the fix of GLFW is large, we will submit separate PRs to GLFW for the following features.
For this reason, we have also separated the fixes for raylib.

Support soft fullscreen mode for IME features.
- The candidate window doesn't be displayed correctly with fullscreen mode.
  - The default fullscreen is exclusive, and this causes the problem for IME displaying the candidate window.
- We already have this fix for GLFW.
  - https://github.com/clear-code/glfw/tree/im-support-fullscreen
- The corresponding fix of raylib is this.
  - https://github.com/clear-code/raylib/tree/better-ime-support-fullscreen
  - This will add FLAG_SOFT_FULLSCREEN config flag to make the fullscreen not exclusive.

Feature to take candidate list (Win32 ONLY)

Usually, the IME displays the candidate window, so the app side doesn't need to get canidate list.

However, sometimes, we need to display it on the app side.
For example, it could be for displaying it correctly on exclusive fullscreen, or for using a custom design.

This is especially common in games for Windows.

This feature is OFF by default.
We can use this by setting the FLAG_MANAGE_PREEDIT_CANDIDATE config flag.

Currently, this feature is available only on Win32, since this feature is especially common on Windows.
We'd like to make it available on every platform if possible, but it may be difficult because of platform specific issues.

This feature is also involved in the issue with fullscreen and IME.
If fullscreen is strongly exclusive, then the candidate window of IME can't be displayed correctly.
This problem is especially serious in Windows, so this feature is one workaround for fullscreen on Windows.

We also have another fix that reduces fullscreen exclusivity for the IME, but the fix of GLFW for Windows is tricky...
(Please see Additional planned fixes)

We can check this behavior in the sample application with MANAGE_PREEDIT_CANDIDATE cmake option.

https://github.com/clear-code/RaylibIMEInputSampleApp/wiki/CMake-Build-Options#manage_preedit_candidate

daipom · 2022-11-25T09:13:28Z

It has not yet been merged into GLFW, so the fails of the tests are as expected.

Please see the wiki of RaylibIMEInputSampleApp about how to check this feature.

https://github.com/clear-code/RaylibIMEInputSampleApp/wiki

raysan5 · 2022-11-28T21:38:43Z

@daipom Thank you very much for all the hard work on this new feature! I couldn't review it carefully yet, probably some function could be renamed but the example looks fantastic! Thanks! 😄

daipom · 2022-11-29T01:07:15Z

Thanks!
If you have any questions about this feature or see something to be fixed, please let me know!

daipom · 2022-11-29T01:20:23Z

I have just rebased this branch to the latest master.

daipom · 2022-11-29T01:34:30Z

@raysan5

In connection with #2814, many characters can be entered at one time when using an IME.
So the current MAX_CHAR_PRESSED_QUEUE value is small for input using IME.

raylib/src/rcore.c

Lines 335 to 337 in 50a716c

    
           #ifndef MAX_CHAR_PRESSED_QUEUE 
        
               #define MAX_CHAR_PRESSED_QUEUE        16        // Maximum number of characters in the char input queue 
        
           #endif

It would be great if this could be made larger, such as 64, or if it could be changed on the application side.
What would you think about this?

daipom · 2022-11-30T13:22:44Z

I have fixed some APIs: SetPreeditCursorRectangle, GetPreeditCursorRectangle.

daipom · 2022-12-15T15:51:09Z

I rebased this branch to the latest master and added the feature to take candidate list (Win32 ONLY).
I have updated the initial comment on this PR, so please see it about this feature.

If you would like this feature to be separated into another PR, please let me know.
As for this feature, it is summarized in the last commit and can be easily removed.

raysan5 · 2023-09-07T16:19:08Z

@daipom Hi! It's been quite long since this amazing improvement was proposed, what is the current state? Is there some workaround to allow this feature to be added in raylib?

Peter0x44 · 2023-10-16T00:06:28Z

This will definitely require some rebasing and modifications now the platform split is merged, to move the relevant code to rcore_desktop.c

DreepyYunky · 2023-11-25T11:19:07Z

I think It'll be better to change SetImeStatus() to TurnIMEOn() or InitIME().

raysan5 · 2023-12-15T18:04:48Z

I just saw there was some update on this PR. I'm keeping it open for a bit longer, it seems lately GLFW is receiving several updates...

ashie · 2024-03-11T03:56:42Z

Hi, I'm a collaborator of this work.
I've rebased this branch on top of current master (1fad827).
The previous one is backed up as https://github.com/clear-code/raylib/tree/4.2-2022-12-14%2Bbetter-ime-support
Since bundled GLFW isn't support IME yet, CI will fail as well as before.

I just saw there was some update on this PR. I'm keeping it open for a bit longer, it seems lately GLFW is receiving several updates...

IME support of GLFW isn't merged yet, it's planned as a part of v3.5: https://github.com/glfw/glfw/milestone/25
So we need to wait next minor update of GLFW (current is v3.4 released 2 weeks ago).

ashie · 2024-03-11T07:27:32Z

A commit is added automatically by CI:
ccf2ddf

I'm not sure we can leave it as is, or we should do something more or not.

This is for GLFW3 Preedit Callback: glfwSetPreeditCallback

This is for GLFW3: glfwSetPreeditCursorRectangle

This is for GLFW3: glfwGetPreeditCursorRectangle

This is for GLFW3: glfwGetInputMode (mode: GLFW_IME)

This is for GLFW3: glfwSetInputMode (mode: GLFW_IME)

This is for GLFW3: glfwResetPreeditText

Support the GLFW preedit candidate feature. This feature supports only Win32 currently. We can use this feature by enabling `FLAG_MANAGE_PREEDIT_CANDIDATE` flag on Win32.

raysan5 · 2024-08-19T13:46:37Z

@daipom @ashie Thanks for the update! Glad to see that this PR is still active! Very nice! 😄

daipom force-pushed the better-ime-support branch from 37eb6f7 to 88a0d77 Compare November 29, 2022 01:18

daipom force-pushed the better-ime-support branch from 88a0d77 to 574f985 Compare November 30, 2022 12:47

daipom force-pushed the better-ime-support branch from 574f985 to 3666805 Compare December 15, 2022 15:33

bjorndm mentioned this pull request Mar 1, 2023

The input method is located outside the app fyne-io/fyne#618

Open

raysan5 changed the title ~~Draft: IME support~~ [rcore] Draft: IME support Feb 4, 2024

ashie force-pushed the better-ime-support branch from 3666805 to 90cb37e Compare March 11, 2024 03:50

daipom and others added 8 commits August 19, 2024 16:43

Add API: SetPreeditCallback

71bb40a

This is for GLFW3 Preedit Callback: glfwSetPreeditCallback

Add API: SetPreeditCursorRectangle

b63d0eb

This is for GLFW3: glfwSetPreeditCursorRectangle

Add API: GetPreeditCursorRectangle

6c7e13e

This is for GLFW3: glfwGetPreeditCursorRectangle

Add API: IsImeOn

8d5d9cc

This is for GLFW3: glfwGetInputMode (mode: GLFW_IME)

Add API: SetImeStatus

d004c66

This is for GLFW3: glfwSetInputMode (mode: GLFW_IME)

Add API: ResetPreedit

c2286f2

This is for GLFW3: glfwResetPreeditText

Support preedit candidate

e56841f

Support the GLFW preedit candidate feature. This feature supports only Win32 currently. We can use this feature by enabling `FLAG_MANAGE_PREEDIT_CANDIDATE` flag on Win32.

rcore_desktop_rgfw: Add skeleton of IME functions

b44c05d

ashie force-pushed the better-ime-support branch from ccf2ddf to b44c05d Compare August 19, 2024 07:57

Update raylib_api.* by CI

99452d6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[rcore] Draft: IME support #2809

[rcore] Draft: IME support #2809

daipom commented Nov 25, 2022 •

edited

Loading

daipom commented Nov 25, 2022

raysan5 commented Nov 28, 2022 •

edited

Loading

daipom commented Nov 29, 2022

daipom commented Nov 29, 2022

daipom commented Nov 29, 2022 •

edited

Loading

daipom commented Nov 30, 2022

daipom commented Dec 15, 2022

raysan5 commented Sep 7, 2023

Peter0x44 commented Oct 16, 2023 •

edited

Loading

DreepyYunky commented Nov 25, 2023 •

edited

Loading

raysan5 commented Dec 15, 2023

ashie commented Mar 11, 2024

ashie commented Mar 11, 2024

raysan5 commented Aug 19, 2024

[rcore] Draft: IME support #2809

Are you sure you want to change the base?

[rcore] Draft: IME support #2809

Conversation

daipom commented Nov 25, 2022 • edited Loading

APIs to be added

Specifications for each platform

Sample Application

Known issues

Additional planned fixes

Feature to take candidate list (Win32 ONLY)

daipom commented Nov 25, 2022

raysan5 commented Nov 28, 2022 • edited Loading

daipom commented Nov 29, 2022

daipom commented Nov 29, 2022

daipom commented Nov 29, 2022 • edited Loading

daipom commented Nov 30, 2022

daipom commented Dec 15, 2022

raysan5 commented Sep 7, 2023

Peter0x44 commented Oct 16, 2023 • edited Loading

DreepyYunky commented Nov 25, 2023 • edited Loading

raysan5 commented Dec 15, 2023

ashie commented Mar 11, 2024

ashie commented Mar 11, 2024

raysan5 commented Aug 19, 2024

daipom commented Nov 25, 2022 •

edited

Loading

raysan5 commented Nov 28, 2022 •

edited

Loading

daipom commented Nov 29, 2022 •

edited

Loading

Peter0x44 commented Oct 16, 2023 •

edited

Loading

DreepyYunky commented Nov 25, 2023 •

edited

Loading