Re-add IME support #762

tomcur · 2024-11-28T12:36:15Z

This adds IME support back into Masonry. This sticks close to linebender/parley#111, except that during IME compose, this version doesn't allow changing the selection anchor, making the code simpler. For reference, there's also linebender/parley#136.

This tweaks the focus update pass: when a widget with IME is unfocused, Masonry sends the platform's IME disable event to the newly focused widget. As a workaround, we synthesize an IME disable event in the focus pass and send it to the widget that is about to be unfocused. A complication is that the handling of that event can request focus to a different widget, and in particular, can request itself to be focused again. This handles that case, too.

Remaining work is setting the IME candidate region to be near the current selection and to make a decision on cursor/selection hiding when the platform sends a None cursor.

masonry/src/text/editor.rs

masonry/src/widget/text_area.rs

masonry/src/text/editor.rs

I believe the cursor should still land at index `self.buffer.len()` (logically following the last cluster). E.g., in linebender/xilem#762, if used without this change, the selection can't span the last cluster using `cursor_at`.

Another regression from #754 ... Needed for #762 --------- Co-authored-by: Tom Churchman <[email protected]>

tomcur · 2024-11-29T10:24:43Z

masonry/src/widget/text_area.rs

+                            if !self.editor.is_composing() && self.editor.selected_text().is_some()
+                            {
+                                // The IME has started composing. Delete the current selection and
+                                // send a TextChange event with the selection removed, but without
+                                // the composing preedit text.
+                                self.editor.transact(fctx, lctx, |txn| {
+                                    txn.delete_selection();
+                                });
+                                submit_text = Some(self.text().to_string());
+                            }


Note PlainEditorTxn::set_compose also removes the selection from the buffer when the IME newly starts composing (i.e., the first compose after or after a clear_compose). That seemed like a useful abstraction for Parley. The Masonry code here as a special-case also removes the selection, because Masonry might not want to send preedit text in TextChange actions, but does want to notify about the removed selection.

DJMcNab

This works really well for me. It does show that underlines for an emoji is in the wrong place, but that's not the fault of this PR.

Some minor points to be considered. I don't think there's anything blocking, but we should also check-in with Matt.

DJMcNab · 2024-11-29T10:26:51Z

masonry/src/text/editor.rs

+            ));
+        } else {
+            // IME indicates nothing is to be selected: collapse the selection to a
+            // caret just in front of the preedit.


I'd normally expect the cursor to be at the end of the region in this circumstance. Is this working on some prior art?

My platform IME can get into a state where it's sending None, in which case this is quite ugly:

typing "body" gives me:

It was more a less an arbitrary choice I lifted from linebender/parley#111, but on my platform, Firefox, Chrome and LibreOffice Writer all place the caret in front of the preedit text (and they appear to never make a spanning selection).

Firefox:

Hmm. Is that hitting this condition where the IME doesn't provide a preedit cursor?

Performing the same IME actions on Winit as on Firefox, provides a preedit cursor that spans the entire preedit text.

(I haven't figured out steps yet with this IME that sends a cursor: None.)

I've done a little bit more research into this, and it seems like Winit gets into a borked state. Any choice seems reasonable here. Alternatively, maybe we should just not have a visible cursor if we're asked to display none?

To clarify, I have confirmed that my IME is still sending cursor positions, but winit is sending us None. I don't know why

It's all slightly messy, with different implementations making different choices. Hiding the cursor completely is what Winit suggests, so perhaps hiding the caret is the best option.

Maybe we should then also call accesskit::node::clear_text_selection, but I'm not sure if that messes with something.

DJMcNab · 2024-11-29T10:32:05Z

masonry/src/text/editor.rs

+            self.update_layout();
+
+            self.editor
+                .set_selection(self.editor.cursor_at(preedit_range.start).into());


This is unfortunate in the commit case, because we do a layout here to be able to set the selection, then moments later we change the text, forcing another layout. Practically this is probably fine, but it would be nice to think about whether it can be avoided.

PlainEditor::layout_dirty gets us part of the way there, but as you mentioned, we can't set the next selection until the layout is updated (it is forced to fit within the current layout). It might work if we have some (private) PlainEditor::defer_set_selection or something similar, that simply waits with setting the selection until inside the update_layout method.

masonry/src/widget/text_area.rs

DJMcNab · 2024-11-29T11:04:17Z

masonry/src/widget/text_area.rs

    }

    fn on_access_event(&mut self, ctx: &mut EventCtx, event: &AccessEvent) {
        if event.action == accesskit::Action::SetTextSelection {
+            if self.editor.is_composing() {
+                return;
+            }


We should check with @mwcampbell if this is right. The scenario is:

The user is inputting some text which uses a pre-edit through their IME

Through their accessibility tool, they have tried to select some text, without their IME finishing the input

What should we do in that case? This code just ignores the second request, which is the easiest method to reason about. The alternatives are quite messy.

masonry/src/widget/text_area.rs

Co-authored-by: Daniel McNab <[email protected]>

masonry/src/passes/update.rs

DJMcNab

A potential simplification of the new pass code.

Either applying this or not, I think we can still land this. The only accessibilty integration problem might occur if there is a pre-edit, and not having IME is a much bigger accessibility fail. I'm unlikely to be able to review again before Monday (at least), but I'm happy for it to land as-is (presuming there aren't any massive new changes, of course)

masonry/src/passes/update.rs

Co-authored-by: Daniel McNab <[email protected]>

tomcur · 2024-11-30T11:49:07Z

I'm going to defer the decision on cursor hiding to a different PR, as it may hit on AccessKit accessibility, too.

tomcur commented Nov 28, 2024

View reviewed changes

masonry/src/text/editor.rs Outdated Show resolved Hide resolved

tomcur commented Nov 28, 2024

View reviewed changes

masonry/src/widget/text_area.rs Outdated Show resolved Hide resolved

tomcur commented Nov 28, 2024

View reviewed changes

masonry/src/text/editor.rs Outdated Show resolved Hide resolved

DJMcNab mentioned this pull request Nov 28, 2024

Restore underline and strikethrough in render_text #763

Merged

tomcur mentioned this pull request Nov 28, 2024

Fix off-by-one error in PlainEditor::cursor_at linebender/parley#187

Merged

github-merge-queue bot pushed a commit that referenced this pull request Nov 28, 2024

Restore underline and strikethrough in render_text (#763)

d981f0d

Another regression from #754 ... Needed for #762 --------- Co-authored-by: Tom Churchman <[email protected]>

tomcur force-pushed the new-ime branch from b0b63e9 to b5c3bde Compare November 28, 2024 16:40

tomcur added 5 commits November 29, 2024 10:55

Initial IME support

f44b8dd

Remove dbg!()

03aa829

Debug-assert not composing

9bb6fb3

Fix off-by-one (selecting last probably should have upstream affinity)

260c523

Underline works again

5c811c6

tomcur force-pushed the new-ime branch from b5c3bde to 5c811c6 Compare November 29, 2024 09:56

tomcur marked this pull request as ready for review November 29, 2024 10:03

tomcur commented Nov 29, 2024

View reviewed changes

DJMcNab approved these changes Nov 29, 2024

View reviewed changes

tomcur and others added 3 commits November 29, 2024 14:16

Update masonry/src/widget/text_area.rs

c2adbcf

Co-authored-by: Daniel McNab <[email protected]>

Synthesize Ime::Disabled in focus update pass

98ce9e4

Import consistency

6a5abfc

DJMcNab reviewed Nov 29, 2024

View reviewed changes

masonry/src/passes/update.rs Outdated Show resolved Hide resolved

Attempt 2 at handling of event handling effects inside focus update

7bdc3b1

DJMcNab reviewed Nov 29, 2024

View reviewed changes

masonry/src/passes/update.rs Show resolved Hide resolved

Simplify IME handling in focus pass

a8b1c24

DJMcNab approved these changes Nov 29, 2024

View reviewed changes

masonry/src/passes/update.rs Show resolved Hide resolved

tomcur and others added 3 commits November 30, 2024 12:17

Keep messy logic local

4737b24

Co-authored-by: Daniel McNab <[email protected]>

Clean up and docs

28ccc00

Move interactivity check below IME handling

6ebd0d5

Check for widget interactivity again

c126f20

tomcur added this pull request to the merge queue Nov 30, 2024

Merged via the queue into linebender:main with commit e9edd38 Nov 30, 2024
17 checks passed

tomcur deleted the new-ime branch November 30, 2024 12:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re-add IME support #762

Re-add IME support #762

tomcur commented Nov 28, 2024 •

edited

Loading

tomcur Nov 29, 2024

DJMcNab left a comment

DJMcNab Nov 29, 2024

DJMcNab Nov 29, 2024

tomcur Nov 29, 2024

DJMcNab Nov 29, 2024

tomcur Nov 29, 2024

tomcur Nov 29, 2024

DJMcNab Nov 29, 2024 •

edited

Loading

tomcur Nov 29, 2024 •

edited

Loading

DJMcNab Nov 29, 2024

tomcur Nov 29, 2024 •

edited

Loading

DJMcNab Nov 29, 2024

DJMcNab left a comment

tomcur commented Nov 30, 2024

Re-add IME support #762

Re-add IME support #762

Conversation

tomcur commented Nov 28, 2024 • edited Loading

Choose a reason for hiding this comment

DJMcNab left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DJMcNab Nov 29, 2024 • edited Loading

Choose a reason for hiding this comment

tomcur Nov 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomcur Nov 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DJMcNab left a comment

Choose a reason for hiding this comment

tomcur commented Nov 30, 2024

tomcur commented Nov 28, 2024 •

edited

Loading

DJMcNab Nov 29, 2024 •

edited

Loading

tomcur Nov 29, 2024 •

edited

Loading

tomcur Nov 29, 2024 •

edited

Loading