Get BoundingBox of letters in StyledTextArea

[kushal256]

Is it possible to get BoundingBox of letters within a StyledTextArea? Looks like this would require access to package private related classes like StyledTextAreaView and VirtualFlow? The implementation might be very similar to StyledTextAreaView::getSelectionBoundsOnScreen(), but not really sure?

My end goal was to build graphical connections (like arrows) between two words that each live in a different StyledTextArea's. Is something like this possible?

[TomasMikula]

You are correct, it would require some reflection hacks and the implementation would be similar to getSelectionBoundsOnScreen().

This is another example of a feature that is hard to support in the current unfortunate Control/Skin architecture, where the Control (i.e. the text area) is supposed to be the model in the MVC terminology and Skin is the view. Only Control (i.e. model) API is exposed to the user, so there is no direct access to the Skin. I started a discussion about this on the JavaFX mailing list, but nothing really has come out of it.

My plan is to switch to not using skins in RichTextFX at all whenever I find the time to do the refactoring.

Even then, though, it seems to me that what you would want is more like getting an ObservableValue<BoundingBox>, i.e. something that updates as the text area is edited or scrolled. That would be a whole other story (though still having the prerequisite of getting rid of the unfortunate skin architecture).

[JordanMartinez]

This is another issue that you should be able to implement now that the skin has been removed in 3f00de3.

[kushal256]

Awesome, thanks, will take a look!

[kushal256]

This definitely helps!

As Tomas noted, there are still parts that are a whole other story to get this done. I hacked away and made some changes and it partially works. If there is any interest, I can submit via a pull request (would love to get feedback and suggestions for improvement!), but here's the gist of what I hacked away.

Added to my subclass of CodeArea:
private void setupCoordMapping() {
// //////////////
// 1. Collect GUI components we need to listen to changes for
// Worked in 0.6.10, need to adjust
// Tied to VirtualFlow ctor last line: getChildren().addAll(content, hbar, vbar);
// final ScrollBar hbar = (ScrollBar) getVirtualFlow().getChildrenUnmodifiable().get(1);
// final ScrollBar vbar = (ScrollBar) getVirtualFlow().getChildrenUnmodifiable().get(2);

    // //////////////
    // 2. Setup streams to listen to
    EventStream<?> stream = EventStreams.merge(
            // EventStreams.changesOf(hbar.valueProperty()), // scrolling horizontally
            // EventStreams.changesOf(vbar.valueProperty()), // scrolling vertically
            EventStreams.changesOf(getVirtualFlow().visibleCells()).thenIgnoreFor(Duration.ofDays(99)),  // Done for initialization
            plainTextChanges()); // Text editing

    // final Parent fxInternalWindow = getParent().getParent();
    // if (fxInternalWindow != null) {
    // stream = EventStreams.merge(stream, EventStreams.changesOf(fxInternalWindow.layoutXProperty()), // moving of window
    // EventStreams.changesOf(fxInternalWindow.layoutYProperty())); // moving of window
    // }
    stream = EventStreams.merge(stream);
    stream.successionEnds(Duration.ofMillis(50))
            .supplyTask(this::buildCoordMappingAsync)
            .awaitLatest(stream)
            .map(Try::get)
            .subscribe(this::doneBuildingCoords);  // API requires either subscribe() or similar call to stop observing this stream.
}

private final ExecutorService executor = Executors.newSingleThreadExecutor();

private Task<TextBoundsDS> buildCoordMappingAsync() {
    final Task<TextBoundsDS> task = new Task<TextBoundsDS>() {
        @Override
        protected TextBoundsDS call() throws Exception {
            return buildCoordMapping();
        }
    };
    this.executor.execute(task);
    return task;
}


private TextBoundsDS buildCoordMapping() {
    final Instant start = Instant.now();

    final TextBoundsDS textBoundsDS = new TextBoundsDS();
    for (final Cell<Paragraph<Collection<String>, Collection<String>>, ParagraphBox<Collection<String>, Collection<String>>> cell : getVirtualFlow()
            .visibleCells()) {
        final ParagraphBox<Collection<String>, Collection<String>> pBox = cell.getNode();
        final ParagraphText<Collection<String>, Collection<String>> pText =
                (ParagraphText<Collection<String>, Collection<String>>) pBox.getChildrenUnmodifiable().get(0); // Only know this from looking at ctor of ParagraphText, first child added. Dirty way to cheat ensapculation, wish there was a getter!
        final int lineCount = pBox.getIndex() + 1;
        // final Bounds pBoxBounds = pBox.getBoundsInParent();
        // System.out.println(" pBox.getGraphicPrefWidth = " + pBox.getGraphicPrefWidth());
        final ObservableList<javafx.scene.Node> nodeList = pText.getChildren();
        int colCount = 0;
        for (final javafx.scene.Node node : nodeList) {
            if (node instanceof Text) { // might be a Shape from either
                // selection or cursor, so only
                // pickup Text classes
                final Text textNode = (Text) node;

                final Bounds curBounds = textNode.localToScene(textNode.getLayoutBounds());
                final String text = textNode.getText();

                textBoundsDS.add(lineCount,
                        new TextBound(text,
                                new BoundingBox(curBounds.getMinX(), // +
                                        // pBox.getGraphicPrefWidth(),
                                        curBounds.getMinY(),
                                        curBounds.getWidth(),
                                        curBounds.getHeight()),
                                colCount));

                colCount += text.length();
            }
        }
        // lineCount++;
    }
    this.textBoundsProperty.set(textBoundsDS);        // This will notify listeners each time, as we're always new object.
    System.out.println("buildCoordMapping took " + Duration.between(start, Instant.now())
            + " to build coords");
    return textBoundsDS;
}

[JordanMartinez]

I've only glanced through this, but I will look through it more later on.

[JordanMartinez]

Ok, I see what you're doing now and why that would be useful.

[kushal256]

Cool, will fork and checkin maybe tomorrow.

[kushal256]

OK checked into here https://github.com/kushal256/RichTextFX

If there is any interested I can submit this as a pull request.

I'm open to any suggestions/cleanups of the code.

[JordanMartinez]

Can you also write a small demo that demonstrates your feature?

[kushal256]

Yes, will do soon, need to cleanup my code.

FYI the reason I'm trying to get a BoundingBox is so I can draw my own components on top of specific words, and these components can receive keyboard/mouse events. This way I can create custom GUI widgets that interconnect two separate chunks of text.

[JordanMartinez]

Yes, will do soon, need to cleanup my code.

Cool.

FYI the reason I'm trying to get a BoundingBox is so I can draw my own components on top of specific words, and these components can receive keyboard/mouse events. This way I can create custom GUI widgets that interconnect two separate chunks of text.

Yeah, I remember you saying something like that beforehand:

My end goal was to build graphical connections (like arrows) between two words that each live in a different StyledTextArea's.

I thought it'd be interesting to see what that would look like in a program. 😄

[kushal256]

I thought it'd be interesting to see what that would look like in a program. 😄

Same here ;) If I had something complete enough to show, it would be public by now. Besides the bugs and working out kinks, I'm still trying to figure out what kinds of custom components to draw. My current ones need polish, and look really ugly when there are too many on the screen simultaneously.

Will try to push out a proof-of-concept soon.

[JordanMartinez]

@kushal256 Any updates on this?

[JordanMartinez]

Closed by #455.