proposed enhancement: ESC sequence to get actual string length

[unxed]

One of the major challenges in building console interfaces is the unpredictability of the actual length of a string (in screen cells) when using Unicode characters. The width of the same characters can be displayed differently across various terminals.

Of course, it is possible to determine this width by displaying the characters and checking how much the cursor has moved (sample in python is below), but this approach is slow and clutters the console log.

It would be great if you could propose an additional terminal extension, such as an ESC sequence, that allows querying the terminal to determine how many screen cells a given string of Unicode characters will occupy.

Thank you!

import os
import sys
import tty
import termios

def get_cursor_position():
    """Gets current cursor position in terminal"""
    sys.stdout.write('\033[6n')
    sys.stdout.flush()

    buf = ""
    while True:
        ch = sys.stdin.read(1)
        buf += ch
        if ch == "R":
            break

    # Answer sample: '\033[12;40R'
    # Extracts position from string
    try:
        rows, cols = map(int, buf.lstrip('\033[').rstrip('R').split(';'))
    except ValueError:
        rows, cols = -1, -1  # In case of an error

    return rows, cols

def hide_cursor():
    """Hides cursor"""
    sys.stdout.write('\033[?25l')
    sys.stdout.flush()

def show_cursor():
    """Shows cursor"""
    sys.stdout.write('\033[?25h')
    sys.stdout.flush()

def print_teststring():
    """Measures screen cells number"""
    teststring = 'a1🙂❤️'
    
    # Save terminal settings
    fd = sys.stdin.fileno()
    old_settings = termios.tcgetattr(fd)
    
    try:
        # Switching terminal to non blocking mode
        tty.setcbreak(sys.stdin.fileno())

        # Saving cursor position
        sys.stdout.write('\033[s')

        # Hiding cursor
        hide_cursor()

        # Moving cursor outside the visible area
        sys.stdout.write('\033[1000A')

        # Getting position before string output
        _, start_col = get_cursor_position()

        # Printing string
        sys.stdout.write(teststring)
        sys.stdout.flush()

        # Getting position after string output
        _, end_col = get_cursor_position()

        # Moving cursor back to saved position
        sys.stdout.write('\033[u')

        # Showing cursor
        show_cursor()

        # Counting screen cells used
        cells_used = end_col - start_col

        # Printing result
        print("String: \"", teststring, f"\", screen cells count: {cells_used}")
    
    finally:
        # Restoring terminal settings
        termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
        # Showing cursor in case of an error
        show_cursor()

if __name__ == "__main__":
    print_teststring()

[kovidgoyal]

You can use the existing escape codes to do this in a perfectly
performant and non screen clutterring manner. Start with sending
the atomic screen update escape code. Then the escape code to save the
cursor position. Then for each sequence you are interested in

1. Move cursor to 0. 0
2. print the sequence
3. query cursor position
4. repeat

finally retore saved cursor and send escape code to finish atomic
update.

[unxed]

Thanks for your answer! And what is atomic screen update escape code? Do you mean switching to alternate screen buffer?

[kovidgoyal]

https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec

[unxed]

@magiblot, the developer of a modern Turbo Vision implementation with Unicode support, makes a fairly compelling argument as to why the approach of outputting characters and measuring cursor offsets might not be the ideal solution. Let me quote him:

I'm afraid that solution is only feasible on Windows. In order to do that in a Unix terminal:

• Turbo Vision uses the alternate screen buffer, which results in scrollback being disabled in most terminal emulators. You would have to print the characters you want to measure in the same screen area where your application is being drawn. For example: if you tried to print an emoji, then measure the cursor movement, then move the cursor back to its initial position, and then overwrite the emoji with the characters that used to be in that part of the screen, it is very likely that the terminal emulator would display the emoji on screen for some time.
• Even if the characters being measured didn't appear on screen, or if that wasn't an issue, the performance of the whole process would be very poor.
• Even if the above weren't an issue, the input stream used for reading the terminal state is the same that's used for reading user input. Therefore, you would have to either ignore user input while measuring text width, or write code that is able to keep the input events that are received while measuring text width. And then you would also have to consider the risk of waiting forever for an answer from the terminal...

So, in my opinion, you would end up with a poor experience for both the user and the programmer.

magiblot/tvision#51 (comment)

[unxed]

https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec

Is there any terminal other than iterm2 that supports this? It doesn't work for me in Kitty.

[kovidgoyal]

Most modern terminals support definitely including kitty

…

On Sat, Sep 14, 2024 at 01:41:14AM -0700, unxed wrote: > https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec Is there any terminal other than iterm2 that supports this? It doesn't work for me in Kitty. -- Reply to this email directly or view it on GitHub: #7789 (comment) You are receiving this because you modified the open/close state. Message ID: ***@***.***>

--

_____________________________________ Dr. Kovid Goyal https://www.kovidgoyal.net https://calibre-ebook.com

_____________________________________