Skip to content

fix(cli): map Shift+<symbol> under xterm modifyOtherKeys=2 - #91224

Open
sanastasiou wants to merge 1 commit into
NousResearch:mainfrom
sanastasiou:fix/shift-symbol-modifyotherkeys
Open

fix(cli): map Shift+<symbol> under xterm modifyOtherKeys=2#91224
sanastasiou wants to merge 1 commit into
NousResearch:mainfrom
sanastasiou:fix/shift-symbol-modifyotherkeys

Conversation

@sanastasiou

Copy link
Copy Markdown

What does this PR do?

Fixes shifted punctuation being untypeable in the interactive CLI on terminals
that speak xterm modifyOtherKeys.

_enable_extended_enter_keys() pushes modifyOtherKeys level 2 (CSI >4;2m) to
Ghostty, iTerm2, WezTerm, VS Code, tmux and Windows Terminal. Those terminals
then re-encode modified keys as ESC[27;<mod>;<codepoint>~.
install_modify_other_keys_aliases() maps Shift+letter but not Shift+symbol, so
on Ghostty 1.3.x pressing Shift+/ inserts the literal text
^[[27;2;63~ into the prompt instead of ?. Same for ! : " _ + <
> ( ) ~ — i.e. most punctuation is unusable while the mode is active.

This is the same class of bug as #87711, which fixed it for letters only.

The symbols were skipped deliberately, with this comment:

Shift+digit symbols are layout-specific (US: '!', AZERTY: '¹', etc.) so they are NOT mapped here — if the terminal sends those under modifyOtherKeys, they will leak, but that's better than wrong input.

That reasoning holds when deriving a shifted character from an unshifted
codepoint. But the modifyOtherKeys encoding reports the codepoint already
shifted by the terminal
, which has applied the user's real keymap — Shift+/
on a US layout arrives as 63 (?), not 47 (/). So echoing
chr(codepoint) is not a layout guess; it is exactly what the terminal says was
typed, and is correct on every layout. There is no "wrong input" risk to trade
against here.

Related Issue

Related to #87711 (same leak, letters only). No separate issue filed.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • hermes_cli/pt_input_extras.py — in install_modify_other_keys_aliases(),
    register ESC[27;2;<cp>~chr(cp) for every printable ASCII codepoint
    (0x20–0x7E) that is not already mapped. Removed the now-obsolete "symbols are
    layout-specific" note and documented why the shifted-codepoint encoding makes
    this layout-safe.
  • tests/cli/test_cli_shift_symbol_keys.py — new, 6 tests.

Deliberately scoped to the modifyOtherKeys (tilde) spelling only — not
CSI-u.
The Kitty protocol reports the unshifted codepoint plus a shift bit,
so registering ESC[47;2uchr(47) would type / for Shift+/. Kitty-protocol
terminals deliver printable keys as plain text anyway, so there is nothing to map.
A test pins this so a later "make it symmetric" refactor cannot silently break it.

Letters and Space are registered earlier in the same function and are skipped by
the if seq not in ANSI_SEQUENCES membership check, so this only fills genuinely
unmapped codepoints. Codepoints 9/32/127 keep their dedicated BackTab / " " /
ControlH meanings.

How to Test

Reproduce (before this patch), in Ghostty:

  1. Start the interactive CLI: hermes
  2. Press Shift+/ — the prompt shows ^[[27;2;63~ instead of ?.
  3. Same for Shift+1, Shift+;, Shift+'

With the patch, each inserts the expected character.

Headless equivalent:

from prompt_toolkit.input.vt100_parser import Vt100Parser
from hermes_cli.pt_input_extras import install_modify_other_keys_aliases

install_modify_other_keys_aliases()
events = []
Vt100Parser(events.append).feed_and_flush("\x1b[27;2;63~")
print([e.key for e in events])   # ['?']  (was: Escape + 9 literal chars)

Test suite:

pytest tests/cli/test_cli_shift_symbol_keys.py -q          # 6 passed
pytest tests/cli/test_ctrl_enter_newline.py \
       tests/cli/test_cli_shift_enter_newline.py \
       tests/cli/test_cli_terminal_shortcuts.py -q          # 21 passed, 2 skipped

The new suite covers: the shifted-symbol table; an exhaustive no-leak sweep over
all printable ASCII; Shift+letter non-regression; Space/Tab/Backspace not being
clobbered; CSI-u spellings left untouched; and installer idempotency.

Verified end-to-end on Ghostty 1.3.1 / macOS with prompt_toolkit 3.0.52.

_enable_extended_enter_keys() pushes modifyOtherKeys level 2 to Ghostty,
iTerm2, WezTerm, VS Code, tmux and Windows Terminal. Those terminals then
re-encode modified keys as ESC[27;<mod>;<codepoint>~.

install_modify_other_keys_aliases() mapped Shift+letter but deliberately
skipped Shift+symbol, so on Ghostty 1.3.x every shifted punctuation key
leaked into the prompt buffer as literal "^[[27;2;NN~" text -- '?', '!',
':', '"', '_', '+', '<' and friends were untypeable in the interactive CLI
while the mode was active. This is the same class of bug as NousResearch#87711, which
fixed it for letters only.

The symbols were skipped on the grounds that they are layout-specific
(US Shift+1 = '!', AZERTY = 'A~'). That holds when deriving the shifted
character from an UNSHIFTED codepoint, but the modifyOtherKeys encoding
reports the codepoint ALREADY SHIFTED by the terminal, which has applied
the user's real keymap. Echoing chr(codepoint) is therefore not a layout
guess -- it is what the terminal says was typed, and is correct on every
layout.

Only the modifyOtherKeys (tilde) spelling is registered, NOT the CSI-u one:
the Kitty protocol reports the UNSHIFTED codepoint plus a shift bit, so
mapping ESC[47;2u to chr(47) would type '/' for Shift+/. Kitty-protocol
terminals deliver printable keys as plain text anyway. Letters and Space
are already registered and are skipped by the membership check, so this
only fills genuinely unmapped codepoints.

Tests: tests/cli/test_cli_shift_symbol_keys.py -- 6 tests covering the
shifted-symbol table, an exhaustive no-leak sweep over printable ASCII,
Shift+letter non-regression, Space/Tab/Backspace not being clobbered,
CSI-u spellings left alone, and installer idempotency.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard labels Aug 21, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference; please use your judgment.

The reasoning flip ("layout-specific" only applies when deriving the shifted char from an unshifted codepoint — modifyOtherKeys reports the already-shifted one) is correct per the xterm spec, the decision to exclude the CSI-u/Kitty spelling is right, and the test suite is exemplary: exhaustive leak check, regression guard on letters, dedicated-key preservation, Kitty non-mapping, and idempotency. Minor notes:

  1. hermes_cli/pt_input_extras.py:338 — same-class gaps remain for higher modifiers: ESC[27;4;63~ (Alt+Shift+/, etc.), 27;6; (Ctrl+Shift), 27;8; are still unmapped, so those combos keep leaking literal text into the buffer under the very mode this module enables. The identical already-shifted argument applies; worth a fast-follow rather than expanding this PR's blast radius.
  2. pt_input_extras.py:341 — terminals that deviate from the spec by reporting the UNSHIFTED codepoint under modifyOtherKeys=2 (a few older VTE builds have historically done this) will now insert the base character instead of leaking the sequence — strictly better than the status quo, but a silent-wrong-input class. If you want belt-and-braces, a startup probe or a documented opt-out env var would give those users an escape hatch. Low priority given how rare that deviation is.
  3. pt_input_extras.py:340 — �[27;{mod};{cp}~ construction now exists in three places (letters above, symbols here, multi-modifier below). A tiny _mok_seq(modifier, codepoint) helper would make the next extension mechanical. (nit)

No blocking issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants