Skip to content

feat(voice): hold one key to talk, with a tone at each end - #87

Merged
ANT1OFF merged 2 commits into
mainfrom
feat/voice-one-key
Nov 2, 2026
Merged

ANT1OFF merged 2 commits into
mainfrom
feat/voice-one-key

Conversation

@ANT1OFF

@ANT1OFF ANT1OFF commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

What

Hold one key to dictate, as Hex does, with an audible start and stop.

  • One modifier held alone can be the dictation key: Right/Left ⌥, ⌘, ⌃ or ⇧ (Alt, Win, Ctrl, Shift off macOS). Record it in Settings → Keyboard by tapping the modifier alone; only Dictate accepts one. A chord such as ⌘⇧Space still works as before (a tap toggles, a hold talks). Bindings are stored as KeyboardEvent.code lowercased (altright), which the keybindings file pattern already accepts. Labels follow the platform through formatKeys: "Right ⌥" on macOS, "Right Alt" on Linux and Windows.
  • Default: Right ⌥ on macOS, Ctrl+Shift+Space elsewhere. On many European layouts Right Alt is AltGr, which types brackets, so it would start and discard a dictation on every {. I asked the orchestrator to confirm this default; it's one line (DICTATE_KEYS in keyboard/catalog.ts) if it should change.
  • Held alone, it works anywhere in Gram's window and dictates into the composer in use: the one whose pane has the focus, else the one used last. That composer's field takes the focus, so the words land there and Escape reaches it. It does nothing inside a dialog.
  • Gesture rules (Hex's as ideas, no code ported, so no notice is needed):
    • Dictation starts at the press, so nothing is missed.
    • A release under 300 ms is a tap and is discarded.
    • Any other key during the hold discards, so ⌥-shortcuts and ⌥-characters keep working.
    • A second tap within 350 ms locks listening on until the next press.
    • A press while dictation is already running (a lock, or the mic button) stops it.
    • A blur keeps a hold and drops a tap.
  • Tones, synthesised with Web Audio (two short sine notes, a fourth up for start and down for stop); no audio files.
    • The start tone plays once capture is live and the press is confirmed as a hold or lock (or it's a mic-button/chord start).
    • The stop tone plays on release, and only after a start tone, so a tap or a discard is silent.
    • The context suspends after 2 s, so it holds no output device.
  • Settings → Voice input → Sounds: a switch plus a volume slider (default on, half volume). Changing either plays the start tone at the new volume. The voice switch's description now describes the held key.

Verification

  • e2e dictation.spec.ts, 17 passed. Each test now pins its binding, so the same presses work on macOS and Linux runners. New:
    • Holding Right ⌥ with the focus outside the composer: the composer takes focus, partials show, the text commits, tones [659, 880].
    • A 150 ms tap with Keep microphone ready on (capture is live inside the tap) and ⌥↑ during a hold (⌥← is Back off macOS, which failed the first Linux CI run): nothing committed, no tones. Mutation check: with the tone gating removed, this test fails with 3 extra tones.
    • Double tap locks, the next press commits. With Sounds off, no tones. Settings shows the Sounds switch and volume.
  • e2e key-capture.spec.ts and keyboard-nav.spec.ts pass. catalog-coverage sees AltRight pressed.
  • Unit:
    • The modifier-hold machine (hold, tap, chord, late chord, double tap, taps too far apart, a tap after a shortcut, the mic button's dictation, blur, Escape).
    • Lone-modifier names and labels, and that they never match as a chord.
    • Recording a lone modifier in the key dialog, only for Dictate, and not when a key joins it.
    • The sounds settings contract, with volume bounded to 0–1.
  • Commands: vitest (dictation, keyboard, settings, palette, contracts settings), typecheck, lint, fmt:check, check:scaffold, check:docs, the three e2e specs.

Known limits

  • Not tried with a real keyboard by hand, or heard on speakers. Key events come from Playwright in Chromium, and the tones were counted at the oscillators, not listened to.
  • Mic opens on ⌥-shortcuts: with Keep microphone ready off, an ⌥-shortcut or ⌥-character on the bound key opens the mic for a moment before the discard (indicator blink). With it on, the stream is already open.
  • Late shortcut after the tone: a shortcut pressed after a hold has confirmed (≥ 300 ms) discards after the start tone has played; the stop tone doesn't play.
  • No globalShortcut: the key only works while a Gram window has the focus.

Screenshots

Settings → Advanced → Voice input, with voice input on (dark)

Before After
settings-voice-input-dark before settings-voice-input-dark after

@ANT1OFF
ANT1OFF force-pushed the feat/voice-one-key branch from 8be43ad to db504ba Compare November 2, 2026 07:36
Dictation can be bound to one modifier held alone (Right ⌥ by default on
macOS; the chord stays elsewhere, where Right Alt is AltGr). Held alone it
works anywhere in the window and dictates into the composer in use: it starts
at the press, a tap under 300 ms or a press with another key discards
silently, and a double tap keeps listening until the next press.

Start and stop tones are synthesised with Web Audio: the start tone once
capture is live and the press is a hold, the stop tone on release, nothing
for a discard. Settings → Voice input gets Sounds with a volume.
@ANT1OFF
ANT1OFF force-pushed the feat/voice-one-key branch from f42269d to 74ee5db Compare November 2, 2026 08:35
@ANT1OFF
ANT1OFF merged commit 1a9b3ef into main Nov 2, 2026
23 checks passed
@ANT1OFF
ANT1OFF deleted the feat/voice-one-key branch November 2, 2026 08:52
ANT1OFF added a commit that referenced this pull request Nov 2, 2026
Desktop release metadata proposal only. Merging this PR does not publish
or tag Gram; follow docs/RELEASING.md to release.
---


## [1.4.0](v1.3.1...v1.4.0)
(2026-11-02)


### Features

* **agents:** identify Gram's Claude processes as the gram entrypoint
([#91](#91))
([f7dd703](f7dd703))
* **desktop:** run on Linux
([#82](#82))
([1d37deb](1d37deb))
* **desktop:** turn voice input off on Linux with one check
([#79](#79))
([470ff8b](470ff8b))
* **orchestration:** a person's PR comment wakes the worker that owns it
([#86](#86))
([19bdc1c](19bdc1c))
* **orchestration:** a wrap-up turn when a worker stops without
reporting ([#88](#88))
([36a07fb](36a07fb))
* **orchestration:** workers say what they ruled out and check before
repeating ([#90](#90))
([8bdce01](8bdce01))
* **release:** publish and update Linux AppImage and .deb
([#95](#95))
([c33c464](c33c464))
* **voice:** an accurate mode with a Parakeet pass on each pause
([#83](#83))
([2b04a52](2b04a52))
* **voice:** hold one key to talk, with a tone at each end
([#87](#87))
([1a9b3ef](1a9b3ef))
* **voice:** start capture at the press, and an opt-in ready microphone
([#81](#81))
([505d240](505d240))
* **web:** a Done section that keeps the last 7 days
([#65](#65))
([8c7bd49](8c7bd49))
* **web:** Done lists every finished chat
([#67](#67))
([c91d9ac](c91d9ac))
* **web:** one attention count, quieter rows, a sidebar and board that
agree ([#92](#92))
([f193da4](f193da4))
* **web:** orchestrator → workers in the sidebar, a board that matches
([#80](#80))
([686e737](686e737))


### Bug Fixes

* **desktop:** harden the Linux key vault and autostart entry
([#85](#85))
([4b35eac](4b35eac))
* **e2e:** hold the t-flaky turn while the smoke test reads its opening
line ([#93](#93))
([2edc4ac](2edc4ac))
* **e2e:** release the dictation tap as soon as a word is heard
([#94](#94))
([7781194](7781194))
* prepare only approved local-check sources
([#32](#32))
([af3fab7](af3fab7))
* repair main after [#32](#32)
([#68](#68))
([ce352e6](ce352e6))
* **voice:** keep the last word after release
([#78](#78))
([cc40a47](cc40a47))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: gram-release-please[bot] <351321620+gram-release-please[bot]@users.noreply.github.com>
Co-authored-by: Johan <32440065+ANT1OFF@users.noreply.github.com>
ANT1OFF added a commit that referenced this pull request Nov 2, 2026
Voice input stops being an experiment: it is a Settings page of its own,
**Voice**, with the switch at the top and a push-to-talk key you can
change from there.

### What changes
- **Voice is a nav group** between Notifications and Advanced (Mic
icon). It is listed and found by search on every platform, whether voice
is on or off. The `experimental` tag on the page is gone, and so is the
redirect to Diagnostics. Diagnostics' Experimental group and Advanced's
"Experimental" row went too, since nothing else was in them. The old
word "experimental" still finds the page in search, the same way
Workspaces kept it after 0.7.
- **Off by default, consent unchanged.** Turning it on still shows the
model's size and licence before any download. Turning it off still
deletes the models. On Linux the page shows the switch disabled with
`VOICE_UNSUPPORTED_NOTE`.
- **Existing users keep their state.** The switch is still stored as
`experimental.voiceInput` (contracts: "the key keeps its name after a
feature graduates"). Nothing is migrated, so anyone who had it on still
has it on. There's a test for this in
`packages/contracts/src/settings.test.ts`.
- **Push-to-talk key row** (shown while voice is on): the current
binding with platform labels, a **Change…** button that opens the same
`KeyCaptureDialog` Settings → Keyboard uses, and the ↺ reset beside the
label (Hint: "Reset to default"). The behaviour help changes with the
binding: for a modifier held alone it is "hold anywhere / double-tap to
keep listening", for a chord it is "tap to start and stop / hold while
you talk". Clashes with other bindings are flagged. Saving goes through
one new hook, `useSaveBindings`, which Keyboard now uses too. Both write
`keybindings.json`, so there is one binding and no second setting.
- **The recorder already captured lone modifiers** (#87). The audit
found it handles Right ⌥/⌘/⌃/⇧ on key-up. Added:
- `fn` as a lone modifier. Whether Electron reports it depends on the
keyboard, so this is not verified on hardware.
- Caveats: off macOS, choosing Right Alt warns that it is AltGr on many
layouts. On macOS, choosing Fn points to "Press 🌐 key to → Do Nothing".
- **Dictate shortcut in Keyboard and in the ? overlay**: the catalog's
`experimental: "voiceInput"` is replaced by `listed: voiceInputOn`. The
shortcut is listed while voice is on and hidden while it's off, because
then it does nothing (and on Linux it never can). The Voice page offers
the key once voice is on.
- Docs: `AGENTS_AND_FEATURES.md` now names Settings → Voice, the stored
key, and both places to rebind.
- No dictation or speech-process behaviour changed.

### What's New
`whats-new/` only accepts fragments for the current package version
(1.4.0), which is already tagged (`assemble.test.ts` enforces both
rules). So the card belongs in the 1.5.0 release PR. Suggested copy:
*"Voice is a setting of its own — Voice input is no longer experimental:
turn it on in Settings under Voice, and pick your push-to-talk key
there, one modifier held alone or a chord. It isn't on Linux yet."*

### Verification
- `bunx vitest run src/settings src/keyboard src/dictation
src/composer/composer-dictation.test.tsx src/palette` (apps/web): 66
files, 514 tests pass. New: `dictate-key-row.test.tsx` (lone modifier,
chord, clash, reset, AltGr warning) and caveat/Fn cases in
`key-capture-dialog.test.tsx`. Updated: listing in `search.test.tsx`,
nav in `settings-nav.test.tsx`, `listedShortcuts` in `catalog.test.ts`.
- `bunx vitest run src/settings.test.ts` (contracts): 46 pass, including
the existing-user case.
- `bun run --filter @gram/web e2e e2e/dictation.spec.ts
e2e/settings.spec.ts --workers=1`: 28/28 pass. This includes the
rewritten Settings test (page listed while off, consent before download)
and a new key-picker test: Right Ctrl held alone, the same binding shown
in Keyboard, a ⌘L clash replaced, Esc cancels, reset.
- `bun run fmt:check`, `bun run lint`, `bun run check:scaffold`: pass.
- `bun run typecheck`: this machine reports 262 errors on untouched
`main` (021df5d) as well as on this branch, and none are in files this
PR touches. That looks environmental (local toolchain); CI is the
authority.

### Known limits
- Fn as a push-to-talk key has not been tried on real hardware.
- The Linux AltGr warning only appears where the recorder can be
reached. On Linux, voice can't be on, so in practice it shows on
Windows.

### Screenshots

**Settings → Voice, with voice input on** (dark)

| Before | After |
| --- | --- |
| ![settings-voice-input-dark
before](https://github.com/user-attachments/assets/e3e1b455-14aa-4777-bec7-afe87cd585ef)
| ![settings-voice-input-dark
after](https://github.com/user-attachments/assets/39b40a43-9f20-4157-9b6e-c3ac8abc92fd)
|

**Settings → Voice, with voice input on** (light)

| Before | After |
| --- | --- |
| ![settings-voice-input-light
before](https://github.com/user-attachments/assets/6dea7f5f-417b-49b9-ab89-3ad2ff0ed970)
| ![settings-voice-input-light
after](https://github.com/user-attachments/assets/98ad884c-91ec-4902-b329-0915f973d4b9)
|

**Settings → Voice, off: listed as a page of its own, with only the
switch** (dark)

| Before | After |
| --- | --- |
| ![settings-voice-input-off-dark
before](https://github.com/user-attachments/assets/5a86cecb-5c37-4597-8aa7-de75040dac9e)
| ![settings-voice-input-off-dark
after](https://github.com/user-attachments/assets/d9e47b67-4c2d-4134-a8a0-0c1fecc6031a)
|

**The dictation key's recorder, Right ⌘ held alone just recorded**
(dark)

| Before | After |
| --- | --- |
| ![dictate-key-recorder-dark
before](https://github.com/user-attachments/assets/827149b9-3f7f-40e8-889a-57325be1698f)
| ![dictate-key-recorder-dark
after](https://github.com/user-attachments/assets/2816dac5-6897-44ef-82ae-a6184c2f1011)
|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant