Skip to content

feat(dictation): use Whisper across composers through DSS - #6771

Closed
404Wolf wants to merge 23 commits into
mainfrom
cursor/dictation-whisper-support-6b88
Closed

404Wolf wants to merge 23 commits into
mainfrom
cursor/dictation-whisper-support-6b88

Conversation

@404Wolf

@404Wolf 404Wolf commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Rebases / updates #6527 onto latest main so the dictation Whisper work can merge.

Conflict resolution:

  • AgentInput keeps main’s file-attachment composer (InputProvider, drop zone, chips, attach action) and adds Whisper dictation (mic button, panel, send/send-next guards, inert overlay).
  • ChannelInput keeps main’s flat / Dynamic composer chrome and dictation controls, including collapsed-mic start and send guards.
  • Comment-import tests keep main’s release_import_lock helper and the explicit advisory-unlock wait.
  • Linux node_modules hash refreshed for @solid-primitives/stream on current bun.lock. Darwin hash is still main’s until the native macOS checker reports the new value.

Validation:

  • Vitest: dictation, AgentInput, ChannelInput slots, ChatInput, dictation query/adapter — 74 tests passed
  • cargo test -p dictation -p ai_usage — 29 passed
  • cargo test -p macro_db_migrator --bin comment_import --features comment-import — 9 passed
  • DSS compiles
  • just check passed
Open in Web Open in Cursor 

jbecke and others added 20 commits September 16, 2026 19:10
Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
…async-openai Whisper

- Split crates/dictation into domain (models/ports/service), inbound
  (axum router) and outbound (Whisper adapter) behind feature flags.
- Detect the audio container from the bytes with infer instead of
  trusting the client Content-Type; LanguageHint newtype for the hint.
- Read OPENAI_API_KEY through a macro_env_var newtype and call Whisper
  through async-openai (audio feature) with a bounded retry budget.
- Require a signed-in user (UserOnly) and rate limit each user to 60
  transcriptions per hour with the shared rate_limit middleware.
- Move the DSS endpoint into the crate's inbound router; wire router
  state, Redis limiter, and swagger schema in the composition root.
- Regenerate workspace-hack and dependency closures.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
…t-driven tests

- Add AudioRecorder: constructed with the callbacks that receive levels,
  the finished recording, limit, and errors; only one instance holds the
  microphone at a time. Levels come from @solid-primitives/stream and
  every MediaRecorder chunk advances both the timeline and the duration
  cap, so the recorder has no timers.
- Primitives take a recorder factory instead of start-recording and
  volume-meter functions; confirm() returns a promise that settles when
  text is committed or the attempt is abandoned.
- Replace the 3s finish fallback with a second confirm that commits the
  words recognized so far.
- Rewrite tests around explicit recorder/recognizer events: no fake
  timers, advanceTimersByTime, or Promise.resolve flushes.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
…er-fallback

# Conflicts:
#	apps/web/src/features/channel/Input/ChannelInput.tsx
…er-fallback

# Conflicts:
#	nix-support/node_modules-hashes.json
Resolve composer conflicts by keeping main's agent file attachments and
channel flat composer, plus Whisper dictation controls and send guards.
Keep the comment-import advisory unlock helper name from main.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
bun.lock now includes @solid-primitives/stream on top of latest main.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: macro-inc/macro/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 1f6d8c43-d8f1-43ab-b847-6b95401807cf

📥 Commits

Reviewing files that changed from the base of the PR and between 8d0b359 and bd12090.

⛔ Files ignored due to path filters (38)
  • .sqlx/query-16a038db0df182a666e0717ac7f9243efeeb51622f9bea833501850b099cc291.json is excluded by !**/.sqlx/**
  • .sqlx/query-40ce78006bc37d7088397407fae17e249fa9f8524426c950480b3df116d6729e.json is excluded by !**/.sqlx/**
  • .sqlx/query-645d4533a5ece189935ffc7859067772f61584ab24b98f4b6a66ba30dbf09ae8.json is excluded by !**/.sqlx/**
  • .sqlx/query-68dcb987626d91eaa4526ab79ba646da0d94e452286665dcc1e136468a7b2df6.json is excluded by !**/.sqlx/**
  • .sqlx/query-81bed360250642bbb9f8f46d705bfe31be1a467c32bb2c8a16cf61cc668fb6c1.json is excluded by !**/.sqlx/**
  • .sqlx/query-8f6b8f408336fd56bf212f1b17f32f36dc1fdbe6d45e4ccfe3ab938974d828c3.json is excluded by !**/.sqlx/**
  • .sqlx/query-c552084079684d8689a7330053ba60061cd45ce5071f59c394858f3fb2c8ea09.json is excluded by !**/.sqlx/**
  • Cargo.lock is excluded by !**/*.lock, !**/Cargo.lock
  • apps/web/src/lib/service-clients/service-cognition/generated/client.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/aiFeature.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/completionUsage.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/completionUsageEntity.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/featureUsage.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/index.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/price.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/pricePricePerAudioMinute.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/setPricingRequest.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/setPricingRequestPricePerAudioMinute.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/setPricingRequestPricePerMilIn.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/setPricingRequestPricePerMilOut.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/usage.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/usageAudioSeconds.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-cognition/generated/schemas/usageSummary.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-storage/generated/schemas/index.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-storage/generated/schemas/transcribeDictationParams.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-storage/generated/schemas/transcribeResponse.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • apps/web/src/lib/service-clients/service-storage/generated/zod.ts is excluded by !**/generated/**, !apps/web/src/lib/service-clients/**/generated/**
  • bun.lock is excluded by !**/*.lock, !**/bun.lock
  • crates/dictation/tests/fixtures/chrome.webm is excluded by !**/*.webm
  • crates/dictation/tests/fixtures/long.ogg is excluded by !**/*.ogg
  • crates/dictation/tests/fixtures/tone.mp4 is excluded by !**/*.mp4
  • crates/dictation/tests/fixtures/tone.ogg is excluded by !**/*.ogg
  • crates/dictation/tests/fixtures/tone.wav is excluded by !**/*.wav
  • crates/dictation/tests/fixtures/tone.webm is excluded by !**/*.webm
  • packages/sdk/generated/cognition/types.gen.ts is excluded by !**/generated/**, !**/*.gen.ts
  • packages/sdk/generated/storage/index.ts is excluded by !**/generated/**
  • packages/sdk/generated/storage/sdk.gen.ts is excluded by !**/generated/**, !**/*.gen.ts
  • packages/sdk/generated/storage/types.gen.ts is excluded by !**/generated/**, !**/*.gen.ts
📒 Files selected for processing (74)
  • .github/workspace-dep-closures.json
  • Cargo.toml
  • apps/web/package.json
  • apps/web/src/components/ui/components/CollapsedInput.tsx
  • apps/web/src/features/block-agent/ui/AgentInput.test.tsx
  • apps/web/src/features/block-agent/ui/AgentInput.tsx
  • apps/web/src/features/channel/Input/ChannelInput.tsx
  • apps/web/src/features/channel/Input/tests/input-slots.test.tsx
  • apps/web/src/features/dictation/browser/audio-recorder.test.ts
  • apps/web/src/features/dictation/browser/audio-recorder.ts
  • apps/web/src/features/dictation/components/dictation-controls.tsx
  • apps/web/src/features/dictation/components/volume-timeline.tsx
  • apps/web/src/features/dictation/composer-dictation.ts
  • apps/web/src/features/dictation/core/recording.ts
  • apps/web/src/features/dictation/core/types.ts
  • apps/web/src/features/dictation/core/volume.ts
  • apps/web/src/features/dictation/primitives/create-recorded-dictation.test.ts
  • apps/web/src/features/dictation/primitives/create-recorded-dictation.ts
  • apps/web/src/lib/core/component/AI/component/input/ChatInput.test.tsx
  • apps/web/src/lib/core/component/AI/component/input/ChatInput.tsx
  • apps/web/src/lib/queries/dictation/keys.ts
  • apps/web/src/lib/queries/dictation/transcribe.test.ts
  • apps/web/src/lib/queries/dictation/transcribe.ts
  • apps/web/src/lib/service-clients/service-cognition/openapi.json
  • apps/web/src/lib/service-clients/service-storage/dictation.test.ts
  • apps/web/src/lib/service-clients/service-storage/dictation.ts
  • apps/web/src/lib/service-clients/service-storage/openapi.json
  • crates/ai_usage/src/domain/mod.rs
  • crates/ai_usage/src/domain/ports.rs
  • crates/ai_usage/src/domain/service.rs
  • crates/ai_usage/src/domain/service/test.rs
  • crates/ai_usage/src/inbound.rs
  • crates/ai_usage/src/inbound/axum_router.rs
  • crates/ai_usage/src/inbound/axum_router/test.rs
  • crates/ai_usage/src/inbound/models.rs
  • crates/ai_usage/src/inbound/models/test.rs
  • crates/ai_usage/src/lib.rs
  • crates/ai_usage/src/outbound/pg_usage_repo.rs
  • crates/ai_usage/src/outbound/pg_usage_repo/test.rs
  • crates/dictation/Cargo.toml
  • crates/dictation/src/domain.rs
  • crates/dictation/src/domain/models.rs
  • crates/dictation/src/domain/ports.rs
  • crates/dictation/src/domain/service.rs
  • crates/dictation/src/domain/service/test.rs
  • crates/dictation/src/inbound.rs
  • crates/dictation/src/inbound/axum_router.rs
  • crates/dictation/src/inbound/axum_router/test.rs
  • crates/dictation/src/lib.rs
  • crates/dictation/src/outbound.rs
  • crates/dictation/src/outbound/media.rs
  • crates/dictation/src/outbound/media/test.rs
  • crates/dictation/src/outbound/whisper.rs
  • crates/dictation/src/outbound/whisper/test.rs
  • crates/dictation/tests/fixtures/README.md
  • crates/dictation/tests/whisper_tracing.rs
  • crates/macro_db_client/migrations/20260918204004_add_audio_usage_pricing.sql
  • crates/macro_db_migrator/src/bin/comment_import/test.rs
  • crates/workspace-hack/Cargo.toml
  • deny.toml
  • docs/AGENT_GUIDE/ai-chat.md
  • docs/AGENT_GUIDE/channels.md
  • nix-support/node_modules-hashes.json
  • packages/sdk/specs/cognition.json
  • packages/sdk/specs/storage.json
  • packages/sdk/src/coverage/skipped.ts
  • services/document_cognition_service/src/api/swagger.rs
  • services/document_storage_service/Cargo.toml
  • services/document_storage_service/src/api.rs
  • services/document_storage_service/src/api/context.rs
  • services/document_storage_service/src/api/swagger.rs
  • services/document_storage_service/src/config.rs
  • services/document_storage_service/src/main.rs
  • services/email_service/Cargo.toml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
@404Wolf
404Wolf marked this pull request as ready for review September 22, 2026 14:50
@404Wolf
404Wolf requested a review from a team as a code owner September 22, 2026 14:50

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bd12090. Configure here.

);
},
{ discrete: true }
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dictation insert corrupts trailing mentions

Medium Severity

Confirming dictation always does selectEnd().insertText() on the last descendant. When the draft ends in a mention or other decorator, that inserts inside the chip instead of after it, so the mention’s display text (and often its identity) is overwritten. Users who @mention someone and then dictate lose or corrupt that mention.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit bd12090. Configure here.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
Keep the native Darwin node_modules hash and touch dictation fixtures so
cloud-storage and Nix hash workflows run on HEAD.

Co-authored-by: Wolf Mermelstein <wolf@404wolf.com>
@404Wolf 404Wolf closed this Sep 22, 2026
@404Wolf 404Wolf reopened this Sep 22, 2026
@404Wolf 404Wolf closed this Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants