Skip to content

Fix audio indicator audibility signal - #6566

Merged
austinywang merged 1 commit into
mainfrom
fix-audio-indicator-audible-only
Jun 22, 2026
Merged

austinywang merged 1 commit into
mainfrom
fix-audio-indicator-audible-only

Conversation

@austinywang

@austinywang austinywang commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • split broad media playback retention from audible-audio UI state
  • report DOM media audibility separately from playing media
  • keep the speaker glyph off for silent, muted, disabled-track, or video-only playback

Verification

  • xcodebuild -project cmux.xcodeproj -scheme cmux-unit -configuration Debug -destination 'platform=macOS' -derivedDataPath /tmp/cmux-fix-audio-indicator-audible-tests -only-testing:cmuxTests/BrowserMediaPlaybackAudioActivityTests -only-testing:cmuxTests/BrowserHiddenWebViewDiscardMediaPlaybackTests test
  • python3 scripts/swift_file_length_budget.py --budget .github/swift-file-length-budget.tsv
  • ./scripts/check-pbxproj.sh
  • ./scripts/lint-pbxproj-test-wiring.sh
  • /Users/austinwang/manaflow/cmuxterm-hq/skills/autoreview/scripts/autoreview --mode branch --base origin/main

View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


Summary by cubic

Separate media playback from audibility so the speaker icon appears only when audio is actually audible; muted, silent, disabled-track, or video-only playback no longer shows the icon while still preventing hidden pane discard.

  • Bug Fixes
    • Injected hook now reports both playing and audible, using mute, volume, audioTracks, and decoded-byte checks; listens to media and volumechange/timeupdate events.
    • Message handler and report include isAudible; panel tracks playingMediaFrameIDs and audibleMediaFrameIDs separately.
    • isPlayingMedia continues to block hidden WebView discard; isPlayingAudio drives the speaker glyph and updates on mute/unmute and navigation resets.
    • Added BrowserMediaPlaybackAudioActivityTests to validate silent vs audible playback behavior.

Written for commit b87f1dc. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • New Features

    • Improved media playback tracking that now distinguishes between actively playing and audible media, accounting for mute status and volume levels.
  • Tests

    • Added test coverage for media playback audio activity detection.

@vercel

vercel Bot commented Jun 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment Jun 22, 2026 4:03am
cmux-staging Building Building Preview, Comment Jun 22, 2026 4:03am

@coderabbitai

coderabbitai Bot commented Jun 22, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f32ca772-238b-438c-8c1b-1261f25d3175

📥 Commits

Reviewing files that changed from the base of the PR and between cddc403 and b87f1dc.

📒 Files selected for processing (6)
  • Sources/Panels/BrowserMediaPlaybackMessageHandler.swift
  • Sources/Panels/BrowserMediaPlaybackReport.swift
  • Sources/Panels/BrowserPanel+MediaPlayback.swift
  • Sources/Panels/BrowserPanel.swift
  • cmux.xcodeproj/project.pbxproj
  • cmuxTests/BrowserMediaPlaybackAudioActivityTests.swift

📝 Walkthrough

Walkthrough

Extends the media playback reporting pipeline with an isAudible flag. BrowserMediaPlaybackReport gains an isAudible field; the WebKit message handler parses it; the injected JavaScript gains dual-flag debounce state with audible detection helpers; and BrowserPanel now tracks two separate frame ID sets (playingMediaFrameIDs, audibleMediaFrameIDs), with a refreshAudioMediaActivity helper that factors in mute state and is called on report apply, reset, and mute changes. New tests cover silent-playing and audible-then-silent transitions.

Changes

Audible Media Tracking

Layer / File(s) Summary
BrowserMediaPlaybackReport and message handler contract
Sources/Panels/BrowserMediaPlaybackReport.swift, Sources/Panels/BrowserMediaPlaybackMessageHandler.swift
Adds isAudible: Bool to BrowserMediaPlaybackReport and updates BrowserMediaPlaybackMessageHandler to parse audible from the payload (defaulting to false) and pass it to the report constructor.
Injected JavaScript audible detection
Sources/Panels/BrowserPanel+MediaPlayback.swift
Adds hasDetectableAudioSource and isElementAudible JS helpers, replaces single lastReported with dual-flag debounce state and a per-element WeakMap, expands tracked events to include volumechange/timeupdate, and posts { playing, audible } on state change and (false, false) on pagehide. The Swift callback now forwards isAudible and logs anyAudible.
BrowserPanel dual frame-set state and mute integration
Sources/Panels/BrowserPanel.swift
Introduces audibleMediaFrameIDs set alongside playingMediaFrameIDs, adds refreshAudioMediaActivity gated on !isMuted, updates applyMediaPlaybackReport to accept isAudible, wires setMuted to call refreshAudioMediaActivity, and changes isPlayingMedia.didSet to invoke reevaluateHiddenWebViewDiscardScheduling directly.
Tests and Xcode project registration
cmuxTests/BrowserMediaPlaybackAudioActivityTests.swift, cmux.xcodeproj/project.pbxproj
Adds BrowserMediaPlaybackAudioActivityTests with two @MainActor test cases for silent-playing and audible-then-silent playback report sequences; registers the file in PBXBuildFile, PBXFileReference, group children, and PBXSourcesBuildPhase.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • manaflow-ai/cmux#4911: Introduced isMuted/mute plumbing in BrowserPanel that this PR's refreshAudioMediaActivity directly reads via !isMuted.
  • manaflow-ai/cmux#5412: Both PRs modify BrowserPanel's media-playback tracking to drive hidden WebView discard scheduling; this PR extends that pipeline with isAudible.
  • manaflow-ai/cmux#5441: Both PRs modify the shared BrowserMediaPlaybackMessageHandler/BrowserMediaPlaybackReport/BrowserPanel+MediaPlayback pipeline; that PR adds playing, this PR extends it with audible.

Poem

🐰 Hop hop, a little ear twitching with glee,
Now I know if the media's loud or just free!
Two sets of frames — playing and audible too,
WeakMaps and debounce, the rabbit flew through.
Mute it or not, the glyph knows the score,
isAudible lands, and silence no more! 🎵


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 errors, 1 warning)

Check name Status Explanation Resolution
Cmux Cache Substitution Correctness ❌ Error The PR swaps fresh authoritative reads for cached values in a snapshot persistence path without handling stale caches. makeWorkspaceSnapshot() uses tab.browserMediaActivity (cached via Workspace) i... Add a freshness-checked fallback (call currentBrowserMediaActivity()) or document graceful-degradation rationale explaining why stale mediaActivity is harmless for sidebar display.
Cmux Source Artifacts ❌ Error PR adds 25 .claude/ directory files (explicitly forbidden by rule), 12 Package.resolved dependency lock files, and other tool artifacts (.coderabbit.yaml, .cursor/rules/, .greptile/, .agents/skills... Remove all .claude/, Package.resolved, .coderabbit.yaml, .cursor/, .greptile/, and .agents/ files from the commit. Keep only the intended audio playback source code changes (BrowserMediaPlayback*.swift and tests).
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (20 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Fix audio indicator audibility signal' clearly and concisely identifies the main fix in the changeset, which is improving how the audio indicator responds to actual audibility state.
Description check ✅ Passed The description includes a summary section explaining what changed and why, and a verification section detailing test execution commands, but lacks demo video and incomplete checklist items.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed Production code introduces no Swift 6 actor isolation mistakes: BrowserMediaPlaybackReport is a pure Sendable value struct matching existing patterns; all mutable state is private in @MainActor Bro...
Cmux Swift Blocking Runtime ✅ Passed The PR introduces no blocking or timing-based synchronization patterns in production Swift code. The changes consist of adding an isAudible field to media playback reporting, updating tracking lo...
Cmux Expensive Synchronous Load ✅ Passed The PR introduces changes to media playback reporting (parsing boolean flags from dictionaries, tracking frame IDs in sets, updating boolean state) with no expensive synchronous loads like Restorab...
Cmux No Hacky Sleeps ✅ Passed No new sleep, timer, setTimeout, setInterval, or polling-based synchronization added. All code changes are in Swift or use event-driven JavaScript (MutationObserver, addEventListener). Date.now() u...
Cmux Algorithmic Complexity ✅ Passed The PR uses O(1) Set operations in hot Swift paths, bounds JavaScript DOM scans to media events with early-exit optimization and explicit debouncing, and documents why full-DOM scans on every mutat...
Cmux Swift Concurrency ✅ Passed No legacy async patterns introduced; PR properly uses MainActor.assumeIsolated for WebKit callback boundary and maintains modern Swift concurrency throughout.
Cmux Swift @Concurrent ✅ Passed All Swift functions are synchronous, properly actor-isolated on @MainActor BrowserPanel, with correct use of MainActor.assumeIsolated for WebKit message delivery. No async functions, @concurrent an...
Cmux Swift File And Package Boundaries ✅ Passed PR adds only 14 lines to BrowserPanel.swift (11,765 lines), well below the 250-line threshold for oversized files. Focused bug fix that splits media playback retention from audible-audio UI state,...
Cmux Swiftpm Lockfiles ✅ Passed PR changes do not involve SwiftPM package dependency changes, .gitignore modifications, or SwiftPM package references. The xcodeproj change only adds a test source file (BrowserMediaPlaybackAudioAc...
Cmux Swift Logging ✅ Passed The PR introduces no logging violations. Production code changes (BrowserMediaPlaybackMessageHandler, BrowserMediaPlaybackReport, new functions in BrowserPanel) contain no unguarded logging. Browse...
Cmux User-Facing Error Privacy ✅ Passed The PR makes internal changes to media playback tracking logic. No user-facing error messages, alerts, or sensitive information (credentials, tokens, vendor names, environment variables, database d...
Cmux Full Internationalization ✅ Passed PR contains no user-facing strings, localization APIs, or string catalog changes. All code changes are internal media playback logic, debug-only logs (#if DEBUG), JavaScript system tokens, and test...
Cmux Swiftui State Layout ✅ Passed PR adds private tracking properties for media playback (playingMediaFrameIDs, audibleMediaFrameIDs) but no new @Published/@observable state. Existing BrowserPanel ObservableObject is touched only i...
Cmux Architecture Rethink ✅ Passed Changes follow sound architecture: single source of truth (playingMediaFrameIDs, audibleMediaFrameIDs), single entry point (applyMediaPlaybackReport), clear invariants, no timing patterns/observers...
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed PR introduces no new NSWindow/NSPanel/NSWindowController/Window/WindowGroup creation. Changes are internal media tracking state and test fixtures only.
Cmux No Test Or Debug Seam In Production Source ✅ Passed #if DEBUG block in BrowserPanel+MediaPlayback.swift gates diagnostic logging (cmuxDebugLog), a real product feature, not a test-observability seam. No test-named members added. Test file uses @test...
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-audio-indicator-audible-only

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR separates media playback retention (keeping a hidden pane alive) from audible-audio UI state (the speaker glyph). Previously, any playing media — including muted, video-only, or silenced playback — triggered the speaker indicator. Now isPlayingMedia retains the pane while the new isAudible signal drives isPlayingAudio, with DOM-level mute/volume/audio-source detection in the injected JS hook and a native-level cmux-mute guard in refreshAudioMediaActivity.

  • Adds isAudible: Bool to BrowserMediaPlaybackReport and a parallel audibleMediaFrameIDs set in BrowserPanel; the speaker glyph is suppressed for muted, zero-volume, disabled-track, or video-only playback.
  • Enriches the injected JS bootstrap with volumechange and timeupdate event listeners plus a per-element WeakMap dedup guard to detect audibility transitions without full-DOM scans on each event.
  • setMuted now immediately re-evaluates isPlayingAudio via refreshAudioMediaActivity, ensuring the glyph clears synchronously on the native-side mute toggle.

Confidence Score: 5/5

Safe to merge — the change correctly separates retention from audio-glyph state, both the Swift and JS logic handle all state transitions, and a new test suite validates the two key invariants.

The split between playingMediaFrameIDs (pane retention) and audibleMediaFrameIDs (speaker glyph) is implemented consistently across applyMediaPlaybackReport, resetMediaPlaybackTracking, and setMuted. The native !isMuted guard in refreshAudioMediaActivity correctly handles the cmux-level mute toggle independently of DOM-level mute reported by JS. The JS reportIfTargetStateChanged WeakMap per-element deduplication properly prevents timeupdate from causing redundant full-DOM scans. No state transitions are left unhandled.

No files require special attention; BrowserPanel.swift carries the most logic but each path is straightforward and consistent.

Important Files Changed

Filename Overview
Sources/Panels/BrowserPanel.swift Core logic change: adds audibleMediaFrameIDs, splits isPlayingMedia (retention) from isPlayingAudio (glyph), adds refreshAudioMediaActivity helper. reevaluateHiddenWebViewDiscardScheduling can fire twice per applyMediaPlaybackReport call (once from isPlayingMedia.didSet, once from setMediaActivity) — idempotent but slightly redundant.
Sources/Panels/BrowserPanel+MediaPlayback.swift JS bootstrap extended with hasAudioSource, isElementAudible, reportIfTargetStateChanged (WeakMap per-element dedup), and volumechange/timeupdate event listeners. Logic and early-exit paths are correct; timeupdate fires frequently but the WeakMap guard keeps each call O(1).
Sources/Panels/BrowserMediaPlaybackReport.swift Adds isAudible: Bool to the report struct. Minimal, clean change.
Sources/Panels/BrowserMediaPlaybackMessageHandler.swift Extracts audible from the JS message body with a safe ?? false default and passes it through to the report. Clean, minimal change.
cmuxTests/BrowserMediaPlaybackAudioActivityTests.swift New Swift Testing suite covering the two key invariants: silent playing keeps isPlayingMedia true but isPlayingAudio false, and audible→silent transition clears isPlayingAudio without affecting isPlayingMedia.
cmux.xcodeproj/project.pbxproj Wires BrowserMediaPlaybackAudioActivityTests.swift into the cmux test target. File references and build-phase entries look consistent.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[JS: media event fires
play/pause/volumechange/timeupdate/...] --> B[reportIfTargetStateChanged]
    B --> C{el matches
video, audio?}
    C -- No --> D[report: full DOM scan]
    C -- Yes --> E[compute isElementPlaying
isElementAudible]
    E --> F{WeakMap state
changed?}
    F -- No --> G[early exit: no-op]
    F -- Yes --> H[update WeakMap
call report]
    H --> D
    D --> I[currentPlaybackState
full DOM scan]
    I --> J{state changed
vs lastReported?}
    J -- No --> K[skip post]
    J -- Yes --> L[post: frameID, playing, audible]
    L --> M[BrowserMediaPlaybackMessageHandler
main thread]
    M --> N[applyMediaPlaybackReport]
    N --> O[update playingMediaFrameIDs
update audibleMediaFrameIDs]
    O --> P[isPlayingMedia = !playingMediaFrameIDs.isEmpty]
    P --> Q[refreshAudioMediaActivity]
    Q --> R[setMediaActivity
isPlayingAudio = !audibleMediaFrameIDs.isEmpty && !isMuted]
    R --> S[onMediaActivityChanged callback
Workspace → tab/sidebar glyph]
    T[setMuted called] --> U[isMuted = muted]
    U --> Q
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[JS: media event fires
play/pause/volumechange/timeupdate/...] --> B[reportIfTargetStateChanged]
    B --> C{el matches
video, audio?}
    C -- No --> D[report: full DOM scan]
    C -- Yes --> E[compute isElementPlaying
isElementAudible]
    E --> F{WeakMap state
changed?}
    F -- No --> G[early exit: no-op]
    F -- Yes --> H[update WeakMap
call report]
    H --> D
    D --> I[currentPlaybackState
full DOM scan]
    I --> J{state changed
vs lastReported?}
    J -- No --> K[skip post]
    J -- Yes --> L[post: frameID, playing, audible]
    L --> M[BrowserMediaPlaybackMessageHandler
main thread]
    M --> N[applyMediaPlaybackReport]
    N --> O[update playingMediaFrameIDs
update audibleMediaFrameIDs]
    O --> P[isPlayingMedia = !playingMediaFrameIDs.isEmpty]
    P --> Q[refreshAudioMediaActivity]
    Q --> R[setMediaActivity
isPlayingAudio = !audibleMediaFrameIDs.isEmpty && !isMuted]
    R --> S[onMediaActivityChanged callback
Workspace → tab/sidebar glyph]
    T[setMuted called] --> U[isMuted = muted]
    U --> Q
Loading

Reviews (1): Last reviewed commit: "Fix audio indicator audibility signal" | Re-trigger Greptile

@austinywang
austinywang merged commit 170e754 into main Jun 22, 2026
30 checks passed
@austinywang
austinywang deleted the fix-audio-indicator-audible-only branch June 22, 2026 04:16

This branch was successfully deployed

1 active deployment
Preview – cmux — b87f1dc7 Deployed Jun 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant