Skip to content

fix: detect app-host tests from embedded bundle #9217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
austinywang wants to merge 2 commits into from
Closed

fix: detect app-host tests from embedded bundle #9217

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b67618
test: detect embedded app-host test bundle
austinywang Jul 30, 2026
530b734
fix: detect app-host tests from embedded bundle
austinywang Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Sources/App/MacSentryStartupPolicy.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import Foundation

struct MacSentryStartupPolicy: Sendable {
let telemetryEnabled: Bool
let isRunningUnderXCTest: Bool
Expand Down Expand Up @@ -36,6 +38,17 @@ struct MacSentryStartupPolicy: Sendable {
if environment["XCInjectBundleInto"] != nil { return true }
if environment["DYLD_INSERT_LIBRARIES"]?.contains("libXCTest") == true { return true }
if environment.keys.contains(where: { $0.hasPrefix("CMUX_UI_TEST_") }) { return true }
if hasEmbeddedXCTestBundle() { return true }
return false
}

private static func hasEmbeddedXCTestBundle() -> Bool {
guard
let plugInsPath = Bundle.main.builtInPlugInsPath,
let plugInNames = try? FileManager.default.contentsOfDirectory(atPath: plugInsPath)
else {
return false
}
return plugInNames.contains { $0.hasSuffix(".xctest") }
}
Comment on lines +41 to +53

@coderabbitai coderabbitai Bot Jul 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | rg '(^|/)MacSentryStartupPolicy\.swift$' || true

echo "== file outline/stats =="
if [ -f Sources/App/MacSentryStartupPolicy.swift ]; then
  wc -l Sources/App/MacSentryStartupPolicy.swift
  ast-grep outline Sources/App/MacSentryStartupPolicy.swift || true
  echo "== relevant file contents =="
  cat -n Sources/App/MacSentryStartupPolicy.swift
fi

echo "== usages =="
rg -n "isRunningUnderXCTest|MacSentryStartupPolicy|Sentry|SentryStartup|hasEmbeddedXCTestBundle|running under XCTest" . --glob '!**/DerivedData/**' --glob '!**/.build/**' || true

Repository: manaflow-ai/cmux

Length of output: 50372

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p=Path('Sources/App/MacSentryStartupPolicy.swift')
if not p.exists():
    raise SystemExit
s=p.read_text()
checks=[
    'static func isRunningUnderXCTest',
    'hasEmbeddedXCTestBundle()',
    'Bundle.main.builtInPlugInsPath',
    'contentsOfDirectory(atPath:',
    'XCTEST_RUNNER',
    'TESTING',
    'XCTest'
]
for term in checks:
    count=s.count(term)
    print(f"{term}: {count}")
# Print function bodies around identifiers.
text=s.splitlines()
for name in ['isRunningUnderXCTest','hasEmbeddedXCTestBundle']:
    idx=None
    for i,line in enumerate(text,1):
        if name in line and ('func ' in line or 'private static func' in line):
            idx=max(1,i-3); break
    if idx:
        print(f"\n-- {name} --")
        for j in range(idx, min(len(text), idx+35)+1):
            print(f"{j:4}: {text[j-1]}")
PY

Repository: manaflow-ai/cmux

Length of output: 2122

Make XCTest detection fail-closed and single-sourced.

Line 41 adds an unrelated check before return false, so the policy currently accepts many compatibility/test markers and hasEmbeddedXCTestBundle() as authoritative test signals instead of using a single, fresh XCTest source. hasEmbeddedXCTestBundle() also treats missing .xctest entries the same as bundle enumeration failures, making an inherited or stale test marker enough to suppress Sentry in a normal app. Collapse these checks to one structured authority and make unsupported/ambiguous signals fail closed.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/App/MacSentryStartupPolicy.swift` around lines 41 - 53, Update the
policy method surrounding hasEmbeddedXCTestBundle() to remove the unrelated
compatibility/test-marker check and use one fresh, structured XCTest detection
source. Distinguish successful enumeration with an actual .xctest entry from
missing or failed bundle inspection, and make unsupported or ambiguous results
fail closed so only confirmed XCTest environments suppress Sentry.

Sources: Coding guidelines, Path instructions

}
6 changes: 3 additions & 3 deletions Sources/AppDelegate.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,9 +529,9 @@ final class AppDelegate: NSObject, NSApplicationDelegate, UNUserNotificationCent
private let cmuxThemePreviewReloadScheduler = MainActorDeferredActionScheduler()

private func isRunningUnderXCTest(_ env: [String: String]) -> Bool {
// On some macOS/Xcode setups, the app-under-test process doesn't get
// `XCTestConfigurationFilePath`. Use a broader set of signals so UI tests
// can reliably skip heavyweight startup work and bring up a window.
// App-host tests embed an .xctest bundle even when Xcode omits its test
// environment keys. Keep those keys as compatibility signals for UI tests
// and other launch paths.
MacSentryStartupPolicy.isRunningUnderXCTest(environment: env)
}

Expand Down
9 changes: 9 additions & 0 deletions cmuxTests/SentryEventScrubberTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,15 @@ import Testing
)
}

@Test func embeddedAppHostTestBundlePreventsSentryStartup() {
#expect(
MacSentryStartupPolicy(
environment: [:],
telemetryEnabled: true
).shouldStart == false
)
}

@Test func explicitTestTelemetryOptInOverridesUITestMarker() {
#expect(
MacSentryStartupPolicy(
Expand Down
Loading