Skip to content

ci(test-matrix): Add logs for getTestMatrix #17673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 17, 2025
Merged

ci(test-matrix): Add logs for getTestMatrix #17673

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e01dca7
ci(test-matrix): Add logs for `getTestMatrix`
s1gr1d Sep 16, 2025
0ba4fd9
trying stderr - or is this erroring in CI
s1gr1d Sep 16, 2025
9bfc47c
review comments
s1gr1d Sep 17, 2025
87f4240
add count
s1gr1d Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 42 additions & 6 deletions dev-packages/e2e-tests/lib/getTestMatrix.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,24 @@ function run(): void {

const { base, head = 'HEAD', optional } = values;

// For GitHub Action debugging
// eslint-disable-next-line no-console
console.error(`Parsed command line arguments: base=${base}, head=${head}, optional=${optional}`);

const testApplications = globSync('*/package.json', {
cwd: `${__dirname}/../test-applications`,
}).map(filePath => dirname(filePath));

// For GitHub Action debugging (using stderr the 'matrix=...' output is not polluted)
// eslint-disable-next-line no-console
console.error(
`Discovered ${testApplications.length} test applications${
testApplications.length > 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's just make this testApplications.join(', ')? Doesn't really matter if this list is long I suppose 😅

? ` (sample: ${JSON.stringify(testApplications.slice(0, 10))}${testApplications.length > 10 ? ' …' : ''})`
: ''
}`,
);

// If `--base=xxx` is defined, we only want to get test applications changed since that base
// Else, we take all test applications (e.g. on push)
const includedTestApplications = base
Expand Down Expand Up @@ -137,11 +151,22 @@ function getAffectedTestApplications(
additionalArgs.push(`--head=${head}`);
}

const affectedProjects = execSync(`yarn --silent nx show projects --affected ${additionalArgs.join(' ')}`)
.toString()
.split('\n')
.map(line => line.trim())
.filter(Boolean);
let affectedProjects: string[] = [];
try {
affectedProjects = execSync(`yarn --silent nx show projects --affected ${additionalArgs.join(' ')}`)
.toString()
.split('\n')
.map(line => line.trim())
.filter(Boolean);
} catch (error) {
// eslint-disable-next-line no-console
console.warn('Failed to compute affected projects via Nx. Running all tests instead.', error);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
console.warn('Failed to compute affected projects via Nx. Running all tests instead.', error);
console.error('Failed to compute affected projects via Nx. Running all tests instead.', error);

here an error is def. warranted xD

return testApplications;
}

// For GitHub Action debugging
// eslint-disable-next-line no-console
console.error(`Nx affected projects: ${JSON.stringify(affectedProjects)}`);

// If something in e2e tests themselves are changed, check if only test applications were changed
if (affectedProjects.includes('@sentry-internal/e2e-tests')) {
Expand All @@ -150,12 +175,19 @@ function getAffectedTestApplications(

// Shared code was changed, run all tests
if (changedTestApps === false) {
// eslint-disable-next-line no-console
console.error('Shared e2e code changed. Running all test applications.');
return testApplications;
}

// Only test applications that were changed, run selectively
if (changedTestApps.size > 0) {
return testApplications.filter(testApp => changedTestApps.has(testApp));
const selected = testApplications.filter(testApp => changedTestApps.has(testApp));
// eslint-disable-next-line no-console
console.error(
`Only changed test applications will run (${selected.length}): ${JSON.stringify(Array.from(changedTestApps))}`,
);
return selected;
}
} catch (error) {
// eslint-disable-next-line no-console
Expand All @@ -182,6 +214,10 @@ function getChangedTestApps(base: string, head?: string): false | Set<string> {
.map(line => line.trim())
.filter(Boolean);

// For GitHub Action debugging
// eslint-disable-next-line no-console
console.error(`Changed files since ${base}${head ? `..${head}` : ''}: ${JSON.stringify(changedFiles)}`);

const changedTestApps: Set<string> = new Set();
const testAppsPrefix = 'dev-packages/e2e-tests/test-applications/';

Expand Down