Skip to content

Conversation

egernst
Copy link
Contributor

@egernst egernst commented Sep 25, 2025

To facilitate ability to gather code coverage metrics without slowing down our integration target, let's add a coverage target.

From this we can run xcrun llvm-profdata merge to get profdata and gather overall code coverage metrics

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Motivation and Context

[Why is this change needed?]

Testing

  • Tested locally
  • Added/updated tests
  • Added/updated docs

@katiewasnothere
Copy link
Contributor

How much slower does adding --enable-code-coverage make running the tests?

@egernst
Copy link
Contributor Author

egernst commented Sep 26, 2025

Not that much. The goal is to run make integration (CLI tests) and the other tests together in a single run, serially (which wouldn’t be ideal otherwise for CI testing)

@egernst egernst force-pushed the cocov branch 2 times, most recently from ffc2967 to a843cdd Compare October 2, 2025 14:45
@egernst
Copy link
Contributor Author

egernst commented Oct 2, 2025

@katiewasnothere @dcantah - updated to add XCRUN so we get the css output for code coverage

Makefile Outdated
--ignore-filename-regex=".pb.swift" \
--ignore-filename-regex=".proto" \
--ignore-filename-regex=".grpc.swift" \
$(BUILD_BIN_DIR)/ContainerPackageTests.xctest/Contents/MacOS/ContainerPackageTests > $(COV_REPORT_FILE) ; \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we referencing xctest here? This doesn't work for me locally

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly b/c this was the path for build output after running tests (make test) on my system. Maybe we can chat through this one in realtime, or others can TAL on their platform for what's unique here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the issue, I have my dev env setup on a case sensitive volume. The xctest bundle that shows up for me is at $(BUILD_BIN_DIR)/containerPackageTests.xctest/Contents/MacOS/containerPackageTests so the capitalized "ContainerPackageTests" is what's causing this to fail locally for me. Can we update the PR with the lowercase containerPackageTests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this makes sense - will fix.

To facilitate ability to gather code coverage metrics without slowing
down our integration target, let's add a coverage target.

Signed-off-by: Eric Ernst <[email protected]>
@egernst egernst merged commit 89c69d6 into apple:main Oct 13, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants