[llvm-cov] Introduce -include-filename-regex#175779
Merged
Conversation
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
Contributor
|
Can you elaborate on the motivation for this? And describe what, if anything, you are changing in existing |
Contributor
Author
|
The motivation is described here: SerenityOS/serenity#26541 But in short, in the project we have multiple libraries and our tests are organized per-libraries. We generate a global coverage report with the result of all combined tests, but I also wanted to generate per-library reports. The goal is to ensure that the coverage result of llvm-cov show $all_test_binaries \
-format html \
-show-line-counts-or-regions -show-directory-coverage \
-Xdemangler $CLANG_BINDIR/llvm-cxxfilt -Xdemangler -n \
-instr-profile "$TEMP_PROFDATA/profiles/$library/$library.profdata" \
-o "$BUILD_DIR/reports/$library" \
-compilation-dir=$SERENITY_ROOT \ # We pass all the sources from the repo.
-ignore-filename-regex=(Meta|Tests|Build|Kernel) \ # Filter out some indesirables.
-include-filename-regex="\/$library\/" # Filter in the source of the library for which we generate the report.This patch adds I hope this answer your questions! |
Contributor
Author
|
Hi @evodius96, |
Contributor
|
@LucasChollet |
This allows to filter the source directory so the coverage outputs only includes the files matching the regex.
LucasChollet
force-pushed
the
llvm-cov-include
branch
from
7f23b34 to
1f76072
Compare
Member
|
@llvm/pr-subscribers-llvm-binary-utilities Author: Lucas Chollet (LucasChollet) ChangesThis allows to filter the source directory so the coverage outputs only includes the files matching the regex. @chapuni @evodius96 Full diff: https://github.com/llvm/llvm-project/pull/175779.diff 7 Files Affected:
diff --git a/llvm/docs/CommandGuide/llvm-cov.rst b/llvm/docs/CommandGuide/llvm-cov.rst
index f4db60cf06fa7..0a9cfbd0d12e3 100644
--- a/llvm/docs/CommandGuide/llvm-cov.rst
+++ b/llvm/docs/CommandGuide/llvm-cov.rst
@@ -289,6 +289,10 @@ OPTIONS
Skip source code files with file paths that match the given regular expression.
+.. option:: -include-filename-regex=<PATTERN>
+
+ Only include source code files with file paths that match the given regular expression.
+
.. option:: -format=<FORMAT>
Use the specified output format. The supported formats are: "text", "html".
diff --git a/llvm/test/tools/llvm-cov/ignore-filename-regex.test b/llvm/test/tools/llvm-cov/ignore-filename-regex.test
index aea9e4646776d..a3941eb796229 100644
--- a/llvm/test/tools/llvm-cov/ignore-filename-regex.test
+++ b/llvm/test/tools/llvm-cov/ignore-filename-regex.test
@@ -46,7 +46,7 @@ RUN: -path-equivalence=/tmp,%S/Inputs -ignore-filename-regex='.*\.cc$' \
RUN: %S/Inputs/sources_specified/main.covmapping \
RUN: | FileCheck -check-prefix=SHOW_IGNORE_CC %s
-# Order of files may differ, check that there are 3 files and not abs.h.
+# Order of files may differ, check that there are 3 files and not main.cc.
SHOW_IGNORE_CC-NOT: {{.*}}main.cc{{.*}}
SHOW_IGNORE_CC: {{.*}}sources_specified{{.*}}
SHOW_IGNORE_CC: {{.*}}sources_specified{{.*}}
diff --git a/llvm/test/tools/llvm-cov/include-and-exlude-filename-regex.test b/llvm/test/tools/llvm-cov/include-and-exlude-filename-regex.test
new file mode 100644
index 0000000000000..9371b3df3a418
--- /dev/null
+++ b/llvm/test/tools/llvm-cov/include-and-exlude-filename-regex.test
@@ -0,0 +1,40 @@
+########################
+# Test "report" command.
+########################
+# Include files with a "a" in their name and exclude header files.
+RUN: llvm-cov report -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs \
+RUN: -include-filename-regex='.*a.*' -ignore-filename-regex='.*\.h$' \
+RUN: %S/Inputs/sources_specified/main.covmapping --show-branch-summary=false \
+RUN: | FileCheck -check-prefix=REPORT_SOURCE_WITH_A %s
+
+REPORT_SOURCE_WITH_A-NOT: {{.*}}dec.h{{.*}}
+REPORT_SOURCE_WITH_A-NOT: {{.*}}inc.h{{.*}}
+REPORT_SOURCE_WITH_A-NOT: {{.*}}abs.h{{.*}}
+REPORT_SOURCE_WITH_A: {{^}}TOTAL 1{{.*}}100.00%{{$}}
+
+# Only include files from "extra" directory and ignore files starting with "d".
+RUN: llvm-cov report -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs \
+RUN: -include-filename-regex='.*extra[/\\].*' -ignore-filename-regex='.[/\\]d.*' \
+RUN: %S/Inputs/sources_specified/main.covmapping --show-branch-summary=false \
+RUN: | FileCheck -check-prefix=REPORT_INCLUDE_DIR_WITHOUT_D %s
+
+REPORT_INCLUDE_DIR_WITHOUT_D-NOT: {{.*}}extra{{[/\\]}}dec.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D: {{.*}}extra{{[/\\]}}inc.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D-NOT: {{.*}}abs.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D-NOT: {{.*}}main.cc{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D: {{^}}TOTAL 1{{.*}}100.00%{{$}}
+
+# Same test as above but the arguments are passed in a different order.
+RUN: llvm-cov report -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs \
+RUN: -ignore-filename-regex='.[/\\]d.*' -include-filename-regex='.*extra[/\\].*' \
+RUN: %S/Inputs/sources_specified/main.covmapping --show-branch-summary=false \
+RUN: | FileCheck -check-prefix=REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED %s
+
+REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED-NOT: {{.*}}extra{{[/\\]}}dec.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED: {{.*}}extra{{[/\\]}}inc.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED-NOT: {{.*}}abs.h{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED-NOT: {{.*}}main.cc{{.*}}
+REPORT_INCLUDE_DIR_WITHOUT_D_REVERSED: {{^}}TOTAL 1{{.*}}100.00%{{$}}
\ No newline at end of file
diff --git a/llvm/test/tools/llvm-cov/include-filename-regex.test b/llvm/test/tools/llvm-cov/include-filename-regex.test
new file mode 100644
index 0000000000000..338f33fdec4d3
--- /dev/null
+++ b/llvm/test/tools/llvm-cov/include-filename-regex.test
@@ -0,0 +1,59 @@
+########################
+# Test "report" command.
+########################
+# Include only source files.
+RUN: llvm-cov report -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs -include-filename-regex='.*\.cc$' \
+RUN: %S/Inputs/sources_specified/main.covmapping --show-branch-summary=false \
+RUN: | FileCheck -check-prefix=REPORT_INCLUDE_SOURCE %s
+
+REPORT_INCLUDE_SOURCE-NOT: {{.*}}dec.h{{.*}}
+REPORT_INCLUDE_SOURCE-NOT: {{.*}}inc.h{{.*}}
+REPORT_INCLUDE_SOURCE-NOT: {{.*}}abs.h{{.*}}
+REPORT_INCLUDE_SOURCE: {{^}}TOTAL 1{{.*}}100.00%{{$}}
+
+# Only include files from "extra" directory.
+RUN: llvm-cov report -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs -include-filename-regex='.*extra[/\\].*' \
+RUN: %S/Inputs/sources_specified/main.covmapping --show-branch-summary=false \
+RUN: | FileCheck -check-prefix=REPORT_INCLUDE_DIR %s
+
+# llvm-cov uses extra as the base directory.
+REPORT_INCLUDE_DIR: {{.*}}dec.h{{.*}}
+REPORT_INCLUDE_DIR: {{.*}}inc.h{{.*}}
+REPORT_INCLUDE_DIR-NOT: {{.*}}abs.h{{.*}}
+REPORT_INCLUDE_DIR-NOT: {{.*}}main.cc{{.*}}
+REPORT_INCLUDE_DIR: {{^}}TOTAL 2{{.*}}50.00%{{$}}
+
+########################
+# Test "show" command.
+########################
+# Include only header files.
+RUN: llvm-cov show -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs -include-filename-regex='.*\.h$' \
+RUN: %S/Inputs/sources_specified/main.covmapping \
+RUN: | FileCheck -check-prefix=SHOW_INCLUDE_HEADERS %s
+
+# Order of files may differ, check that there are 3 files and not main.cc.
+SHOW_INCLUDE_HEADERS-NOT: {{.*}}main.cc{{.*}}
+SHOW_INCLUDE_HEADERS: {{.*}}sources_specified{{.*}}
+SHOW_INCLUDE_HEADERS: {{.*}}sources_specified{{.*}}
+SHOW_INCLUDE_HEADERS: {{.*}}sources_specified{{.*}}
+
+########################
+# Test "export" command.
+########################
+# Use a temp .json file as output in a single line. Only include headers that have
+# name in a format of 3 symbols followed by ".h".
+RUN: llvm-cov export -instr-profile %S/Inputs/sources_specified/main.profdata \
+RUN: -path-equivalence=/tmp,%S/Inputs -include-filename-regex='.*...\.h$' \
+RUN: %S/Inputs/sources_specified/main.covmapping \
+RUN: > %t.export.json
+
+RUN: FileCheck -check-prefix=NO-EXPORT_INCLUDE_3_SYMBOLS_H %s < %t.export.json
+RUN: FileCheck -check-prefix=EXPORT_INCLUDE_3_SYMBOLS_H %s < %t.export.json
+
+NO-EXPORT_INCLUDE_3_SYMBOLS_H: {{"filename":"(/|\\\\)tmp(/|\\\\)sources_specified(/|\\\\)abs.h"}}
+NO-EXPORT_INCLUDE_3_SYMBOLS_H: {{"filename":"(/|\\\\)tmp(/|\\\\)sources_specified(/|\\\\)extra(/|\\\\)dec.h"}}
+NO-EXPORT_INCLUDE_3_SYMBOLS_H: {{"filename":"(/|\\\\)tmp(/|\\\\)sources_specified(/|\\\\)extra(/|\\\\)inc.h"}}
+EXPORT_INCLUDE_3_SYMBOLS_H-NOT: {{"filename":"(/|\\\\)tmp(/|\\\\)sources_specified(/|\\\\)main.cc"}}
diff --git a/llvm/tools/llvm-cov/CodeCoverage.cpp b/llvm/tools/llvm-cov/CodeCoverage.cpp
index a112dc8bced40..0066d10156499 100644
--- a/llvm/tools/llvm-cov/CodeCoverage.cpp
+++ b/llvm/tools/llvm-cov/CodeCoverage.cpp
@@ -147,7 +147,7 @@ class CodeCoverageTool {
std::vector<StringRef> ObjectFilenames;
CoverageViewOptions ViewOpts;
CoverageFiltersMatchAll Filters;
- CoverageFilters IgnoreFilenameFilters;
+ CoverageFilters FilenameFilters;
/// True if InputSourceFiles are provided.
bool HadSourceFiles = false;
@@ -222,7 +222,7 @@ void CodeCoverageTool::addCollectedPath(const std::string &Path) {
return;
}
sys::path::remove_dots(EffectivePath, /*remove_dot_dot=*/true);
- if (!IgnoreFilenameFilters.matchesFilename(EffectivePath))
+ if (!FilenameFilters.matchesFilename(EffectivePath))
SourceFiles.emplace_back(EffectivePath.str());
HadSourceFiles = !SourceFiles.empty();
}
@@ -734,6 +734,12 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {
"regular expression"),
cl::cat(FilteringCategory));
+ cl::list<std::string> IncludeFilenameRegexFilters(
+ "include-filename-regex", cl::Optional,
+ cl::desc("Only include source code files with file paths that match the "
+ "given regular expression"),
+ cl::cat(FilteringCategory));
+
cl::opt<double> RegionCoverageLtFilter(
"region-coverage-lt", cl::Optional,
cl::desc("Show code coverage only for functions with region coverage "
@@ -935,8 +941,11 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {
// Create the ignore filename filters.
for (const auto &RE : IgnoreFilenameRegexFilters)
- IgnoreFilenameFilters.push_back(
- std::make_unique<NameRegexCoverageFilter>(RE));
+ FilenameFilters.push_back(std::make_unique<NameRegexCoverageFilter>(RE));
+
+ for (const auto &RE : IncludeFilenameRegexFilters)
+ FilenameFilters.push_back(std::make_unique<NameRegexCoverageFilter>(
+ RE, NameRegexCoverageFilter::FilterType::Include));
if (!Arches.empty()) {
for (const std::string &Arch : Arches) {
@@ -953,7 +962,7 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {
}
}
- // IgnoreFilenameFilters are applied even when InputSourceFiles specified.
+ // FilenameFilters are applied even when InputSourceFiles specified.
for (const std::string &File : InputSourceFiles)
collectPaths(File);
@@ -1164,7 +1173,7 @@ int CodeCoverageTool::doShow(int argc, const char **argv,
if (SourceFiles.empty() && !HadSourceFiles)
// Get the source files from the function coverage mapping.
for (StringRef Filename : Coverage->getUniqueSourceFiles()) {
- if (!IgnoreFilenameFilters.matchesFilename(Filename))
+ if (!FilenameFilters.matchesFilename(Filename))
SourceFiles.push_back(std::string(Filename));
}
@@ -1276,7 +1285,7 @@ int CodeCoverageTool::doReport(int argc, const char **argv,
CoverageReport Report(ViewOpts, *Coverage);
if (!ShowFunctionSummaries) {
if (SourceFiles.empty())
- Report.renderFileReports(llvm::outs(), IgnoreFilenameFilters);
+ Report.renderFileReports(llvm::outs(), FilenameFilters);
else
Report.renderFileReports(llvm::outs(), SourceFiles);
} else {
@@ -1360,7 +1369,7 @@ int CodeCoverageTool::doExport(int argc, const char **argv,
}
if (SourceFiles.empty())
- Exporter->renderRoot(IgnoreFilenameFilters);
+ Exporter->renderRoot(FilenameFilters);
else
Exporter->renderRoot(SourceFiles);
diff --git a/llvm/tools/llvm-cov/CoverageFilters.cpp b/llvm/tools/llvm-cov/CoverageFilters.cpp
index bc1ddb41087f9..17fe18934b5b4 100644
--- a/llvm/tools/llvm-cov/CoverageFilters.cpp
+++ b/llvm/tools/llvm-cov/CoverageFilters.cpp
@@ -31,7 +31,8 @@ bool NameRegexCoverageFilter::matches(
}
bool NameRegexCoverageFilter::matchesFilename(StringRef Filename) const {
- return llvm::Regex(Regex).match(Filename);
+ bool regex_match = llvm::Regex(Regex).match(Filename);
+ return Type == FilterType::Exclude ? regex_match : !regex_match;
}
bool NameAllowlistCoverageFilter::matches(
diff --git a/llvm/tools/llvm-cov/CoverageFilters.h b/llvm/tools/llvm-cov/CoverageFilters.h
index 3cee23ae50dbf..39a19a0e65c5c 100644
--- a/llvm/tools/llvm-cov/CoverageFilters.h
+++ b/llvm/tools/llvm-cov/CoverageFilters.h
@@ -55,10 +55,20 @@ class NameCoverageFilter : public CoverageFilter {
/// Matches functions whose name matches a certain regular expression.
class NameRegexCoverageFilter : public CoverageFilter {
+public:
+ enum class FilterType {
+ Include,
+ Exclude,
+ };
+
+private:
StringRef Regex;
+ FilterType Type;
public:
- NameRegexCoverageFilter(StringRef Regex) : Regex(Regex) {}
+ NameRegexCoverageFilter(StringRef Regex,
+ FilterType Type = FilterType::Exclude)
+ : Regex(Regex), Type(Type) {}
bool matches(const coverage::CoverageMapping &CM,
const coverage::FunctionRecord &Function) const override;
|
Contributor
Author
|
@chapuni |
Contributor
Author
|
@chapuni, @evodius96 |
evodius96
approved these changes
Jan 27, 2026
Contributor
evodius96
left a comment
evodius96
left a comment
There was a problem hiding this comment.
Ok thanks! Let us know if you need help merging.
Contributor
Author
|
@evodius96 |
|
@LucasChollet Congratulations on having your first Pull Request (PR) merged into the LLVM Project! Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR. Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues. How to do this, and the rest of the post-merge process, is covered in detail here. If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again. If you don't get any reports, no action is required from you. Your changes are working as expected, well done! |
Contributor
Author
|
Thank you for the merge! |
stomfaig
pushed a commit
to stomfaig/llvm-project
that referenced
this pull request
Jan 28, 2026
This allows to filter the source directory so the coverage outputs only includes the files matching the regex.
sshrestha-aa
pushed a commit
to sshrestha-aa/llvm-project
that referenced
this pull request
Feb 4, 2026
This allows to filter the source directory so the coverage outputs only includes the files matching the regex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This allows to filter the source directory so the coverage outputs only includes the files matching the regex.
@chapuni @evodius96
This is my first LLVM PR, and I'll happily make any requested change.