-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
C++ Coverage on Linux is broken #9406
Comments
I have a similar problem. Try to run this script in an empty directory: echo -n > WORKSPACE
echo 'cc_test(name = "foo_test", srcs = ["main.cpp"])' > BUILD
echo 'int main() {}' > main.cpp
bazel coverage //:foo_test --instrument_test_targets
tail -n 1 bazel-testlogs/foo_test/test.log On my PC, the script above prints the following log:
Note that the last line says there was no coverage. My environment:
|
All versions larger than 0.24.1 have this problem. |
It seems that in GCC 9.1.0, the meaning of the
Instead of generating a “.gcov” file, a “.gcov.json.gz” file is generated. This change may cause my bazel failing to collect coverage data. |
I also tested GCC 5.4.0, and noticed that the gcov log doesn’t contain “Creating” text so the following grep command will fail: bazel/tools/test/collect_cc_coverage.sh Line 129 in 6193cfd
So for now, I see two problems:
|
@EFanZh, Have you tried gcc version 4.9.2? I am using this version with bazel 0.25.0 to generate an empty coverage.dat file. |
No, I haven’t tried GCC 4.9.2 or Bazel 0.25.0. But you can find the files matching “*.gcov.log” in Bazel cache directory, and see their content. If there is no “Creating” text, then Bazel cannot generate coverage reports. |
I can't find any files mathes “.gcov.log” or ".gcov" in Basel cache directory. All versions larger than 0.24.1 are the same problem. If you are convenient, you can try gcc 4.9.2 with bazel 0.29.1 version. |
Is there a “test.log” file in your “bazel-testlogs” directory? |
Yes, there is a "test.log" file and there is "creating" text in the file. But the "coverage.dat" file is empty, when generated coverage with bazel version higher than 0.24.1. $ ll bazel-testlogs/id_test/ Some logs are as follows:
|
The coverage tests are run in a sandbox environment. Add “--sandbox_debug” to the command line that runs the coverage test, so that the log files are kept in its original position. |
Using this parameter does not work either. The generated coverage files are empty whether in a sandbox or standalone environment. |
That flag is for debugging. It won’t affect the outcome. Now you should be able to find “*.gcov.log” files in your Bazel cache after adding the flag. |
Even with this parameter, no ".gcov.log" file is generated by converage command in the bazel cache dir. |
Try this command:
|
With this command, i find the *.gcov.log file and there is not "creating" text in the file. |
@ishikhman |
@hlopko could you please help @ruifangChen ? Thanks! |
0.25 0.26 0.28, all of these versions have the same problem |
As well as 0.29.1 |
@hlopko Hi, could you give me some help? How to get C++ coverage with bazel 0.26.1 and gcc 4.9.2. Thanks very much! |
Hi, This branch also activates the -b (branch) option for gcov as the described bug is resolved in gcov 8+. Now i see that gcov generates branch output but now it seems that the lcov_merger removes the added branch data again (data can be seen in the gcov output but not in the final .dat files). This should be updated in the lcov_merger and along with supporting the new .gcov.json.gz files. |
There's some in the gcov docs: |
I did some experiments with it on Tuesday and it's very similar to the old format, just in Json, so it should be possible to extend Bazel's coverage merger to load it into its in-memory structures (which are similar to the old format). |
There is an open issue in lcov project. Do these two issues have any relation? |
FWIW, Envoy switched to use Clang based solution by using |
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format in parallel to the existing GCov parser Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format in parallel to the existing GCov parser Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: bazelbuild#9406
GCC 9 changed the intermediate format to a JSON based format and with it changed the meaning of the `-i` flag. Because of these changes it was not possible to generate code coverage with GCC 9. This patch addresses that by adding its format next to the existing GCov parser. Addresses: #9406 Closes #11538. PiperOrigin-RevId: 316641962
GCC 9.2.1 and bazel 3.3.1 is still not working. Got zero-sized coverage.dat.
|
@xingxinghuo1000 Bazel 3.3.1 does not include the changes I made. |
Define configuration options for collecting coverage data with Bazel. The coverage command for Bazel is fairly fragile and many configurations do not work [1,2]. The `--javabase` option is set as suggested in this bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu 18.04. [1]: bazelbuild/bazel#9406 [2]: bazelbuild/bazel#10457 [3]: bazelbuild/bazel#7953
Define configuration options for collecting coverage data with Bazel. The coverage command for Bazel is fairly fragile and many configurations do not work [1,2]. The `--javabase` option is set as suggested in this bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu 18.04. [1]: bazelbuild/bazel#9406 [2]: bazelbuild/bazel#10457 [3]: bazelbuild/bazel#7953
I experience the very same issue of non-working coverage on MacOS and Java tests:
Am I cooking it right? |
I tried 3.4.1. It works well. Thanks 👍 |
bazel 3.4.1 works well on my hello world demo project. But in real project, on some test targets, an error occured when collecting coverage by " bazel coverage //XXX" gcc: 9.2.1 lcov tool version: bazel version 3.4.1
@helaan Could you please look at the error above? Thank you. I wrote a simple python script to resolve all the json-format gcov data. I DID found data like "8.00292e+09"
Is this a bug? I Guess the question is: GsonBuiler().setLongSerializationPolicy(LongSerializationPolicy.STRING) could be the solution |
It looks like this was fixed. |
Define configuration options for collecting coverage data with Bazel. The coverage command for Bazel is fairly fragile and many configurations do not work [1,2]. The `--javabase` option is set as suggested in this bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu 18.04. [1]: bazelbuild/bazel#9406 [2]: bazelbuild/bazel#10457 [3]: bazelbuild/bazel#7953
bazel 4.0 + gcc 7.3 still not work. |
Thank you. It looks like this is [tested](https://cs.opensource.google/bazel/bazel/+/master:src/test/shell/bazel/bazel_cc_code_coverage_test.sh;l=298. Right now we are not prioritizing work in this area. If you isolate the problem to a bug in Bazel instead of a problem in your environment, please feel free to send a patch if you figure out what's wrong. |
Description of the problem / feature request:
When I execute the bazel coverage command with version 0.24.1, I can generate a valid coverage.dat file, but when using a version higher than 0.24.1, the coverage.dat file is empty.
This problem exists only when generating coverage for the C++ language.
$ ll bazel-testlogs/id_test/
total 3136
drwxr-xr-x 2 admin admin 4096 Sep 19 15:29 ./
drwxr-xr-x 3 admin admin 4096 Sep 19 15:07 ../
-r-xr-xr-x 1 admin admin 581600 Sep 19 15:07 baseline_coverage.dat*
-r-xr-xr-x 1 admin admin 68477 Sep 19 15:29 coverage.dat*
-r-xr-xr-x 1 admin admin 179 Sep 19 15:29 test.cache_status*
-r-xr-xr-x 1 admin admin 2539566 Sep 19 15:29 test.log*
-rw-r--r-- 1 admin admin 904 Sep 19 15:28 test.xml
$ bazel version
Build label: 0.24.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Apr 2 16:29:26 2019 (1554222566)
Build timestamp: 1554222566
Build timestamp as int: 1554222566
$ ll bazel-testlogs/id_test/
total 4108
drwxr-xr-x 2 admin admin 4096 Sep 19 15:25 ./
drwxr-xr-x 3 admin admin 4096 Sep 19 15:07 ../
-r-xr-xr-x 1 admin admin 581600 Sep 19 15:07 baseline_coverage.dat*
-r-xr-xr-x 1 admin admin 0 Sep 19 15:25 coverage.dat*
-r-xr-xr-x 1 admin admin 176 Sep 19 15:25 test.cache_status*
-r-xr-xr-x 1 admin admin 3600923 Sep 19 15:25 test.log*
-rw-r--r-- 1 admin admin 904 Sep 19 15:25 test.xml
$ bazel version
Build label: 0.25.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:45:01 2019 (1556747101)
Build timestamp: 1556747101
Build timestamp as int: 1556747101
Feature requests: what underlying problem are you trying to solve with this feature?
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
What operating system are you running Bazel on?
What's the output of
bazel info release
?If
bazel info release
returns "development version" or "(@non-git)", tell us how you built Bazel.What's the output of
git remote get-url origin ; git rev-parse master ; git rev-parse HEAD
?Have you found anything relevant by searching the web?
Any other information, logs, or outputs that you want to share?
The text was updated successfully, but these errors were encountered: