Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++ Coverage on Linux is broken #9406

Closed
ruifangChen opened this issue Sep 19, 2019 · 46 comments
Closed

C++ Coverage on Linux is broken #9406

ruifangChen opened this issue Sep 19, 2019 · 46 comments
Labels
coverage P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug

Comments

@ruifangChen
Copy link

ATTENTION! Please read and follow:

  • if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to [email protected]
  • if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

When I execute the bazel coverage command with version 0.24.1, I can generate a valid coverage.dat file, but when using a version higher than 0.24.1, the coverage.dat file is empty.
This problem exists only when generating coverage for the C++ language.

$ ll bazel-testlogs/id_test/
total 3136
drwxr-xr-x 2 admin admin 4096 Sep 19 15:29 ./
drwxr-xr-x 3 admin admin 4096 Sep 19 15:07 ../
-r-xr-xr-x 1 admin admin 581600 Sep 19 15:07 baseline_coverage.dat*
-r-xr-xr-x 1 admin admin 68477 Sep 19 15:29 coverage.dat*
-r-xr-xr-x 1 admin admin 179 Sep 19 15:29 test.cache_status*
-r-xr-xr-x 1 admin admin 2539566 Sep 19 15:29 test.log*
-rw-r--r-- 1 admin admin 904 Sep 19 15:28 test.xml
$ bazel version
Build label: 0.24.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Apr 2 16:29:26 2019 (1554222566)
Build timestamp: 1554222566
Build timestamp as int: 1554222566

$ ll bazel-testlogs/id_test/
total 4108
drwxr-xr-x 2 admin admin 4096 Sep 19 15:25 ./
drwxr-xr-x 3 admin admin 4096 Sep 19 15:07 ../
-r-xr-xr-x 1 admin admin 581600 Sep 19 15:07 baseline_coverage.dat*
-r-xr-xr-x 1 admin admin 0 Sep 19 15:25 coverage.dat*
-r-xr-xr-x 1 admin admin 176 Sep 19 15:25 test.cache_status*
-r-xr-xr-x 1 admin admin 3600923 Sep 19 15:25 test.log*
-rw-r--r-- 1 admin admin 904 Sep 19 15:25 test.xml
$ bazel version
Build label: 0.25.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:45:01 2019 (1556747101)
Build timestamp: 1556747101
Build timestamp as int: 1556747101

Replace this line with your answer.

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Replace this line with your answer.

What operating system are you running Bazel on?

Replace this line with your answer.

What's the output of bazel info release?

Replace this line with your answer.

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Replace this line with your answer.

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

Replace this line with your answer.

Have you found anything relevant by searching the web?

Replace these lines with your answer.

Places to look:

Any other information, logs, or outputs that you want to share?

Replace these lines with your answer.

If the files are large, upload as attachment or provide link.

@EFanZh
Copy link

EFanZh commented Sep 19, 2019

I have a similar problem.

Try to run this script in an empty directory:

echo -n > WORKSPACE
echo 'cc_test(name = "foo_test", srcs = ["main.cpp"])' > BUILD
echo 'int main() {}' > main.cpp
bazel coverage //:foo_test --instrument_test_targets
tail -n 1 bazel-testlogs/foo_test/test.log

On my PC, the script above prints the following log:

INFO: Using default value for --instrumentation_filter: "//".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed target //:foo_test (21 packages loaded, 478 targets configured).
INFO: Found 1 test target...
Target //:foo_test up-to-date:
  bazel-bin/foo_test
INFO: Elapsed time: 2.936s, Critical Path: 2.70s
INFO: 7 processes: 6 linux-sandbox, 1 worker.
INFO: Build completed successfully, 15 total actions
//:foo_test                                                              PASSED in 0.2s

Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 15 total actions
WARNING: There was no coverage found.

Note that the last line says there was no coverage.

My environment:

  • OS: Arch Linux
  • Bazel version: 0.29.1
  • GCC version: 9.1.0

@ruifangChen
Copy link
Author

All versions larger than 0.24.1 have this problem.
My environment:
OS: GNU/Linux
Bazel version: larger than 0.24.1
GCC version: 4.9.2

@EFanZh
Copy link

EFanZh commented Sep 20, 2019

It seems that in GCC 9.1.0, the meaning of the -i flag of gcov has changed:

Instead of generating a “.gcov” file, a “.gcov.json.gz” file is generated. This change may cause my bazel failing to collect coverage data.

@EFanZh
Copy link

EFanZh commented Sep 20, 2019

I also tested GCC 5.4.0, and noticed that the gcov log doesn’t contain “Creating” text so the following grep command will fail:

cat "$gcov_log" | grep "Creating" | cut -d " " -f 2 | cut -d"'" -f2 | \

So for now, I see two problems:

  1. If the version of GCC is greater than or equal to 5.4, the output of executing gcov does not contain “Creating” text, so this line should be updated:

    cat "$gcov_log" | grep "Creating" | cut -d " " -f 2 | cut -d"'" -f2 | \

  2. If the version of GCC is greater than or equal to 9.1, the meaning of -i flag of gcov has changed, instead of generating a “.gov” file, a “.gcov.json.gz” file is generated. So the collecting logic should be updated as well.

@ruifangChen
Copy link
Author

@EFanZh, Have you tried gcc version 4.9.2? I am using this version with bazel 0.25.0 to generate an empty coverage.dat file.

@EFanZh
Copy link

EFanZh commented Sep 20, 2019

No, I haven’t tried GCC 4.9.2 or Bazel 0.25.0. But you can find the files matching “*.gcov.log” in Bazel cache directory, and see their content. If there is no “Creating” text, then Bazel cannot generate coverage reports.

@ruifangChen
Copy link
Author

I can't find any files mathes “.gcov.log” or ".gcov" in Basel cache directory. All versions larger than 0.24.1 are the same problem. If you are convenient, you can try gcc 4.9.2 with bazel 0.29.1 version.

@EFanZh
Copy link

EFanZh commented Sep 23, 2019

Is there a “test.log” file in your “bazel-testlogs” directory?

@ruifangChen
Copy link
Author

Yes, there is a "test.log" file and there is "creating" text in the file. But the "coverage.dat" file is empty, when generated coverage with bazel version higher than 0.24.1.

$ ll bazel-testlogs/id_test/
total 4068
drwxr-xr-x 2 admin admin 4096 Sep 23 11:39 ./
drwxr-xr-x 4 admin admin 4096 Sep 20 20:34 ../
-r-xr-xr-x 1 admin admin 581600 Sep 19 15:07 baseline_coverage.dat*
-r-xr-xr-x 1 admin admin 0 Sep 23 11:38 coverage.dat*
-r-xr-xr-x 1 admin admin 176 Sep 23 11:39 test.cache_status*
-r-xr-xr-x 1 admin admin 3563900 Sep 23 11:39 test.log*
-rw-r--r-- 1 admin admin 912 Sep 23 11:38 test.xml

Some logs are as follows:

  • grep Creating
  • cut '-d'''' -f2
  • read gcov_file
  • cut -d ' ' -f 2
  • read gcno_path
    ++ dirname bazel-out/k8-opt/bin/_objs/id_test/id_test.pic.gcno
    ++ basename bazel-out/k8-opt/bin/_objs/id_test/id_test.pic.gcno .gcno
  • local gcda=/home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/id_test/id_test.pic.gcda
  • [[ -f /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/id_test/id_test.pic.gcda ]]
  • read gcno_path
    ++ dirname bazel-out/k8-opt/bin/_objs/ray_common/client_connection.gcno
    ++ basename bazel-out/k8-opt/bin/_objs/ray_common/client_connection.gcno .gcno
  • local gcda=/home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.gcda
  • [[ -f /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.gcda ]]
  • read gcno_path
    ++ dirname bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno
    ++ basename bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno .gcno
  • local gcda=/home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcda
  • [[ -f /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcda ]]
  • '[' '!' -f /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno ']'
    ++ dirname bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno
  • mkdir -p /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common
  • cp /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcno
    ++ dirname /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcda
  • /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/gcov -i -o /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/bazel-out/k8-opt/bin/_objs/ray_common/client_connection.pic.gcda
  • cat /home/admin/.cache/bazel/_bazel_admin/9386c25d2bda9cfe2c78d99fea7976b6/sandbox/processwrapper-sandbox/1/execroot/com_github_ray_project_ray/_coverage/id_test/test/_cc_coverage.gcov.gcov.log
  • cut -d ' ' -f 2
  • read gcov_file
  • cut '-d'''' -f2
  • grep Creating
  • read gcno_path

@EFanZh
Copy link

EFanZh commented Sep 23, 2019

The coverage tests are run in a sandbox environment. Add “--sandbox_debug” to the command line that runs the coverage test, so that the log files are kept in its original position.

@ruifangChen
Copy link
Author

Using this parameter does not work either. The generated coverage files are empty whether in a sandbox or standalone environment.
You can try to collect c++ language coverage in gcc 4.9.2 and bazel 0.29.1 environments to see if the generated coverage.dat file is empty.

@EFanZh
Copy link

EFanZh commented Sep 23, 2019

That flag is for debugging. It won’t affect the outcome. Now you should be able to find “*.gcov.log” files in your Bazel cache after adding the flag.

@ruifangChen
Copy link
Author

Even with this parameter, no ".gcov.log" file is generated by converage command in the bazel cache dir.
find ./bazel-ray/ -name "
.gcov.log"

@EFanZh
Copy link

EFanZh commented Sep 23, 2019

Try this command:

find "$(bazel info output_base)" -name '*.gcov.log'

@ruifangChen
Copy link
Author

With this command, i find the *.gcov.log file and there is not "creating" text in the file.

@ruifangChen
Copy link
Author

@ishikhman
If it is determined that this is a bug, then when will it be possible to solve this problem? Because we are now generating c++ coverage for the project, but the bazel version we are using now is 0.26.1.

@ishikhman
Copy link
Contributor

If it is determined that this is a bug, then when will it be possible to solve this problem? Because we are now generating c++ coverage for the project, but the bazel version we are using now is 0.26.1.

@hlopko could you please help @ruifangChen ? Thanks!

@xingxinghuo1000
Copy link

0.25 0.26 0.28, all of these versions have the same problem

@xingxinghuo1000
Copy link

0.25 0.26 0.28, all of these versions have the same problem

As well as 0.29.1

@ruifangChen
Copy link
Author

@hlopko Hi, could you give me some help? How to get C++ coverage with bazel 0.26.1 and gcc 4.9.2. Thanks very much!

@hlopko hlopko added P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed untriaged labels Oct 14, 2019
@AxelZvb
Copy link

AxelZvb commented Nov 1, 2019

Hi,
I have check why coverage was not generating output for me. As @EFanZh mentioned "Creating" is not mentioned in the gcov output. For me (gcov 8.3.0) the "--json-format" is not included yet.
To resolve the empty .dat files for me I made a branch (AxelZvb@418e7f0) that check the gcov version, and changes its file collection in accordance.
0.0<version<5.4 = old method
5.4<version<9.1 = get the .gcov file
9.1<version = get the .gcov.json.gz file
I currently am not sure where the change to the gcov output happened, as I cannot find a specific mention in the man pages of the gcov versions.

This branch also activates the -b (branch) option for gcov as the described bug is resolved in gcov 8+. Now i see that gcov generates branch output but now it seems that the lcov_merger removes the added branch data again (data can be seen in the gcov output but not in the final .dat files). This should be updated in the lcov_merger and along with supporting the new .gcov.json.gz files.

@ulfjack
Copy link
Contributor

ulfjack commented Apr 20, 2020

There's some in the gcov docs:
https://gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html

@ulfjack
Copy link
Contributor

ulfjack commented Apr 23, 2020

I did some experiments with it on Tuesday and it's very similar to the old format, just in Json, so it should be possible to extend Bazel's coverage merger to load it into its in-memory structures (which are similar to the old format).

@panzhongxian
Copy link
Contributor

linux-test-project/lcov#58

There is an open issue in lcov project. Do these two issues have any relation?

@lizan
Copy link

lizan commented May 29, 2020

FWIW, Envoy switched to use Clang based solution by using llvm-cov export, which works well in recent versions of Clang+LLVM. envoyproxy/envoy#10909

helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format in parallel to the existing GCov parser

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format in parallel to the existing GCov parser

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: bazelbuild#9406
helaan added a commit to helaan/bazel that referenced this issue Jun 2, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: bazelbuild#9406
bazel-io pushed a commit that referenced this issue Jun 16, 2020
GCC 9 changed the intermediate format to a JSON based format and with it
changed the meaning of the `-i` flag. Because of these changes it was not
possible to generate code coverage with GCC 9. This patch addresses that
by adding its format next to the existing GCov parser.

Addresses: #9406

Closes #11538.

PiperOrigin-RevId: 316641962
@xingxinghuo1000
Copy link

xingxinghuo1000 commented Jul 2, 2020

GCC 9.2.1 and bazel 3.3.1 is still not working. Got zero-sized coverage.dat.
Thank you for the fix! @helaan Do I use the right version?

bazel version
Build label: 3.3.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 30 15:09:23 2020 (1593529763)
Build timestamp: 1593529763
Build timestamp as int: 1593529763
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/9/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --without-isl --enable-gnu-indirect-function --enable-cet=auto --with-tune=generic --build=x86_64-redhat-linux
Thread model: posix
gcc version 9.2.1 20190812 (Alibaba 9.2.1-2 2.17) (GCC) 
bazel coverage //...
INFO: Using default value for --instrumentation_filter: "//".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed 3 targets (22 packages loaded, 561 targets configured).
INFO: Found 1 target and 2 test targets...
INFO: Elapsed time: 11.613s, Critical Path: 6.69s
INFO: 27 processes: 26 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 49 total actions
//:function_1_test                                                       PASSED in 1.1s
//:function_2_test                                                       PASSED in 1.1s

Executed 2 out of 2 tests: 2 tests pass.
INFO: Build completed successfully, 49 total actions
find bazel-testlogs/ -name coverage.dat | xargs  ls -laa
-r-xr-xr-x 1 root root 0 Jul  2 21:43 bazel-testlogs/function_1_test/coverage.dat
-r-xr-xr-x 1 root root 0 Jul  2 21:43 bazel-testlogs/function_2_test/coverage.dat

@helaan
Copy link
Contributor

helaan commented Jul 2, 2020

@xingxinghuo1000 Bazel 3.3.1 does not include the changes I made.

oliverlee added a commit to oliverlee/recorder that referenced this issue Jul 14, 2020
Define configuration options for collecting coverage data with Bazel.

The coverage command for Bazel is fairly fragile and many configurations
do not work [1,2]. The `--javabase` option is set as suggested in this
bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu
18.04.

[1]: bazelbuild/bazel#9406
[2]: bazelbuild/bazel#10457
[3]: bazelbuild/bazel#7953
oliverlee added a commit to oliverlee/recorder that referenced this issue Jul 14, 2020
Define configuration options for collecting coverage data with Bazel.

The coverage command for Bazel is fairly fragile and many configurations
do not work [1,2]. The `--javabase` option is set as suggested in this
bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu
18.04.

[1]: bazelbuild/bazel#9406
[2]: bazelbuild/bazel#10457
[3]: bazelbuild/bazel#7953
@MikhailTymchukFT
Copy link

I experience the very same issue of non-working coverage on MacOS and Java tests:

$ bazel version
Build label: 3.4.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jul 14 06:32:14 2020 (1594708334)
Build timestamp: 1594708334
Build timestamp as int: 1594708334
$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.59)
Target: x86_64-apple-darwin19.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ bazel coverage //...
Starting local Bazel server and connecting to it...
INFO: Using default value for --instrumentation_filter: "//".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed 7 targets (24 packages loaded, 589 targets configured).
INFO: Found 5 targets and 2 test targets...
INFO: Elapsed time: 49.419s, Critical Path: 32.22s
INFO: 10 processes: 3 remote cache hit, 3 darwin-sandbox, 4 worker.
INFO: Build completed successfully, 11 total actions
//:reference_test                                               (cached) PASSED in 2.8s
//:target_test                                                           PASSED in 1.1s

Executed 1 out of 2 tests: 2 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout
INFO: Build completed successfully, 11 total actions
$ find bazel-testlogs/ -name coverage.dat | xargs  ls -la
-r-xr-xr-x  1 mikhailtymchuk  wheel  0 Jul 15 23:43 bazel-testlogs//reference_test/coverage.dat
-r-xr-xr-x  1 mikhailtymchuk  wheel  0 Jul 16 20:26 bazel-testlogs//target_test/coverage.dat

Am I cooking it right?

@xingxinghuo1000
Copy link

@xingxinghuo1000 Bazel 3.3.1 does not include the changes I made.

I tried 3.4.1. It works well. Thanks 👍

@xingxinghuo1000
Copy link

xingxinghuo1000 commented Jul 27, 2020

bazel 3.4.1 works well on my hello world demo project.

But in real project, on some test targets, an error occured when collecting coverage by " bazel coverage //XXX"

gcc: 9.2.1

lcov tool version:
I tried both current master version and 1.14

bazel version 3.4.1

Jul 27, 2020 6:19:32 AM com.google.devtools.coverageoutputgenerator.Main main
SEVERE: Unhandled exception on lcov tool: com.google.gson.JsonSyntaxException: com.google.gson.JsonSyntaxException: com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 8.00292e+09 at line 1 column 6080139
Jul 27, 2020 6:19:33 AM com.google.devtools.coverageoutputgenerator.Main lambda$parseFilesInParallel$0
INFO: Parsing file /root/.cache/bazel/_bazel_root/****/execroot/****/_coverage/src/ranger/XXX_test/test/YYY.pic.gcda.gcov.json.gz

@helaan Could you please look at the error above? Thank you.

I wrote a simple python script to resolve all the json-format gcov data. I DID found data like "8.00292e+09"
But "json.loads" in python can resolve this json data( scientific notation ).

import gzip
import json
import sys

file_name = sys.argv[1]
with gzip.open(file_name, 'rt') as f:
    text = f.read()
    try:
        a = json.loads(text)
    except:
        print("Error when parse json, file is :", file_name, "\n text:", text)
    if 'e+0' in text:
        print(text)

Is this a bug?

I Guess the question is: GsonBuiler().setLongSerializationPolicy(LongSerializationPolicy.STRING) could be the solution
compatible with scientific notation

@ulfjack @lberki FYI

@oquenchil
Copy link
Contributor

It looks like this was fixed.

oliverlee added a commit to oliverlee/recorder that referenced this issue Nov 21, 2020
Define configuration options for collecting coverage data with Bazel.

The coverage command for Bazel is fairly fragile and many configurations
do not work [1,2]. The `--javabase` option is set as suggested in this
bug report[3]. Coverage has been manually tested with GCC 7 on Ubuntu
18.04.

[1]: bazelbuild/bazel#9406
[2]: bazelbuild/bazel#10457
[3]: bazelbuild/bazel#7953
@111qqz
Copy link

111qqz commented Mar 10, 2021

bazel 4.0 + gcc 7.3 still not work.
no coverage.dat file generated at all

@oquenchil oquenchil reopened this Mar 10, 2021
@oquenchil
Copy link
Contributor

Thank you.

It looks like this is [tested](https://cs.opensource.google/bazel/bazel/+/master:src/test/shell/bazel/bazel_cc_code_coverage_test.sh;l=298.

Right now we are not prioritizing work in this area. If you isolate the problem to a bug in Bazel instead of a problem in your environment, please feel free to send a patch if you figure out what's wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
coverage P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug
Projects
None yet
Development

No branches or pull requests