Skip to content

gcc: prevent runtime references via __FILE__ but in a reversible manner#279455

Merged
symphorien merged 1 commit intoNixOS:stagingfrom
symphorien:dont-mangle-debuginfo
Jan 8, 2024
Merged

gcc: prevent runtime references via __FILE__ but in a reversible manner#279455
symphorien merged 1 commit intoNixOS:stagingfrom
symphorien:dont-mangle-debuginfo

Conversation

@symphorien
Copy link
Member

the mangling done by mangle-NIX_STORE-in-FILE.patch also applies to source paths embedded in debug symbols. When putting a breakpoint in a template instanciation from another lib, the path that gdb looks for is therefore mangled (/nix/store/eeeeeee;...-the-lib-dev/include/foo.h) This severely degrades the debugging experience. To alleviate that, it's possible to make the mangling reversible: a debuginfod server can then reverse the mangling. I plan to implement that in nixseparatedebuginfod. The reversible mangling that was chosen in making the hash of the store path uppercase.

cc @trofi @Artturin #255192

will undraft after I rebuild gcc13

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

the mangling done by mangle-NIX_STORE-in-__FILE__.patch also applies to
source paths embedded in debug symbols. When putting a breakpoint in a
template instanciation from another lib, the path that gdb looks for is
therefore mangled (/nix/store/eeeeeee;...-the-lib-dev/include/foo.h)
This severely degrades the debugging experience. To alleviate that, it's
possible to make the mangling reversible: a debuginfod server can then
reverse the mangling. I plan to implement that in nixseparatedebuginfod.
The reversible mangling that was chosen in making the hash of the store
path uppercase.
@symphorien symphorien requested a review from a user January 7, 2024 21:23
@symphorien symphorien marked this pull request as draft January 7, 2024 21:23
@ofborg ofborg bot added 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Jan 8, 2024
@symphorien symphorien changed the base branch from master to staging January 8, 2024 20:05
@symphorien symphorien marked this pull request as ready for review January 8, 2024 20:06
@symphorien symphorien merged commit ae17622 into NixOS:staging Jan 8, 2024
@symphorien symphorien changed the title Draft: gcc: prevent runtime references via __FILE__ but in a reversible manner gcc: prevent runtime references via __FILE__ but in a reversible manner Jan 8, 2024
@symphorien
Copy link
Member Author

hum the merge was a misclick. Given that it was already approved I'll not open a revert right now but feel free to request one. Sorry

@Cynerd
Copy link
Contributor

Cynerd commented Jan 31, 2024

This patch breaks coverage. The simple example would be:

$ cat main.c                                                                                                                                                                         flake-rewqork*%=
#include <stdio.h>

int main() {
	printf("Hello\n");
	return 0;
}
$ gcc --coverage main.c                                                                                                                                                              flake-rewqork*%=
$ ./a.out                                                                                                                                                                            flake-rewqork*%=
Hello
$ gcovr .                                                                                                                                                                            flake-rewqork*%=
(ERROR) GCOV produced the following errors processing /home/cynerd/elektroline/template-c/b/a-main.gcda:
	Cannot open source file /nix/store/3MMVGB08QY8N6N37MNPRF77FNP4RSSI9-glibc-2.38-27-dev/include/bits/stdio2.h

	(gcovr could not infer a working directory that resolved it.)
To ignore this error use option --gcov-ignore-errors=no_working_dir_found.
Traceback (most recent call last):
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 80, in worker
    work(*args, **kwargs)
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/gcov.py", line 351, in process_datafile
    raise RuntimeError(errors_output)
RuntimeError: GCOV produced the following errors processing /home/cynerd/elektroline/template-c/b/a-main.gcda:
	Cannot open source file /nix/store/3MMVGB08QY8N6N37MNPRF77FNP4RSSI9-glibc-2.38-27-dev/include/bits/stdio2.h

	(gcovr could not infer a working directory that resolved it.)
To ignore this error use option --gcov-ignore-errors=no_working_dir_found.
Traceback (most recent call last):
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 80, in worker
    work(*args, **kwargs)
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/gcov.py", line 351, in process_datafile
    raise RuntimeError(errors_output)
RuntimeError: GCOV produced the following errors processing /home/cynerd/elektroline/template-c/b/a-main.gcda:
	Cannot open source file /nix/store/3MMVGB08QY8N6N37MNPRF77FNP4RSSI9-glibc-2.38-27-dev/include/bits/stdio2.h

	(gcovr could not infer a working directory that resolved it.)
To ignore this error use option --gcov-ignore-errors=no_working_dir_found.
(ERROR) Uncaught EXCEPTION
Traceback (most recent call last):
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/bin/.gcovr-wrapped", line 9, in <module>
    sys.exit(main())
             ^^^^^^
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/__main__.py", line 328, in main
    covdata = collect_coverage_from_gcov(options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/__main__.py", line 380, in collect_coverage_from_gcov
    with Workers(
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 173, in __exit__
    self.wait()
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 164, in wait
    raise self.exceptions[0][1]
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/__main__.py", line 387, in collect_coverage_from_gcov
    contexts = pool.wait()
               ^^^^^^^^^^^
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 164, in wait
    raise self.exceptions[0][1]
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/workers.py", line 80, in worker
    work(*args, **kwargs)
  File "/nix/store/8dn974lnyrn3qgbmckax1s8wmdjbzh6r-python3.11-gcovr-6.0/lib/python3.11/site-packages/gcovr/gcov.py", line 351, in process_datafile
    raise RuntimeError(errors_output)
RuntimeError: GCOV produced the following errors processing /home/cynerd/elektroline/template-c/b/a-main.gcda:
	Cannot open source file /nix/store/3MMVGB08QY8N6N37MNPRF77FNP4RSSI9-glibc-2.38-27-dev/include/bits/stdio2.h

	(gcovr could not infer a working directory that resolved it.)
To ignore this error use option --gcov-ignore-errors=no_working_dir_found.

Edit: @symphorien just for my curiosoty, why isn't stripping enough?

@trofi
Copy link
Contributor

trofi commented Jan 31, 2024

nixpkgs' gcovr would probably need to apply the same reversion mechanism to resolve back to filenames (or optionally it could be fixed to not crash on non-existent files by producing limited coverage details only for resolved sources).

@Cynerd
Copy link
Contributor

Cynerd commented Jan 31, 2024

nixpkgs' gcovr would probably need to apply the same reversion mechanism to resolve back to filenames (or optionally it could be fixed to not crash on non-existent files by producing limited coverage details only for resolved sources).

Ok but is that a correct approach? Shouldn't there be rather a patch that reverts it directly for the gcno files? To support this you would need not only change in gcovr but also in lcov and those are only tools I know about, but might not be everything.

@trofi
Copy link
Contributor

trofi commented Jan 31, 2024

One thing that gets me confused is that originally __FILE__ mangling was expected to be applied to only -fmacro-prefix-map=, but not -fdebug-prefix-map= (which this PR tries to rectify) and not -fprofile-prefix-map= (which I presume breaks gcovr).

This probably comes from the fact that if (maps == macro_prefix_maps) is incorrect when macro_prefix_maps == NULL and debug_prefix_maps with profile_prefix_maps are NULL which aliases them by accident.

I'll try to come up with something that only affects -fmacro-prefix-map=. That should repair both debugging and profiling info.

@trofi
Copy link
Contributor

trofi commented Feb 1, 2024

Proposed the tweak as:

Now it only affects -fmacro-prefix-map= (explicit __FILE__ & co usage in runtime expressions). Debugging and Coverage data should work now. I did a very superficial test for it and it seems to work. Please give it a try.

@Cynerd
Copy link
Contributor

Cynerd commented Feb 2, 2024

@trofi thank you. I tested coverage generation with your patch (your nixpkgs branch exactly) and it works as expected. 👍

@symphorien symphorien mentioned this pull request Jul 12, 2025
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants