Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiler: Fix some real and theoretical miscompilations with allowzero and volatile #21882

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

alexrp
Copy link
Member

@alexrp alexrp commented Nov 2, 2024

Also some miscellaneous fixes while I was mucking about in the affected code.

Closes #15816.

@alexrp
Copy link
Member Author

alexrp commented Nov 2, 2024

cc @Rexicon226 for RISC-V bits.

src/arch/riscv64/CodeGen.zig Outdated Show resolved Hide resolved
src/arch/riscv64/CodeGen.zig Outdated Show resolved Hide resolved
AstGen requires inline assembly to either have outputs or be marked volatile, so
there doesn't appear to be any point in doing these checks.
These can all potentially operate on volatile pointers.
…inters.

This informs optimization passes that they shouldn't assume that a load from a
null pointer invokes undefined behavior.

Closes ziglang#15816.
…spaces.

LLVM considers null pointers to be valid for such address spaces.
Also fix some cases where we were being overzealous in applying volatile.
Copy link
Member

@andrewrk andrewrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test coverage is far more important than the fixes. Please don't land this without full test coverage for the changed behavior.

Consider that this code would all change if some of these proposals being actively considered are implemented. Your fixes will be lost, however your test coverage would remain and be translated to the new language semantics, thus preserving your efforts here.

@alexrp
Copy link
Member Author

alexrp commented Nov 3, 2024

@andrewrk I'd love to add test coverage, but I'm not actually sure how. allowzero and volatile are by nature pretty tough to test outside of some kind of freestanding environment.

One possible way to test is to verify the generated IR, but do we have precedent for this?

@andrewrk
Copy link
Member

andrewrk commented Nov 3, 2024

Ah, yes. There is no precedent for this, so it is nontrivial to add tests. Still important, though.

Testing the generated machine code is certainly an option. That testing infrastructure would also be valuable for covering non-regression of optimizations.

Another way to test volatile is to test in an optimized mode and run the code in a VM. For example if you output webassembly code, the VM can verify that a given memory load or store on the expected address did in fact occur.

@Rexicon226
Copy link
Contributor

Testing the generated machine code is certainly an option. That testing infrastructure would also be valuable for covering non-regression of optimizations.

I definitely agree with this, we'll need it for our own optimization framework anyways.

@alexrp
Copy link
Member Author

alexrp commented Nov 3, 2024

So to be clear, most of the changes here are to the LLVM backend, with just a couple of necessary fixes to the self-hosted ones to get tests passing due to the AIR mustLower() changes. The self-hosted backends still have many violations of volatile semantics and atomic ordering side effects, for example. I'll get to those eventually, but fixing them here would increase the scope of this PR too much. So, I wouldn't say that adding testing for the self-hosted backends is essential for merging this PR in particular.

Testing for the LLVM backend does seem reasonable for this PR, though. But here I would argue we don't want to test the machine code output because that's going to change with every LLVM release (even patch releases). Similarly, verifying post-optimization LLVM IR would be prone to frequent changes. I think the right thing to do here is to verify the LLVM IR we generate, i.e. that we're conforming to the LLVM language reference. What happens from there is on LLVM.

Another way to test volatile is to test in an optimized mode and run the code in a VM. For example if you output webassembly code, the VM can verify that a given memory load or store on the expected address did in fact occur.

Are there any existing VMs we can reuse for this, or would this involve writing our own test VM? It sounds useful for hard-to-test features like this in any case, and something I'd actually like to work on, but I think I'd like to do it as follow-up work and stick to verifying LLVM IR output for this PR.

@xxxbxxx
Copy link
Contributor

xxxbxxx commented Nov 4, 2024

drive-by idea to test 'volatile': using ptrace and hardware debug registers to count mem accesses

tried it (in C, because the linux.user struct insn't in std yet):
https://gist.github.com/xxxbxxx/903349b1568cb86cfaa96cc62d85a972

also found this example that seem to support arm as well:
https://android.googlesource.com/platform/bionic/+/master/tests/sys_ptrace_test.cpp

@alexrp
Copy link
Member Author

alexrp commented Nov 4, 2024

drive-by idea to test 'volatile': using ptrace and hardware debug registers to count mem accesses

That's a neat idea. I think if we want to test the run-time semantics, I would still prefer the WebAssembly VM idea just because it's more portable and the thing we're testing isn't actually target-specific. Can probably write a bare-bones VM in 1-2k lines of code, or even less if we limit it to just what we need for testing.

We also need to be able to test allowzero; I'm not sure how possible that is with the ptrace() approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storing to an allowzero zero pointer is not emitted in ReleaseFast build mode
4 participants