[WebAssembly] Implement GlobalISel

[QuantumSegfault]

A WIP attempt to implement the new GlobalISel system for the WebAssembly backend.

It may not end up being worth the effort in the short term, but hopefully if it works out, it will open the doors to improved code maintainability and future GlobalISel-exclusive optimizations.

Based largely on the AArch GlobalISel, combined with the old SelectionDAG-based (WASM) code.

---

This PR isn't to be merged. This is just a development branch, and will be closed once the smaller PRs have been merged and main has been brought up to speed.

Partial PRs:

1. [WebAssembly][GlobalISel] Part 1 - Setup skeleton #178796
2. [WebAssembly][GlobalISel] Part 2 - CallLowering lowerFormalArguments #180263
3. more to come...

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[QuantumSegfault]

@dschuff @sparker-arm

If you want to keep tabs on it, or have any suggestions.

Probably going to a little while before there's anything worth seriously reviewing.

[QuantumSegfault]

@dschuff

I'm getting close to having a MVP for you to review. But in the meanwhile, I have a question (if you don't know, maybe you'll at least know who does).

I'm working on the instruction selection phase currently. The compatibility layer works pretty well for importing the existing (SelectionDAG) tblgen patterns to work with GISel. So far one of the biggest issues is that my p0 (LLT; pointers in address space 0) aren't getting matched existing patterns expecting an i32/i64 (LLT s32 or s64?). For instance, G_CONSTANT matches to CONST_I32/CONST_I64 when giving a constant s32 or s64, but fails when providing a p0 (e.g. 0 to represent null pointer).

What's the best way to fix this. Do I just have to manually implement select for such operations, or is there a way to tell SelectionDAGCompat to treat p0 as either i32/s32 or i64/s64 depending on the current address size?

Note that regbankselect is already putting them in i32regbank or i64regbank as appropriate, so that's working. Its just isel that isn't matching them.

[dschuff]

Yeah, unfortunately I don't know that much about GIsel specifically. This sounds like a good question to bring up on Discourse.

[ppenzin]

@topperc you might find this interesting.

[QuantumSegfault]

@dschuff
Alright! I've gotten it into a decent state.

It's obviously not fully complete, but I think its far along enough to warrant a review (does that mean I should un-draft it?). I'm not entirely sure what LLVM's policy is on PR size/completeness, but I get the strong impression the GlobalISel implementation doesn't need to be 100% the first go.

Feature wise, here's a list of major things still missing (to my recollection)

• f16 - need to figure out how to properly legalize/lower the FPEXT and FPTRUNC call
• i128/wide-arithmetic
• Complete SwiftCC support - currently there's some issue somewhere with alloca ptr swifterror
• Bulk memory - need to figure out how to port the CFG triangle over, and figure out the return type on the libcall variant
• Atomics
• SIMD - I have very little understanding of how the vectors fit in across the board, so that might be a job better suited to somebody else
• Error handling
• Intrinsics
• FMAXNUM/FMINNUM (and _IEEE variants?)
• WASM Global (addrspace 1) access
• WASM table operations
• Reference types (funcref, exnref, externref)

Besides those, there are other things that still need adding.

• Alternate instruction mappings for regbanks (so that floating-point loads are lowered directly instead of becoming e.g. i32.load + f32.reinterpret_i32) is one.
• Optimization is another. Luckily I think most of the opts currently missing are builtin GISel combine patterns, they just have to be enabled/configured
• Tests. Currently there are absolutely no new tests. I'm not sure exactly what to test, and in how much depth.

Which of these do you want done before a review, and which do you think should be complete for merging this initial PR.

---

My main focus was to get this in a functional state as quickly as possible, so there's undoubtedly plenty of room for cleanup. Call lowering in particular needs addressing, as I had to copy & paste private code from elsewhere rather than use the higher-level public API, due to WASM's unique value-stack [no phys reg, no stack spilling].

This is also basically my first time working with significantly with the LLVM codebase (short of that one small PR and issue last month), so I'm learning along the way. There seem to be lots of ways to accomplish certain tasks, so it sometimes gets confusing what the correct/optimal approach is.

---

I suppose this is a good a time as any to ask, is there still interest in all of this? How far along does this need to get before others will start working with and improving it?

[QuantumSegfault]

I wonder how much #159880 and the already merged #158269 would do to help me remove the custom patterns for p0.

It just feels like I'm missing something simple to say p0 goes in either I32 or I64 depending on HwMode. Doesn't seem like this new stuff addresses that.

[github-actions]

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:

git-clang-format --diff origin/main HEAD --extensions cpp,h -- llvm/lib/Target/WebAssembly/GISel/WebAssemblyCallLowering.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyCallLowering.h llvm/lib/Target/WebAssembly/GISel/WebAssemblyInstructionSelector.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyLegalizerInfo.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyLegalizerInfo.h llvm/lib/Target/WebAssembly/GISel/WebAssemblyO0PreLegalizerCombiner.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyPostLegalizerCombiner.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyPreLegalizerCombiner.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.cpp llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.h llvm/lib/Target/WebAssembly/WebAssembly.h llvm/lib/Target/WebAssembly/WebAssemblySubtarget.cpp llvm/lib/Target/WebAssembly/WebAssemblySubtarget.h llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp --diff_from_common_commit

⚠️
The reproduction instructions above might return results for more than one PR
in a stack if you are using a stacked PR workflow. You can limit the results by
changing origin/main to the base branch/commit you want to compare against.
⚠️

View the diff from clang-format here.

diff --git a/llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.cpp b/llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.cpp
index cd6ca9b72..7f9fbe3f5 100644
--- a/llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.cpp
+++ b/llvm/lib/Target/WebAssembly/GISel/WebAssemblyRegisterBankInfo.cpp
@@ -279,9 +279,9 @@ WebAssemblyRegisterBankInfo::getInstrMapping(const MachineInstr &MI) const {
   case G_INTRINSIC:
   case G_INTRINSIC_W_SIDE_EFFECTS: {
     switch (cast<GIntrinsic>(MI).getIntrinsicID()) {
-      case Intrinsic::clear_cache:
-        reportFatalUsageError("llvm.clear_cache is not supported on wasm");
-        break;
+    case Intrinsic::clear_cache:
+      reportFatalUsageError("llvm.clear_cache is not supported on wasm");
+      break;
     }
     return getInstructionMapping(DefaultMappingID, /*Cost=*/1, nullptr, 0);
   }

[QuantumSegfault]

@dschuff

Have you given any more thought to if, when, and how you want to begin reviewing and working towards merging this in?

[QuantumSegfault]

@dschuff

Are you still contemplating this (still interested?)? It's in a state where its not quite "finished" or feature complete yet, but seems to be working well for a good chunk of the WASM CodeGen testsuite. It's far along enough I feel at least a preliminary review would be helpful. Seems more complete than some other GISel implementations in it's current state, but some of the major pieces still missing are:

• fp16
• int128
• SwiftCC
• Atomics
• True SIMD support. Right now all vector ops are scalarized unconditionally
• Exception handling
• Intrinsics
• WASM specific features including funcref, tables, and addrspace 1 globals.
• Optimizations (combiners)
• Test suite - right now I'm checking codegen against the existing tests; no new tests are in place yet

---

And there's also the question of breaking it up into smaller PRs vs keeping it monolithic.

[dschuff]

Thanks for pinging on this again.
I think it's fair to say that I have interest, but also general concerns about GISel. Let me try to lay them out.

Mostly I just want to ensure that the benefits outweigh the costs.

In the long term, it seems likely that we'll want to have GISel support, but "long term" could be very long. GISel is now about 10 years old and AFAIK it's still the case that no backend is fully switched (at least, none that wasn't GISel only from the start). In the meantime there is a cost to understanding and reviewing the code as it goes in, and a maintenance cost to having the code around (especially if GISel infrastructure has more churn compared to other LLVM code as it matures). And there is risk of it becoming unmaintained or under-maintained.

As for benefits, they seem fairly marginal, at least for the medium term. As I said in the long term, if there becomes a critical mass of backends, then we'd want to follow them, and if the benefits of GISel pan out (i.e. the reasons it's being developed in the first place) then we'd get those as well. But it's less clear before that.

But I think we can try to mitigate the risks and maximize the benefits. Since you've stuck around this long so far and seem competent and responsive to feedback, hopefully that means you can continue doing good work and commit to helping out with maintenance of it. We can also try to target some intermediate milestone(s) that would maybe get us some benefit before the glorious future when we're all globally selecting our instructions instead of drowning in a sea of nodes. Maybe we could shoot for replacing FastISel; in that case we'd want to have a reasonably complete set of features but wouldn't necessarily need 100%, we wouldn't have a high performance bar, but we'd want to try to keep things as fast as possible to avoid regressing compile speed too much, or debug info. Also, maybe we can consult other folks developing GISel to help with the review and development. And there will certainly limitations to my review bandwidth, so this wouldn't necessarily be a short-term project.

Does this sound like a reasonable plan to you?

[QuantumSegfault]

That all makes sense.

True, maintaining this will require extra effort, and the short or even mid-term benefits are hazy at best. Combiners don't seem 100% up-to-par yet, so codegen quality is likely to be a little worse, rather than the ideal better. But given time that can be fixed (as either work on this particular backend/target or as GlobalISel matures in general). Improved code maintainability was another hypothetical, and I personally do feel that GlobalISel is nicer to work with than SelectionDAG, and more robust in certain ways; then again, I'm biased (having at this point spent more time learning the new system than the old one). However that point is also kind of moot if we still have to maintain this in addition to the existing system.

However, at the same time, if we do want to reap the benefits long-term, I see little value in waiting. The sooner more backends try adopting it, the more we can expose and work towards fixing the remaining shortcomings (mainly missing optimization passes) of GlobalISel to the benefit of all. Also, one way or another this is going to take a while just to get the ball rolling. We need to time to build it, and to attract both testers and contributors. The sooner the better.

Then again, without a clear commitment to slowly, steadily shift efforts this way, it WILL end up deadweight, as there won't be enough users nor maintainers to justify it. Something of a chicken-or-the-egg, but I think we leave that as a secondary concern. First is getting a usable base to build upon and implementation to test (i.e. what I/we are doing here).

---

I think targeting FastISel first sounds like a good intermediate step. I had been focusing primarily on replacing SelectionDAG, but considering the narrower feature set and lesser optimizations, it's probably a closer match in this PR's current state, and would be easier to quietly swap in without too much fuss. I'll take a look and see what's still missing to bring it up to FastISel standards.

Thanks!

[QuantumSegfault]

Well... so far it seems that while there are still edge cases to work out, as it stands, the GlobalISel here is generally more feature complete than FastISel, when comparing against the current testsuite. And it produces similar, or even slightly more optimized code. I still need to go through and check each test in more detail, but so far so good!

Though I have no sense of the speed difference. Is there any particular way to benchmark FastISel?

[QuantumSegfault]

Are you able to disable the CI build checks for this PR for now (but keep the formatting check)? It's just a waste of CI time.

[dschuff]

That all makes sense.

True, maintaining this will require extra effort, and the short or even mid-term benefits are hazy at best. Combiners don't seem 100% up-to-par yet, so codegen quality is likely to be a little worse, rather than the ideal better. But given time that can be fixed (as either work on this particular backend/target or as GlobalISel matures in general). Improved code maintainability was another hypothetical, and I personally do feel that GlobalISel is nicer to work with than SelectionDAG, and more robust in certain ways; then again, I'm biased (having at this point spent more time learning the new system than the old one). However that point is also kind of moot if we still have to maintain this in addition to the existing system.

So far no backend has managed to fully replace an SDISel-based pipeline with a GISel pipeline, so my default assumption is that it will take a long time. Part of the reason is that it's a moving target, with most development still happening on SDISel. But also, compilers are critical infrastructure and testing upstream is never as comprehensive as you'd want it to be, so moving defaults is hugely risky and has to be done carefully. Once we have something upstream that can be enabled for testing and we think is close to mature enough to switch to, we'll want to start testing with a lot of real users. (This is where it helps to be part of a large organization that's already building huge codebases, and to have lots of contacts with users). But that process frequently raises issues for big migrations and it can't be rushed.

It does help that our backend is simpler than most. But we are getting more improvements as we speak, and I don't want to slow or stop that progress while we work on new infrastructure.

However, at the same time, if we do want to reap the benefits long-term, I see little value in waiting. The sooner more backends try adopting it, the more we can expose and work towards fixing the remaining shortcomings (mainly missing optimization passes) of GlobalISel to the benefit of all. Also, one way or another this is going to take a while just to get the ball rolling. We need to time to build it, and to attract both testers and contributors. The sooner the better.

Then again, without a clear commitment to slowly, steadily shift efforts this way, it WILL end up deadweight, as there won't be enough users nor maintainers to justify it. Something of a chicken-or-the-egg, but I think we leave that as a secondary concern. First is getting a usable base to build upon and implementation to test (i.e. what I/we are doing here).

"sooner the better" maybe true if "we" means all the backends together, but less true if you have just a small number of people and are unable to unilaterally drive much progress on shared infrastructure. As you say, it's sort of a chicken-and-egg/collective action problem. But even more well-resourced teams have lots of priorities and near-term benefits they need to deliver, so it can be hard to do this kind of thing, and this history of LLVM is full of unfinished migrations.

If we are unable to contribute meaningfully to the shared infrastructure, then the optimal solution (for us, not for the community) is to shift focus right when the critical mass of the community shifts, rather than sooner.

But one nice thing about open source is that it's (sometimes) easy to get more contributor effort for this kind of thing.

I think targeting FastISel first sounds like a good intermediate step. I had been focusing primarily on replacing SelectionDAG, but considering the narrower feature set and lesser optimizations, it's probably a closer match in this PR's current state, and would be easier to quietly swap in without too much fuss. I'll take a look and see what's still missing to bring it up to FastISel standards.

FastISel actually is pretty lacking in terms of features (both in terms of wasm features like post-MVP instructions, and also in terms of less-used LLVM IR features. So it shouldn't be too hard to match it. I do think we should do a bit of performance testing to ensure the compile time regression isn't too much. I haven't done much compile time testing before; I can check internally to see if we can measure this with our internal build system. It might also be worth trying to set up CTMark for wasm. I assume there will be at least some unavoidable regression. One way to help mitigate it might be to reduce the number of times we need to fall back to SDISel (which of course is slow) compared to FastISel. I haven't looked recently how often this happens, but there are some statistics we can look at for that.

[dschuff]

Regarding skipping CI, I don't know of a way to do this for a blanket PR; I think if you add [ci skip] to your commit message you can do it manually (https://docs.github.com/en/actions/how-tos/manage-workflow-runs/skip-workflow-runs)

[QuantumSegfault]

Alright, so to keep this from being a pain to review and merge, I'm breaking this up into parts starting with #178796

Hopefully this way we can finally start getting this reviewed and merged in.

I don't have commit permissions to LLVM, so I can't easily do stacked PRs. I'll just submit one sub-PR at a time to keep things simple for everyone.