Upstream the llvm patches

[erickt]

We have a handful of llvm patches that need to be pushed upstream. llvm just had a release, so have a nice window to get our patches reviewed and integrated for the next release.

Remaining patches to upstream

• ARM segmented stacks
  • rust-lang/llvm@b24ba55
  • rust-lang/llvm@b70c16c
  • rust-lang/llvm@fd8a627
• no-split-stack
  • rust-lang/llvm@ae43eea
• win64 segmented stacks
  • rust-lang/llvm@12d8851
• getPassInfo() (optional)
  • rust-lang/llvm@ed0d140

[fabiand]

Is there any progress or timeline when you want to push your changes to upstream?

[sanxiyn]

Referencing #5368.

[brson]

@fabiand we actually are carrying no key llvm patches at the moment besides those dealing with ARM. I don't know the ETA for upstreaming those.

[fabiand]

@brson That's good to know. Is there currently a build switch to build rust without pulling and building llvm? (So using existing llvm binaries?)

[brson]

@fabiand configure has an --llvm-root flag for specifying a custom LLVM but I don't know if it works

[fabiand]

@brson, thanks - It doesn't seem to work. The linker can't find the installed version. But I did not yet nail it down further. THere was one patch needed for llvm3.2 (wllvm32 branch)

[brson]

Not for 0.6.

[isanbard]

I just wanted to jump in and say that LLVM is in the midst of planning their next release. If you have any patches for LLVM, please let us know! :-) Either by creating a PR or sending them to llvm-commits.

Thanks. :-)

[bstrie]

Nominated for milestone due to time-sensitivity.

[pnkfelix]

visiting for triage email 2013-07-22.

@brson @graydon : Any way we can up the priority on this task? Do we think we missed the window of time described in isanbard's comment?

From my reading of the releases at the LLVM download page and the LLVM release process docs, it seems like they release every 6 months, and we might expect the next release to take place to occur in December. (Maybe isanbard was referring to the 3.3 release, which came out on June 17th.) So this may not be super-duper high priority, but that might be all the more reason to actually start working this through our own pipeline...

[isanbard]

Unfortunately, you did miss the window for the 3.3 release. The next major release is around the end of the year. Though there may be a "dot" release before that. (I believe one of our contributors is in the middle of doing this.)

[bstrie]

Are we also certain that we won't have more patches for LLVM, what with the new scheduler, actual GC, TBAA, etc.?

[thestinger]

We don't need any patches for a TBAA pass, it can be loaded as a dynamic library by LLVM.

[brson]

There will probably one more patch for #1226 (unless upstream has already done it). Additionally, I think that the patches we do have are probably not up to the quality standards llvm expects (missing tests, etc.).

[brson]

@pnkfelix I haven't been very motivated to pursue this, sadly. It will be significant work, we still have at least some known new patches to write (mentioned above), the patches we do have need to be improved, and from what I understand some of our arm patches are somewhat special-cased and likely need further design work.

[brson]

Also, last time I tried to upgrade LLVM I hit errors on windows, and as that so often goes, I just gave up rather than try to debug there.

[omasanori]

Certainly we, the Rust project, have to maintain our own fork, it's still good for the upstream, other projects and package maintainers to merge patches into the upstream as soon as it gets mature, isn't it?
(you have many tasks so it might have a lower priority, of cource.)

[emberian]

@omasanori we don't need our own fork, the only reason we do is to work around issues in llvm

[omasanori]

@cmr Certainly it's not a complete fork.

[thestinger]

The intention is for Rust to be using a vanilla LLVM; we shouldn't have our own fork. The current patches can be reworked until they're accepted upstream or just dropped by removing the dependency on them.

[alexcrichton]

As of right now, here's a list of patches we have to upstream.

Of those, two can be ignored. One is already upstream (Use a mutex instead of an RWMutex), and the other is a fix to get around a bug in 3.3 that was already fixed upstream (Skip i8 case in fastisel)

[lucab]

A fellow Debian and LLVM upstream developer agreed to take a look at our patch queue and integrating the non-debatable ones upstream. I'll keep you posted on the status.

[emberian]

☀️ wonderful!

[fabiand]

🎉 (icons!) I'd love to get an official package in place for Fedora 21

[alexcrichton]

@lucab, awesome! If you want to connect me to them or vice versa, I've been dealing with the LLVM upgrades recently so I could try to help out where needed.

[bstrie]

@lucab Any updates? Also, I'm curious what the cutoff is for getting patches into the next stable LLVM release.

[lucab]

@sylvestre said he would have taken care of this, not sure about the status right now...

[isanbard]

This is the documentation on how to write them:

http://llvm.org/docs/DeveloperPolicy.html#test-cases

It's not very extensive, but there are many examples in the code base of tests (see the test/ directories). Just use good testing practices to figure out what to test.

[brson]

Nominating. Is it a goal to be able to use system LLVM for 1.0?

[thestinger]

@brson: it would need to happen to have a distribution like Fedora or Debian to package it, so I think it should be

[sylvestre]

Indeed. I haven't followed that closely but is the bootstrap "issue" been fixed ?

[sanxiyn]

@sylvestre What is the "bootstrap issue"?

[sylvestre]

As explained here https://github.com/mozilla/rust
"Since the Rust compiler is written in Rust, it must be built by a precompiled "snapshot" version of itself (made in an earlier state of development). As such, source builds require a connection to the Internet, to fetch snapshots, and an OS that can execute the available snapshot binaries."
In Debian or Ubuntu build system, the network is not available and we want to be able to bootstrap the build from scratch. For example, OpenJDK is built using gcj.

[thestinger]

The snapshot can be downloaded in advance and this definitely works as Rust won't keep downloading it on each build if snapshots.txt hasn't been updated. I don't think it's an issue.

[lucab]

@sylvestre I'm collecting some preliminary ideas on that at https://wiki.debian.org/Teams/RustPackaging/Bootstrap but that's not the point of this bug report. The LLVM upstreaming issue is a separate one which also the Fedora guys would like to have addressed.

I believe this is not an hard requirements for both distro, but will make package review harder (nobody wants yet another full LLVM copy in the archive).

[sylvestre]

OK, thanks mate :)

About the bug, if there is no tests, it is going to be impossible to see the patches applied upstream.

[thestinger]

The no-split-stack attribute was explicitly rejected upstream, and I have no doubt that fixed-stack-segment would also be rejected. It's not just about tests as these things have to be implemented in the right way to be merged rather than just what Rust happens to need at the moment.

If stop using the segmented stack support, then most of this problem will go away.

[pnkfelix]

We discussed what our requirements with respect to this issue in the weekly meeting. Our current stance is this: We want to be compatible with a reference host-provided LLVM for Rust 1.0, and we believe that this is an achievable goal (at least on x86 hosts).

So, we will attempt to support and/or shepherd-in changes that move us towards the goal of being able to work with a host-provided LLVM. (e.g. @alexcrichton has some ideas on how to deal with no_split_stack.)

Bu: working atop a host-provided LLVM is not something that should block the Rust 1.0 release.

[pnkfelix]

(removing the I-nominated tag since we have now had the delayed discussion of the matter.)

[emberian]

@alexcrichton what ideas are those?

[alexcrichton]

Any no_split_stack function can be written in C. I would much rather redouble efforts to upstream the patch, I don't believe that LLVM won't take it.

[thestinger]

@alexcrichton: They expressed that they wanted split_stack instead of no_split_stack.

[alexcrichton]

The win64 segmented stack patch was just upstreamed, I've updated the issue description to reflect remaining work.

[alexcrichton]

The segmented stacks for ARM patches have been upstreamed.

[alexcrichton]

The equivalent of a no-split-stack patch has been upstreamed. On the next LLVM update we will have to change our codegen to attach the "split-stack" attribute to all functions we generate.

[alexcrichton]

The last remaining patch is just an optimization, and I don't think it's critical that we upstream it. We can keep it around locally for awhile, however.

We've got a bit of work on our hands to get compatible with upstream LLVM 3.5, but it'll be hairy regardless because we'll have to start dealing with c++11 compilers and such.

With all the patches upstreamed, I'm going to close this issue. Hurray!

[brson]

\o/
On Apr 10, 2014 4:18 PM, "Alex Crichton" [email protected] wrote:

Closed #4259 #4259.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4259
.

[omasanori]

👍

[cnd]

👍

[lucab]

@alexcrichton This (and the libuv too) are great news. Thanks for all your efforts!