Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARM nightlies #8359

Closed
ViralBShah opened this issue Sep 15, 2014 · 30 comments
Closed

ARM nightlies #8359

ViralBShah opened this issue Sep 15, 2014 · 30 comments
Labels
building Build system, or building Julia or its dependencies system:arm ARMv7 and AArch64

Comments

@ViralBShah
Copy link
Member

It would be nice to have ARM nightlies for Ubuntu in the julianightlies PPA, if such a thing is possible. For people running on underpowered boards, this should be an easy way to get Julia rather than building it for hours.

Cc: @staticfloat

@ViralBShah ViralBShah added building Build system, or building Julia or its dependencies system:arm ARMv7 and AArch64 labels Sep 15, 2014
@staticfloat
Copy link
Member

I have contacted Launchpad support and got ARM builds enabled on the julia-nightlies repository. I'm certain that the first few builds will fail horribly as they will require us to create dependencies, etc.... but we'll see. ARM builds have a 4 hour build limit, and as the hardware Canonical is using as ARM buildslaves is undoubtedly less powerful than their x86 buildslaves, I wonder if we'll hit the limit. We can try though. ;)

@ihnorton
Copy link
Member

Can we add a customized Make.user for this build? I backed out the attempt to automatically use ARM config options, so right now it needs include $(JULIAHOME)/ARM.inc in Make.user.

Do you know what they are using for build slaves? If it is something like a PandaBoard ES then 4 hours might be enough.

@ViralBShah
Copy link
Member Author

I wonder if it will hit 4 hours despite using all the various system libraries. The system image takes forever.

@staticfloat
Copy link
Member

@ihnorton No idea, we can do whatever we want here. If you want to output debugging information to see what kind of machines we will run on, just let me know. The system image is indeed what I'm worried about.

@ihnorton
Copy link
Member

Ok, cool - it would be good to get output from cat /proc/cpuinfo.

I looked around for an LLVM 3.5 PPA for ARM, but didn't find one (I guess I
should read up on how to create one). It would make an enormous difference
to use a PPA build or otherwise cache the LLVM 3.5 build.

On Mon, Sep 15, 2014 at 2:20 PM, Elliot Saba [email protected]
wrote:

@ihnorton https://github.com/ihnorton No idea, we can do whatever we
want here
https://github.com/staticfloat/julia-debian/blob/master/debian/rules#L13-L19.
If you want to output debugging information to see what kind of machines we
will run on, just let me know. The system image is indeed what I'm worried
about.


Reply to this email directly or view it on GitHub
#8359 (comment).

@staticfloat
Copy link
Member

Ah, do we need to use 3.5 instead of 3.3? Should we be using 3.5 for master on x86 in general?

@ihnorton
Copy link
Member

For ARM, yes. Otherwise, not yet (there is an open issue about it).

On Mon, Sep 15, 2014 at 4:53 PM, Elliot Saba [email protected]
wrote:

Ah, do we need to use 3.5 instead of 3.3? Should we be using 3.5 for
master?


Reply to this email directly or view it on GitHub
#8359 (comment).

@tkelman
Copy link
Contributor

tkelman commented Sep 16, 2014

On making a PPA, it's way easier if you start from an existing package or PPA. Though enabling ARM builders might require special steps like Elliot had to do. I was able to tweak the MinGW package for example, and as long as you remain within the same Ubuntu version the hardest part is getting GPG keys to work with Launchpad.

@ViralBShah
Copy link
Member Author

@staticfloat Is this the right place to see what's happening with the arm builds?

https://launchpadlibrarian.net/200186425/buildlog_ubuntu-precise-armhf.julia_0.4.0-1956~ubuntu12.04.1_BUILDING.txt.gz

@staticfloat
Copy link
Member

Yep! That's the place. Looks like I've got some tweaking to do still.
-E

On Fri, Mar 13, 2015 at 11:01 PM, Viral B. Shah [email protected]
wrote:

@staticfloat https://github.com/staticfloat Is this the right place to
see what's happening with the arm builds?

https://launchpadlibrarian.net/200186425/buildlog_ubuntu-precise-armhf.julia_0.4.0-1956~ubuntu12.04.1_BUILDING.txt.gz


Reply to this email directly or view it on GitHub
#8359 (comment).

@staticfloat
Copy link
Member

Okay, I've made some tweaks to the debian packaging rules, we'll see if we get past the dependency verification point now.

@ViralBShah
Copy link
Member Author

It is now dying on the Rmath dependency. Best to let it just build that if possible. Looks like the amd64 build is also failing.

Also, you have to include ARM.inc in the Make.user for things to build. Perhaps we should completely integrate the ARM build in the Makefile as there are enough people trying things out.

@ViralBShah
Copy link
Member Author

With #12310 it is now possible to do make binary-dist on arm. In earlier trials of mine, the resulting tarball was unusable as julia would segfault (although I had not had a chance to figure out why). Now, I can use the resulting tarballs on other paths on the same machine, and I feel that we may just have enough things in place to start doing ARM nightlies.

Would the simplest thing to do for now be to just build them on my computer and upload them to s3? Perhaps a cross-compiled one may be the better way to go.

@staticfloat
Copy link
Member

@ViralBShah we could try that. Launchpad is trying to build them, but as usual, there are little problems with the build system that cause them to fail. For instance, now that we're trying to build LLVM 3.6.1 on ARM (which isn't bundled with the typical source upload package) it tries to download them and fails. If you want to experiment with doing our own arm builds, I'm totally okay with that.

@ViralBShah
Copy link
Member Author

I can set up a crontab on my chromebook to upload binaries nightly - not very reliable, but will at least get us going. What should I do to upload?

@staticfloat
Copy link
Member

You should install the aws command line utility, putting your AWS credentials into ~/.awssecret, as appropriate so you can login. Then, go ahead and run something like:

aws put --fail --public julianightlies/bin/linux/armhf/<major_version>/julia-<version>-<shortcommit>-linux_armhf.tar.gz <localfile>

Where the parameters can be found inside Julia as:

major_version = "$(Base.VERSION.major).$(Base.VERSION.minor)"
version = "$(Base.VERSION)"
shortcommit = "$(Base.GIT_VERSION_INFO.commit_short)"

These should result in URLs that look similar to:

https://s3.amazonaws.com/julianightlies/bin/linux/x64/0.4/julia-0.4.0-9b789b8c9a-linux64.tar.gz

All this assuming that the architecture is actually armhf, that is.

@ViralBShah
Copy link
Member Author

I will try set this up next week.

@PallHaraldsson
Copy link
Contributor

I see all arm builds have failed here:

https://launchpad.net/~staticfloat/+archive/ubuntu/julianightlies/+packages

is it (still) because of a time limit? Is there a link to a binary (any one, could be not brand new) that I've not found) or would I have too look into "the aws command line utility" (or compile myself..)?

@staticfloat
Copy link
Member

It's because the ARM build requires a different version of LLVM than the normal ones, and that version of LLVM isn't being bundled in the source tarball properly right now.

@staticfloat
Copy link
Member

I've attempted a fix here: JuliaCI/julia-buildbot@efe9f8b

@staticfloat
Copy link
Member

I will note that we've got some ARM builds that have made it much farther than any previous attempt right now; but they've been going for 6+ hours and aren't even done compiling LLVM. I fully expect Canonical to tell me to knock it off, especially if we attempt to build these ARM binaries with the same frequency as the non-ARM binaries (e.g. every day). It may be the case that we really do just need to make armhf tarballs instead of trying to have a PPA.

@nalimilan
Copy link
Member

@staticfloat Couldn't you build LLVM as a separate package so that you don't need to do that everyday?

@ViralBShah
Copy link
Member Author

Tarballs are always preferable. Also, you could try using the stock llvm 3.6.1 binaries if they have those for the ppa.

@ViralBShah
Copy link
Member Author

I am wondering if we should bother with PPAs in general, given that they always end up using dependent libraries that have known bugs, which are fixed in the tarballs.

@nalimilan
Copy link
Member

Well, packages are the way Linux users expect to get their software, and they offer automated updates which would be relatively painful to do manually when you don't follow Julia's development very closely. (Though this doesn't really apply to ARM at this point, given that it's not completely stabilized.)

@staticfloat
Copy link
Member

Welp, it doesn't matter either way, since building LLVM currently fails with:

/«PKGBUILDDIR»/deps/llvm-3.6.1/lib/Target/ARM/Disassembler/ARMDisassembler.cpp:11:45: fatal error: MCTargetDesc/ARMAddressingModes.h: No such file or directory
 #include "MCTargetDesc/ARMAddressingModes.h"
                                             ^
compilation terminated.

But in general, I agree with @nalimilan that distribution packages are a nice way to make Julia available. Currently, we are dodging the "old library" issue by bundling a bunch of .tar.gz files in the source .tar.gz that we upload to Canonical's buildd servers. That's not a great solution, global warming-wise, since that means we do things like build LLVM every day. If we were the ones actually building the .deb's, we could build them the exact same way that we build any other binary, which (if we end up needing to build our own armhf binaries) is probably what we'll have to transition towards.

@tkelman
Copy link
Contributor

tkelman commented Jan 20, 2016

We have them now: https://status.julialang.org/download/linux-arm

@tkelman tkelman closed this as completed Jan 20, 2016
@nalimilan
Copy link
Member

Cool. Should they be listed at http://julialang.org/downloads/?

@ViralBShah
Copy link
Member Author

+1

@ViralBShah
Copy link
Member Author

Added to the website in the nightlies section. Perhaps we should have a separate arm/x86 tables in there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
building Build system, or building Julia or its dependencies system:arm ARMv7 and AArch64
Projects
None yet
Development

No branches or pull requests

6 participants