Auto-set Make.arm options #11287

staticfloat · 2015-05-15T17:32:20Z

@ViralBShah following up on our conversation in #8359, do you think this is a good change? This way, there's less mucking about with Make.arm, and hopefully the Ubuntu ARM nightlies will make a little more progress.

ViralBShah · 2015-05-15T17:53:38Z

Looks good. LLVM_VER should be 3.6.0. Let's also remove the neon stuff. The remainder options will get tweaked as we go along.

ViralBShah · 2015-05-15T17:54:39Z

Do we need to get cross compiling to work for the build farms? Or do they have ARM boxes?

staticfloat · 2015-05-15T18:00:29Z

They have ARM boxes, but we're running into LLVM compilation errors. I haven't been including Make.arm during the build process however, so that could be the reason why. ;)

ViralBShah · 2015-05-15T18:13:38Z

I guess squash and merge when CI is green, and then we can find out. :-)

garrison · 2015-05-16T00:05:34Z

I'd also suggest modifying README.arm.md, in particular to remove the reference to Make.arm.

vtjnash · 2015-05-16T02:09:05Z

Make.inc

+# If we are running on ARM, set certain options automatically
+ifneq (,$(findstring arm,$(ARCH)))
+LLVM_ASSERTIONS=1
+LLVM_FLAGS+="--with-cpu=cortex-a9 --with-float=hard --with-abi=aapcs-vfp --enable-targets=arm --enable-optimized --enable-assertions"


--enable-optimized and --enable-assertions and --enable-targets shouldn't be in this list as they may break other parts of deps/Makefile

LLVM_ASSERTIONS should also generally not be set for the user

How about we leave them on until we get all tests passing for ARM?

afaik, we haven't been running into any llvm errors thus far

For the time being, I think we should leave the assertions in as the ARM builds are running into a lot of problems right now. Additionally, this doesn't really change the user experience much, as our documentation at the moment suggests a course of action that has the same effect as this PR. I'll take out --enable-optimized and --enable-targets though. Note that --enable-assertions is apparently the default.

staticfloat · 2015-05-17T01:35:16Z

How's this look to people?

tkelman · 2015-05-17T01:55:23Z

+1 from me. I've had a gumstix board sitting on my desk for ages, I should try building Julia on it again.

Auto-set Make.arm options

ViralBShah · 2015-05-17T10:07:58Z

Merging for now. We need to fine tune the LLVM options anyways. So leaving disabling of LLVM assertions for a near future commit.

ViralBShah · 2015-05-17T10:09:58Z

I have a scaleway setup if you guys want to ssh into an ARM box. Ping me on email for a password. I can boot up the image any time.

ScottPJones · 2015-05-17T12:56:40Z

Are the ARM builds building everything (i.e. all the BLAS, FFTW, etc. packages)?
I'm experimenting with making a cut-down branch of Julia, it eliminates some of the fancier math stuff,
along with all the GPL stuff... so far it seems fine.

ViralBShah · 2015-05-17T14:02:57Z

See the ARM README and Make.inc. These things keep changing, but the goal is to have the ARM port build everything eventually.

tkelman · 2015-05-17T21:24:44Z

I'm experimenting with making a cut-down branch of Julia

But regardless, that sounds pretty cool. A non-default build option for that would be interesting, as long as it's not too messy to support.

ScottPJones · 2015-05-17T22:31:01Z

After everybody seemed to make it look like it would be really difficult, I just went in, added a flag BUILD_LITE, which disabled building a lot of stuff... (I'm sure I'd have more to do), but does all seem to hold together (it just doesn't have a bunch of statistics, linear algebra, sparse matrix, fftw, etc. stuff that I doubt would be really necessary on a Raspberry Pi (or ARM based smartphone), but I could be wrong!
It wasn't messy at all... I'll be playing around with it more this summer...

tkelman · 2015-05-17T22:34:49Z

After everybody seemed to make it look like it would be really difficult

Did we? I think the point was that no one has worked on it much yet. And it'll fail vast swaths of the tests and probably not be good enough to run a sizable portion of packages, but definitely worth experimenting with.

ScottPJones · 2015-05-17T23:03:20Z

My point is, the people who would want this "lite" version wouldn't be interested in any packages that would depend on the things removed either... the harder work will be to put things that are currently in base, and that I've removed with the BUILD_LITE flag, back into packages (or a small set of precompiled modules), that would add in all the current math/statistics stuff currently in Base.

tkelman · 2015-05-17T23:43:24Z

It's rarely a good idea to anticipate what functionality users will or will not want to get working.

the harder work will be to put things that are currently in base, and that I've removed with the BUILD_LITE flag, back into packages (or a small set of precompiled modules), that would add in all the current math/statistics stuff currently in Base.

Absolutely. This is #5155, another big project that could do with some people starting to hack away at it. I look forward to seeing your work in this direction :)

ScottPJones · 2015-05-17T23:57:21Z

I've been basing my "J-Lo" branch decisions on what I know I need for string/database processing, and things I think are basic enough that I may be able to use... I found that there are things stuck together that aren't related that should probably be split up (like, why are there a bunch of BLAS functions in util.jl?). I'm just trying to 1) figure out what a minimum build that will be useful for me (and my clients) is, 2) learn more about how the system builds, and the intertwined dependencies, 3) test things without the GPL dependencies (a requirement for my clients), and 4) possibly make something that could be useful for other Julia users...

tkelman · 2015-05-17T23:59:41Z

We're way off-topic for this issue.

why are there a bunch of BLAS functions in util.jl?

Probably no good reason other than because no one has fixed #10548 yet.

ScottPJones · 2015-05-18T00:03:55Z

Shall we move over to #5155? BTW, this is one of the things I read a while ago about it being very difficult (along with other things said in the Google groups):

Summary: Pulling modules out of Base is really hard to do without major breakage, because of the way different versions of the same package export identifiers (particularly types and module names) into the current namespace. The situation is compounded by Julia (currently) disallowing the redefinition of types and the current scoping rules precluding conditional redefinition of existing identifiers with runtime if statements (see JuliaGraphics/Graphics.jl#1 (comment)).

tkelman · 2015-05-18T00:07:29Z

Shall we move over to #5155?

Once you have something new to contribute, can point to an under-development branch, and would like to get feedback or help addressing any issues you hit, yes. If you're just going to repeat things that have already been said, that's unnecessary noise.

ScottPJones · 2015-05-18T01:17:13Z

@tkelman - that seems a bit harsh! I was responding to your comment:

After everybody seemed to make it look like it would be really difficult

Did we?

I'm sorry if responding to your comment was "unnecessary noise".

Look at the spj/jlite branch of ScottPJones/julia, to see what I did (I just spent under an hour on it, to see if it would even build, and second, to see if my code would work on it... which it did just fine).

Later on, I definitely may need to ask for some help, in paring things down.

tkelman · 2015-05-18T01:20:15Z

Sorry. I should have posted my initial comment #11287 (comment) in #5155 in the first place.

Pot meet kettle, but please try to keep on-topic.

ViralBShah · 2015-05-23T11:53:58Z

I got my chromebook building julia once more (dev mode got wiped accidentally). The arm specific options are not getting picked up. What could be going wrong? This PR looks fine.

(precise)viral@localhost:~$ gcc -print-multiarch
arm-linux-gnueabihf

staticfloat · 2015-05-23T17:40:39Z

What does gcc -dumpmachine give?

ViralBShah · 2015-05-23T17:44:35Z

(precise)viral@localhost:~$ gcc -dumpmachine
arm-linux-gnueabihf

staticfloat · 2015-05-23T17:46:47Z

Bizarre. Can you add a $(warning ARCH IS: $(ARCH)) down in Make.inc just before the ifneq here? That should give us an idea of what's going on. Perhaps ARCH is getting overridden by something somewhere.

ViralBShah · 2015-05-24T04:27:48Z

Interesting -

(precise)viral@localhost:~/julia$ make
/home/viral/julia/Make.inc:384: ARCH IS:

I also get:

Make.inc:133: *** No known generic arm cflags. Please specify arch type.  Stop.
make[1]: *** [openlibm/libopenlibm.so] Error 2

staticfloat · 2015-05-24T04:29:57Z

Is line 442 being run at all? You could put a $(warning ....) in there as well to check.

ViralBShah · 2015-05-24T11:42:34Z

Yes, that is getting run and is indeed being set to arm. So, is it somehow getting reset somewhere?

staticfloat · 2015-05-24T19:00:29Z

Is your Make.user doing anything funky?

ViralBShah · 2015-05-25T03:50:08Z

I don't have a Make.user. It is a fresh clone.

ViralBShah · 2015-05-25T03:57:34Z

I don't have a Make.user. It is a fresh clone.

I see that the ARCH is only set on line 468, but we are checking for it at line 384.

staticfloat · 2015-05-25T05:14:45Z

Oh derp. I thought we should do it before including Make.user, but since we tend to put override in Make.user anyway, it probably doesn't matter. Let's try moving this block down below the ARCH setting.

ViralBShah · 2015-05-25T10:24:11Z

I have a patch ready. Seems to be ok, but will commit once I have everything built. Will take a few hours as I upgrade ubuntu to get the right compilers to build llvm, etc.

ViralBShah added the system:arm ARMv7 and AArch64 label May 15, 2015

staticfloat force-pushed the sf/autoarm branch from eb50e54 to 98f3b6b Compare May 15, 2015 18:16

vtjnash reviewed May 16, 2015
View reviewed changes

Auto-set Make.arm options, also remove NEON flags and bump LLVM to 3.6.0

074db58

staticfloat force-pushed the sf/autoarm branch from 98f3b6b to 074db58 Compare May 17, 2015 01:34

ViralBShah added a commit that referenced this pull request May 17, 2015

Merge pull request #11287 from JuliaLang/sf/autoarm

c039cf2

Auto-set Make.arm options

ViralBShah merged commit c039cf2 into master May 17, 2015

ViralBShah added the building Build system, or building Julia or its dependencies label May 17, 2015

tkelman deleted the sf/autoarm branch May 17, 2015 21:23

Auto-set Make.arm options #11287

Auto-set Make.arm options #11287

Conversation

staticfloat commented May 15, 2015

ViralBShah commented May 15, 2015

ViralBShah commented May 15, 2015

staticfloat commented May 15, 2015

ViralBShah commented May 15, 2015

garrison commented May 16, 2015

vtjnash May 16, 2015

Choose a reason for hiding this comment

ViralBShah May 16, 2015

Choose a reason for hiding this comment

vtjnash May 16, 2015

Choose a reason for hiding this comment

staticfloat May 17, 2015

Choose a reason for hiding this comment

staticfloat commented May 17, 2015

tkelman commented May 17, 2015

ViralBShah commented May 17, 2015

ViralBShah commented May 17, 2015

ScottPJones commented May 17, 2015

ViralBShah commented May 17, 2015

tkelman commented May 17, 2015

ScottPJones commented May 17, 2015

tkelman commented May 17, 2015

ScottPJones commented May 17, 2015

tkelman commented May 17, 2015

ScottPJones commented May 17, 2015

tkelman commented May 17, 2015

ScottPJones commented May 18, 2015

tkelman commented May 18, 2015

ScottPJones commented May 18, 2015

tkelman commented May 18, 2015

ViralBShah commented May 23, 2015

staticfloat commented May 23, 2015

ViralBShah commented May 23, 2015

staticfloat commented May 23, 2015

ViralBShah commented May 24, 2015

staticfloat commented May 24, 2015

ViralBShah commented May 24, 2015

staticfloat commented May 24, 2015

ViralBShah commented May 25, 2015

ViralBShah commented May 25, 2015

staticfloat commented May 25, 2015

ViralBShah commented May 25, 2015