Arch Linux is not shipping sys.so, causing long startup times

[Wilfred]

Arch Linux has decided not to include sys.so in their package of Julia 0.3. They delete sys.so as part of the packaging process: https://projects.archlinux.org/svntogit/community.git/tree/trunk/sysfix.install?h=packages/julia due to issues with architecture mismatches: https://bugs.archlinux.org/task/39721

As a result, Julia 0.3 still has a ~3 second startup time: https://bugs.archlinux.org/task/41666

What's the right approach here?

Related: #5459.

[nalimilan]

Ah, I thought the Arch maintainer of the Julia package said they were shipping it. AFAIK the right approach is to ship it, and build Julia with MARCH=pentium4 for 32-bit, and MARCH=x86-64 for 64-bit. I've added a comment downstream.

EDIT: Makefile seems to prefer MARCH=x86_64 instead of x86-64.

[vchuravy]

Ah that is the reason why the startup became slower when I switched from self building to the packaged version.

A possible solution would be with #8074 to add a post-install step that regenerates sys.so instead of deleting like it currently does: https://projects.archlinux.org/svntogit/community.git/tree/trunk/sysfix.install?h=packages/julia

[nalimilan]

@vchuravy There's no need for that. The baseline image should suit all users with a reasonable performance.

[vchuravy]

Yes it should be working, but since march=x86_64 is the setting default on Archlinux (especially) for packagers https://wiki.archlinux.org/index.php/makepkg

And the sys.so only gets removed when Julia is falling to start up. So the question is why if the package is indeed compiled with march=x86_64 and JULIA_CPU_TARGET=core2,
the line

julia --version 2>/dev/null || rm /usr/lib/julia/sys.so

triggered on my machine which is a sandybridge processor. Since I don't have a sys.so file installed

[nalimilan]

@vchuravy Well, it's hard to tell. Could you extract the sys.so file from the package, put it at /usr/lib/julia/sys.so, run julia --version and tell us what's the error?

[vchuravy]

Extracting sys.so from the latest julia build for Archlinux julia-2:0.3.0-1-x86_64 and sudo cp sys.so /usr/lib/julia/sys.so

I get

julia --version
Target architecture mismatch. Please delete or regenerate sys.{so,dll,dylib}.

running analyze-x86 on sys.so

I get this:

./analyze-x86 /usr/lib/julia/sys.so 
instructions:
 cpuid: 0        nop: 380        call: 0         count: 624149
 i486:   2
 i686:   1529
 mmx:    9806
 sse:    11370
 sse2:   962
 sse3:   12
 ssse3:  4
 sse4.1:         39

running analyze-x86 on an old sys.so that I had build with march=corei7-avx and JULIA_CPU_TARGET=native

./analyze-x86 ../aur/julia/usr/lib/julia/sys.so
instructions:
 cpuid: 0        nop: 427        call: 0         count: 685979
 i486:   2
 i686:   1542
 mmx:    9528
 sse4.2:         1
 avx:    13066

[xyproto]

The subject of this issue is misleading. sys.so is placed in /usr/lib/julia/sys.so if julia works when sys.so is present. When it is removed, the startup times are longer. Blaming Arch Linux is premature.

[nalimilan]

It seems the Arch sys.so has not actually been built for core2. When I do strings sys.so | grep native, I find a match for the Arch file, but not for the sys.so in my RPMs, built with JULIA_CPU_TARGET equal to i386 or x86-64.

@xyproto I don't understand setconf in the Arch package code, but are you sure JULIA_CPU_TARGET=core2 is correctly passed to make? Do you have a log of the build somewhere? Anyway, you could try with MARCH=(x86_64|pentium4) (for 64/32 bits), which is the recommended way now.

[vchuravy]

It looks like the true culprit is https://projects.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/julia#n25

This line is supposed to set JULIA_CPU_TARGET = core2 but if you actually run that line on the Make.inc file nothing changes and JULIA_CPU_TARGET is still set to native. Leading to the observed architecture mismatch even though my CPU supports all the features given.

running this on a sys.so generated by the aforementioned PKGBUILD gives me

julia> bytestring(cglobal(("jl_sysimg_cpu_target","./sys.so"), Cchar))
native

instead of core2

[xyproto]

I think you nailed it, vchuravy. Thank you. Rebuilding the julia package.

[vchuravy]

@xyproto JULIA_CPU_TARGET will still default to native
@nalimilan I can't compile Julia when I manually set MARCH to x86_64 gcc is expecting x86-64

Also could one set MARCH to i686 instead of pentium4? Pentium4 is i768 and I think Archlinux accepted minimum is i686

-- edit

 error: bad value (x86_64) for -march= switch

[xyproto]

@vchuravy You're right. The i686 support in Arch Linux is without SSE2 - for the oldest versions of i686. Pentium4 is probably too high. (Unfortunately. The Go package has the same problem, I wish SSE2 was supported for i686. I'll see if I can convince the other Arch Linux people).

I'll try rebuilding with MARCH set to x86_64 for 64-bit and MARCH set to i686 for 32-bit, then.

[vchuravy]

@xyproto I am currently building for x86_64 and will report back if I get a sensible result.

In the mean time I am using

build() {
  make -C "$pkgname" prefix=/usr sysconfdir=/etc \
    MARCH=${CARCH/_/-} \
    USE_SYSTEM_LLVM=0 \
    ...

[vchuravy]

So Julia build successfully with x86-64 :) and the sys.so also reports the correct cpu-target

julia> bytestring(cglobal(("jl_sysimg_cpu_target","./sys.so"), Cchar))
"x86-64"

-- edit
@xyproto at least for me this works
@nalimilan should I open an issue for the x86_64 vs x86-64 issue?

[xyproto]

Updating the julia package for x86-64 now.

@vchuravy Please confirm that it works as expected. Added your name in the PKGBUILD. Thanks for testing.

[pao]

@vchuravy x86-/_64 was #7433

There's not much point in going < pentium4 for the 32-bit build because of dSFMT, see #7185 (comment).

[xyproto]

@pao
Arch Linux currently supports i686, which includes processors that arrived (right before) pentium 4, that does not support SSE2. I personally think this is unpractical, and that i686 should be dropped entirely (and be replaced with an 64-bit ARM architecture instead), but here opinions vary greatly.

So currently, for Arch Linux to have a julia package for i686, it can't be built for pentium4.

[vchuravy]

@pao Ah I see, but the the MARCH option is still x86-64 since this is given directly to gcc
@xyproto happy to be of help and also happy to report that it works for me now.

[nalimilan]

It seems all distributions would like to support all i686 CPUs, but at the moment it's not possible. Anyway it's better to support > Pentium4 than having no 32-bit support at all. And honestly, if your package doesn't work on < P4, nobody is going to notice (if you don't need to get it reviewed)...

@vchuravy Just use MARCH=x86-64, it seems to work for me.

[xyproto]

@nalimilan When the go package was compiled in a way that required SSE2, a bug report was quickly filed and someone was not happy that it did not work with their pentium 3... I've already sorta tried to do it the sneaky way and it didn't work out.

[StefanKarpinski]

Julia does not work on Pentium 3 and most likely never will, so I'm not sure what to say. Is it really a requirement for Arch Linux that all software support hardware that's 15 years old?

[xyproto]

@StefanKarpinski Currently, there are three types of architectures for binary Arch Linux packages: "i686", "x86_64" and "any". The latter, "any", must work on any architecture, while "i686", regrettably, can not require the presence of SSE2.

It is fully possible to only offer julia as an x86_64 package only, if that is preferable to a lobotomized SSE2-less version for i686.

[vchuravy]

@xyproto There has been at least one case (Chromium 35) https://bugs.archlinux.org/task/40479. Upstream decided to drop i686 without SSE2 and Archlinux also dropped support for i686 without SSE2 for Chromium.

[nalimilan]

@xyproto Interesting to hear. I didn't think people were really using such old machines for development work. I used to have a P2 working circa 2006, but then that's already 8 years back. ;-)

[Wilfred]

As of the Julia package 2:0.3.0-2, I'm now seeing the performance expected on my x86_64 box. I don't think there's anything left to do in this issue.

[StefanKarpinski]

😄

[nalimilan]

@sebastien-villemot Unrelated to the Arch issue, but you may be interested in using MARCH as detailed above.

[xyproto]

@vchuravy Talked to the maintainer of Chromium on Arch Linux. He said that Chromium was added with SSE2 as an exception to the rule, because people with non-SSE2 computers were not likely to be running Chromium in any case.

If Julia is considered to be a "fast computers only"-application, enabling SSE2 should be a possibility.

If so, please state that somewhere on the webpage, in the manpage or elsewhere, so that bug reports about missing SSE2-support can be directed there. That would help a lot. Thanks.

[nalimilan]

@xyproto At the moment, make testall fails with anything older than Pentium 4 (#7185), so indeed you should pass MARCH=pentium4, which will enable SSE2. Else some computations may be incorrect.

If you want a public statement somewhere, e.g. in README.md, you could make a pull request.