tests failing on 32-bit Ubuntu 13.04

[amitmurthy]

Julia version:

 | | |_| | | | (_| |  |  Version 0.2.0-1796.rbac428f9
 _/ |\__'_|_|_|\__'_|  |  Commit bac428f990 2013-06-04 03:41:03
|__/                   |

output of runtests.jl:

amitm@amitm-ThinkPad-T61:~/Work/julia/julia/test$ julia runtests.jl all
    From worker 2:       * core
    From worker 3:       * keywordargs
    From worker 3:       * numbers
    From worker 2:       * strings
    From worker 2:       * unicode
    From worker 2:       * corelib
    From worker 2:  exception on 2: ERROR: test failed: :(contains(r,3))
    From worker 2:   in default_handler at test.jl:21
    From worker 2:   in do_test at test.jl:38
    From worker 2:   in runtests at /home/amitm/Work/julia/julia/test/testdefs.jl:5
    From worker 2:   in anonymous at multi.jl:416
    From worker 2:  at corelib.jl:94
    From worker 2:       * hashing
    From worker 2:       * remote
    From worker 2:       * iostring
    From worker 2:       * arrayops
    From worker 2:       * linalg
    From worker 3:  exception on 3: ERROR: test failed: :(isprime(10000000019))
    From worker 3:   in default_handler at test.jl:21
    From worker 3:   in do_test at test.jl:38
    From worker 3:   in runtests at /home/amitm/Work/julia/julia/test/testdefs.jl:5
    From worker 3:   in anonymous at multi.jl:416
:1444   ��\m worker 3:  at ��\
    From worker 3:       * blas
    From worker 3:       * fft
    From worker 3:       * dsp
    From worker 3:       * sparse
        From worker 3:       * bitarray
    From worker 3:       * random
    From worker 2:       * math
    From worker 3:       * functional
    From worker 3:       * bigint
    From worker 3:       * sorting
    From worker 2:       * statistics
    From worker 2:       * spawn
    From worker 2:         [stdio passthrough ok]
    From worker 3:       * parallel
    From worker 2:       * priorityqueue
    From worker 2:       * arpack
    From worker 3:       * file
    From worker 2:       * perf
    From worker 3:       * suitesparse
    From worker 3:       * version
    From worker 3:       * resolve
    From worker 2:       * pollfd
    From worker 2:       * mpfr
    From worker 3:       * broadcast
    From worker 2:       * complex
ERROR: test failed: :(isprime(10000000019))
:1444\  ��\
at /home/amitm/Work/julia/julia/test/runtests.jl:17

[amitmurthy]

The isprime() function is failing when used with 64-bit ints on 32-bit machines.

A proposed solution is to treat 64 bit integers as BigInt on 32-bit machines. The addition of the following lines should help

if WORD_SIZE == 32
    isprime(n::Uint64) =
        n <= typemax(Uint32) ? isprime(uint32(n)) : isprime(BigInt(n))
    isprime(n::Int64) = n < 2 ? false :
        n <= typemax(Int32)  ? isprime(int32(n))  : isprime(BigInt(n))
end

test/corelib.jl is failing because of the following code on 32-bit machines:

julia> using Base.Test

julia> r = (-4*int64(maxintfloat())):5
-36028797018963968:-36028797018963963

julia> @test contains(r,3)
ERROR: test failed: :(contains(r,3))
 in default_handler at test.jl:21
 in do_test at test.jl:38

julia> r
-36028797018963968:-36028797018963963

If the test in test/corelib.jl is for contains, the above should be fixed with changing the range generation to (-4*int64(maxintfloat(Float32))):5 while also creating an issue to fix the range generation operator. It also fails with explicitly specifying the second operator as int64:

julia> r = (-4*int64(maxintfloat())):int64(5)
-36028797018963968:-36028797018963963

[StefanKarpinski]

I'm not a fan of the isprime fix – it's a bandaid. There isprime algorithm should be fixed instead. Unfortunately, I don't have access to a 32-bit machine to test on. The range containment issue also seems like it's hiding the symptom rather than fixing the issue: why is (-4*int64(maxintfloat())):5 ending up as -36028797018963968:-36028797018963963? There some overflow happening in there that shouldn't.

[pao]

Unfortunately, I don't have access to a 32-bit machine to test on.

Couldn't you cross-compile the 32-bit version? I could also add a 32-bit Vagrant configuration.

[StefanKarpinski]

I could, but where am I going to run it?

[JeffBezanson]

Lengths are Int, which is Int32 on 32-bit platforms. This includes ranges.
I'll bring my old 32-bit netbook for testing.

[ViralBShah]

Whatever the fixes, the 32 bit build should not be failing. I think these are reasonable to fix the build, and other issues can be opened for the real fixes. Or we could disable those tests.

[ViralBShah]

Yup, I noticed that len was 32 bit even for a 64 bit int Range1 object.

[StefanKarpinski]

This is basically the same problem as BigInt ranges.

[pao]

I could, but where am I going to run it?

Wouldn't a 32-bit userspace be enough? Or are you on Mac right now? Then you might have to use a 32-bit VM (which gets back to the Vagrant comment--if @staticfloat has 32-bit dependencies in his PPA, you could just switch from the precise64 base box to the precise32 base box in the Vagrantfile.)

[StefanKarpinski]

True, using a VM would work (although I'd have to set one up). We don't, as far as I know, support generating a 32-bit Julia on 64-bit machines or vice versa, but I could be wrong about that. @vtjnash and @loladiro have managed some crazy stuff.

[pao]

We don't, as far as I know, support generating a 32-bit Julia on 64-bit machines or vice versa, but I could be wrong about that.

Ah, I expected such a cross-compile was possible--at any rate, this is off the issue topic.

[staticfloat]

I can confirm this happens on my 32-bit VM, using the builds from my nightlies PPA.

Linux is wonderful. :) You can sudo apt-get install julia:i386 and it will install Julia and all of her 32-bit dependencies from my PPA. If you already have those dependencies installed, you will need to remove them first, which could be a problem, however for testing it was enough for me to confirm that the error still exists after doing this.

[amitmurthy]

I'll prepare a 32-bit EC2 AMI with dev env and julia sources. It can be used for such situations in the future.

[staticfloat]

Great, I'm compiling a kernel that hopefully has AVX disabled. If it works, we can include it with your AMI for ease of use, so others don't have to patch their own kernel.

[staticfloat]

Compiling large sets of source really does take a while on these machines. It's been going for 8 hours so far.

[amitmurthy]

Are you using a free micro instance? It will probably take a really long time since they are heavily throttled and fit only for periodically bursty compute scenarios - definitely not for compiling source!

[staticfloat]

Yeap. :D I don't mind waiting, this doesn't seem to be a time-critical issue.

[ViralBShah]

It is worth checking if openblas has an environment variable to change the target arch at runtime. If not, we can perhaps request @xianyi to support it.

[staticfloat]

This was my initial idea, however I am pretty certain this isn't the solution we're looking for. Any other piece of software we want to run that may use AVX will encounter the same problem; it's really not a bug in OpenBLAS, it's a bug in the EC2 virtualization environment, and can be patched in the kernel.

[amitmurthy]

Just so that there is no confusion - the AVX discussion above relates to #3263 and is unrelated to this issue, which is purely a Julia issue.

[staticfloat]

whoops, I totally mixed two issues up. Thanks @amitmurthy!

[amitmurthy]

ami-8bdda8e2 is a public 32-bit ubuntu 13.04 AMI with Julia dev / sources installed.

userid : ubuntu
region : N. Virginia (us-east)