Illegal instruction (julia package on Ubuntu 13.04 on EC2 t1.micro)

[cmey]

Hi, just reporting a problem,

I tried to run julia on Ubuntu 13.04 on a free EC2 instance (t1.micro) and got an Illegal instruction just after typing julia.

I tried both 32 and 64 bit Ubuntu.
Haven't tried this on a non-free instance.

Steps to reproduce:

• create a t1.micro instance running the OS : Ubuntu server 13.04 (32 or 64 bits) ;
• sudo apt-get install julia
• execute 'julia'
• "Illegal instruction (core dumped)"

[arnim]

Just gave @staticfloat s PPA a shot
(on a fresh 64 bit Ubuntu 13.04 on a t1.micro EC2 instance)

sudo add-apt-repository ppa:staticfloat/julia-deps -y
sudo add-apt-repository ppa:staticfloat/julianightlies -y
sudo apt-get update -qq -y
sudo apt-get install -y libpcre3-dev
sudo apt-get install julia -y

julia

resulting (as above) in
Illegal instruction (core dumped)

Version seems to be as in Setting up julia (0.2.0+nightly20130531~raring1)

Buildlog on launchpad
https://launchpadlibrarian.net/141163597/buildlog.txt.gz

[cmey]

I tried compiling julia from source on this instance (it took 12h).
It gets to the point where is calls julia on several .jl files, and again, I get the dreaded "Illegal instruction (core dumped)" error message.

end of compilation output log:

CC src/jlapi.o
CC src/gc.o
LINK usr/lib/libjulia-release.so
PERL base/pcre_h.jl
PERL base/errno_h.jl
PERL base/build_h.jl
PERL base/file_constants.jl
PERL base/uv_constants.jl
CC ui/repl.o
CC ui/repl-basic.o
LINK usr/bin/julia-release-basic
CC ui/repl-readline.o
LINK usr/bin/julia-release-readline
JULIA usr/lib/julia/sys.ji
int.jl
Illegal instruction (core dumped)
*** This error is usually fixed by running 'make clean'. If the error persists, try 'make cleanall'. ***

This error is not fixed by running 'make clean' or 'make cleanall' ;-)

[staticfloat]

This is likely a problem with virtualization. I've run into this on Virtualbox, so I would not be surprised if other virtualization technologies have similar issues.

Can you run cat /proc/cpuinfo for me so I can get a look at what kind of features your box is advertising it has? My guess is we may need to artificially reduce the number of instructions julia is attempting to use because the virtualization software can't support it, even though the cpu can. (For example, virtualbox doesn't allow AVX code to pass through; so even though the CPU can run it, software trying to use it will fail)

[arnim]

@staticfloat

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

Welcome to Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Sat Jun  1 23:42:04 UTC 2013

  System load:  0.64             Processes:           70
  Usage of /:   9.1% of 7.75GB   Users logged in:     0
  Memory usage: 20%              IP address for eth0: 10.224.3.150
  Swap usage:   0%

  Graph this data and manage this system at https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

  Use Juju to deploy your cloud instances and workloads:
    https://juju.ubuntu.com/#cloud-raring

0 packages can be updated.
0 updates are security updates.

ubuntu@ip-10-224-3-150:~$ cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 45
model name  : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
stepping    : 7
microcode   : 0x70a
cpu MHz     : 1799.999
cache size  : 20480 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes avx hypervisor lahf_lm
bogomips    : 3599.99
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

ubuntu@ip-10-224-3-150:~$ sudo add-apt-repository ppa:staticfloat/julia-deps -y
gpg: keyring `/tmp/tmpcgt2j3/secring.gpg' created
gpg: keyring `/tmp/tmpcgt2j3/pubring.gpg' created
gpg: requesting key 3D3D3ACC from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpcgt2j3/trustdb.gpg: trustdb created
gpg: key 3D3D3ACC: public key "Launchpad PPA for Elliot Saba" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
ubuntu@ip-10-224-3-150:~$ sudo add-apt-repository ppa:staticfloat/julianightlies -y
gpg: keyring `/tmp/tmppe3tly/secring.gpg' created
gpg: keyring `/tmp/tmppe3tly/pubring.gpg' created
gpg: requesting key 3D3D3ACC from hkp server keyserver.ubuntu.com
gpg: /tmp/tmppe3tly/trustdb.gpg: trustdb created
gpg: key 3D3D3ACC: public key "Launchpad PPA for Elliot Saba" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
ubuntu@ip-10-224-3-150:~$ sudo apt-get update -qq -y
ubuntu@ip-10-224-3-150:~$ sudo apt-get install -y libpcre3-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  binutils cpp cpp-4.7 gcc gcc-4.7 libc-dev-bin libc6-dev libcloog-ppl1
  libgcc-4.7-dev libgmpxx4ldbl libgomp1 libitm1 libmpc2 libmpfr4 libpcrecpp0
  libppl-c4 libppl12 libquadmath0 linux-libc-dev manpages-dev
Suggested packages:
  binutils-doc cpp-doc gcc-4.7-locales gcc-multilib make autoconf automake1.9
  libtool flex bison gdb gcc-doc gcc-4.7-multilib libmudflap0-4.7-dev
  gcc-4.7-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libquadmath0-dbg
  libmudflap0-dbg binutils-gold glibc-doc
The following NEW packages will be installed:
  binutils cpp cpp-4.7 gcc gcc-4.7 libc-dev-bin libc6-dev libcloog-ppl1
  libgcc-4.7-dev libgmpxx4ldbl libgomp1 libitm1 libmpc2 libmpfr4 libpcre3-dev
  libpcrecpp0 libppl-c4 libppl12 libquadmath0 linux-libc-dev manpages-dev
0 upgraded, 21 newly installed, 0 to remove and 16 not upgraded.
Need to get 25.1 MB of archives.
After this operation, 65.9 MB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libgmpxx4ldbl amd64 2:5.0.5+dfsg-2ubuntu3 [9,150 B]
Get:2 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libppl12 amd64 1.0-1ubuntu2 [565 kB]
Get:3 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libppl-c4 amd64 1.0-1ubuntu2 [1,486 kB]
Get:4 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libcloog-ppl1 amd64 0.16.1-1 [64.8 kB]
Get:5 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libgomp1 amd64 4.7.3-1ubuntu1 [27.3 kB]
Get:6 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libitm1 amd64 4.7.3-1ubuntu1 [36.2 kB]
Get:7 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libmpfr4 amd64 3.1.1-1 [203 kB]
Get:8 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libpcrecpp0 amd64 1:8.31-2 [16.5 kB]
Get:9 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libquadmath0 amd64 4.7.3-1ubuntu1 [125 kB]
Get:10 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libmpc2 amd64 0.9-4build1 [39.8 kB]
Get:11 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main binutils amd64 2.23.2-2ubuntu1 [2,393 kB]
Get:12 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main cpp-4.7 amd64 4.7.3-1ubuntu1 [5,412 kB]
Get:13 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main cpp amd64 4:4.7.3-1ubuntu10 [27.7 kB]
Get:14 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libgcc-4.7-dev amd64 4.7.3-1ubuntu1 [2,464 kB]
Get:15 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main gcc-4.7 amd64 4.7.3-1ubuntu1 [6,088 kB]
Get:16 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main gcc amd64 4:4.7.3-1ubuntu10 [5,120 B]
Get:17 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libc-dev-bin amd64 2.17-0ubuntu5 [81.4 kB]
Get:18 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring-updates/main linux-libc-dev amd64 3.8.0-23.34 [917 kB]
Get:19 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libc6-dev amd64 2.17-0ubuntu5 [3,088 kB]
Get:20 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libpcre3-dev amd64 1:8.31-2 [266 kB]
Get:21 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main manpages-dev all 3.44-0ubuntu1 [1,738 kB]
Fetched 25.1 MB in 2s (9,601 kB/s)   
Selecting previously unselected package libgmpxx4ldbl:amd64.
(Reading database ... 52215 files and directories currently installed.)
Unpacking libgmpxx4ldbl:amd64 (from .../libgmpxx4ldbl_2%3a5.0.5+dfsg-2ubuntu3_amd64.deb) ...
Selecting previously unselected package libppl12:amd64.
Unpacking libppl12:amd64 (from .../libppl12_1.0-1ubuntu2_amd64.deb) ...
Selecting previously unselected package libppl-c4:amd64.
Unpacking libppl-c4:amd64 (from .../libppl-c4_1.0-1ubuntu2_amd64.deb) ...
Selecting previously unselected package libcloog-ppl1:amd64.
Unpacking libcloog-ppl1:amd64 (from .../libcloog-ppl1_0.16.1-1_amd64.deb) ...
Selecting previously unselected package libgomp1:amd64.
Unpacking libgomp1:amd64 (from .../libgomp1_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package libitm1:amd64.
Unpacking libitm1:amd64 (from .../libitm1_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package libmpfr4:amd64.
Unpacking libmpfr4:amd64 (from .../libmpfr4_3.1.1-1_amd64.deb) ...
Selecting previously unselected package libpcrecpp0:amd64.
Unpacking libpcrecpp0:amd64 (from .../libpcrecpp0_1%3a8.31-2_amd64.deb) ...
Selecting previously unselected package libquadmath0:amd64.
Unpacking libquadmath0:amd64 (from .../libquadmath0_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package libmpc2:amd64.
Unpacking libmpc2:amd64 (from .../libmpc2_0.9-4build1_amd64.deb) ...
Selecting previously unselected package binutils.
Unpacking binutils (from .../binutils_2.23.2-2ubuntu1_amd64.deb) ...
Selecting previously unselected package cpp-4.7.
Unpacking cpp-4.7 (from .../cpp-4.7_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package cpp.
Unpacking cpp (from .../cpp_4%3a4.7.3-1ubuntu10_amd64.deb) ...
Selecting previously unselected package libgcc-4.7-dev:amd64.
Unpacking libgcc-4.7-dev:amd64 (from .../libgcc-4.7-dev_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package gcc-4.7.
Unpacking gcc-4.7 (from .../gcc-4.7_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package gcc.
Unpacking gcc (from .../gcc_4%3a4.7.3-1ubuntu10_amd64.deb) ...
Selecting previously unselected package libc-dev-bin.
Unpacking libc-dev-bin (from .../libc-dev-bin_2.17-0ubuntu5_amd64.deb) ...
Selecting previously unselected package linux-libc-dev:amd64.
Unpacking linux-libc-dev:amd64 (from .../linux-libc-dev_3.8.0-23.34_amd64.deb) ...
Selecting previously unselected package libc6-dev:amd64.
Unpacking libc6-dev:amd64 (from .../libc6-dev_2.17-0ubuntu5_amd64.deb) ...
Selecting previously unselected package libpcre3-dev:amd64.
Unpacking libpcre3-dev:amd64 (from .../libpcre3-dev_1%3a8.31-2_amd64.deb) ...
Selecting previously unselected package manpages-dev.
Unpacking manpages-dev (from .../manpages-dev_3.44-0ubuntu1_all.deb) ...
Processing triggers for man-db ...
Setting up libgmpxx4ldbl:amd64 (2:5.0.5+dfsg-2ubuntu3) ...
Setting up libppl12:amd64 (1.0-1ubuntu2) ...
Setting up libppl-c4:amd64 (1.0-1ubuntu2) ...
Setting up libcloog-ppl1:amd64 (0.16.1-1) ...
Setting up libgomp1:amd64 (4.7.3-1ubuntu1) ...
Setting up libitm1:amd64 (4.7.3-1ubuntu1) ...
Setting up libmpfr4:amd64 (3.1.1-1) ...
Setting up libpcrecpp0:amd64 (1:8.31-2) ...
Setting up libquadmath0:amd64 (4.7.3-1ubuntu1) ...
Setting up libmpc2:amd64 (0.9-4build1) ...
Setting up binutils (2.23.2-2ubuntu1) ...
Setting up cpp-4.7 (4.7.3-1ubuntu1) ...
Setting up cpp (4:4.7.3-1ubuntu10) ...
Setting up libgcc-4.7-dev:amd64 (4.7.3-1ubuntu1) ...
Setting up gcc-4.7 (4.7.3-1ubuntu1) ...
Setting up gcc (4:4.7.3-1ubuntu10) ...
Setting up libc-dev-bin (2.17-0ubuntu5) ...
Setting up linux-libc-dev:amd64 (3.8.0-23.34) ...
Setting up libc6-dev:amd64 (2.17-0ubuntu5) ...
Setting up libpcre3-dev:amd64 (1:8.31-2) ...
Setting up manpages-dev (3.44-0ubuntu1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
ubuntu@ip-10-224-3-150:~$ sudo apt-get install julia -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  git git-man libamd2.2.0 libarpack2 libcholmod1.7.1 libcolamd2.7.1
  libdouble-conversion0 liberror-perl libfftw3-3 libfftw3-double3
  libfftw3-single3 libgfortran3 liblapack3 libopenblas-base libopenlibm-base
  librmath-base librmath-dev libumfpack5.4.0 libunwind8
Suggested packages:
  git-daemon-run git-daemon-sysvinit git-doc git-el git-arch git-cvs git-svn
  git-email git-gui gitk gitweb ess libfftw3-bin libfftw3-dev
The following NEW packages will be installed:
  git git-man julia libamd2.2.0 libarpack2 libcholmod1.7.1 libcolamd2.7.1
  libdouble-conversion0 liberror-perl libfftw3-3 libfftw3-double3
  libfftw3-single3 libgfortran3 liblapack3 libopenblas-base libopenlibm-base
  librmath-base librmath-dev libumfpack5.4.0 libunwind8
0 upgraded, 20 newly installed, 0 to remove and 16 not upgraded.
Need to get 38.2 MB of archives.
After this operation, 114 MB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libfftw3-double3 amd64 3.3.3-2ubuntu1 [662 kB]
Get:2 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main libamd2.2.0 amd64 1:4.2.0-juliadeps1~raring [29.7 kB]
Get:3 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libfftw3-single3 amd64 3.3.3-2ubuntu1 [696 kB]
Get:4 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libgfortran3 amd64 4.7.3-1ubuntu1 [349 kB]
Get:5 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main libunwind8 amd64 1.0.1-4ubuntu2 [55.9 kB]
Get:6 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/universe libdouble-conversion0 amd64 1.1.1-1 [33.0 kB]
Get:7 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main libcolamd2.7.1 amd64 1:4.2.0-juliadeps1~raring [23.2 kB]
Get:8 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/universe libopenblas-base amd64 0.2.6-1~exp1 [4,356 kB]
Get:9 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main libcholmod1.7.1 amd64 1:4.2.0-juliadeps1~raring [410 kB]
Get:10 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main liblapack3 amd64 3.4.2-1~exp3 [5,066 kB]
Get:11 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/universe libfftw3-3 amd64 3.3.3-2ubuntu1 [1,640 B]
Get:12 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/universe libarpack2 amd64 3.1.2-2~exp1 [128 kB]
Get:13 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main liberror-perl all 0.17-1 [23.8 kB]
Get:14 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main git-man all 1:1.8.1.2-1 [653 kB]
Get:15 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ raring/main git amd64 1:1.8.1.2-1 [6,863 kB]
Get:16 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main libumfpack5.4.0 amd64 1:4.2.0-juliadeps1~raring [367 kB]
Get:17 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main libopenlibm-base amd64 0.1~gitc9cf16d-juliadeps4~raring [159 kB]
Get:18 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main librmath-base amd64 2.15.2-juliadeps2~raring [102 kB]
Get:19 http://ppa.launchpad.net/staticfloat/julia-deps/ubuntu/ raring/main librmath-dev amd64 2.15.2-juliadeps2~raring [9,952 B]
Get:20 http://ppa.launchpad.net/staticfloat/julianightlies/ubuntu/ raring/main julia amd64 0.2.0+nightly20130531~raring1 [18.2 MB]
Fetched 38.2 MB in 23s (1,605 kB/s)                                            
Selecting previously unselected package libfftw3-double3:amd64.
(Reading database ... 55790 files and directories currently installed.)
Unpacking libfftw3-double3:amd64 (from .../libfftw3-double3_3.3.3-2ubuntu1_amd64.deb) ...
Selecting previously unselected package libfftw3-single3:amd64.
Unpacking libfftw3-single3:amd64 (from .../libfftw3-single3_3.3.3-2ubuntu1_amd64.deb) ...
Selecting previously unselected package libgfortran3:amd64.
Unpacking libgfortran3:amd64 (from .../libgfortran3_4.7.3-1ubuntu1_amd64.deb) ...
Selecting previously unselected package libunwind8.
Unpacking libunwind8 (from .../libunwind8_1.0.1-4ubuntu2_amd64.deb) ...
Selecting previously unselected package libdouble-conversion0:amd64.
Unpacking libdouble-conversion0:amd64 (from .../libdouble-conversion0_1.1.1-1_amd64.deb) ...
Selecting previously unselected package libopenblas-base.
Unpacking libopenblas-base (from .../libopenblas-base_0.2.6-1~exp1_amd64.deb) ...
Selecting previously unselected package liblapack3.
Unpacking liblapack3 (from .../liblapack3_3.4.2-1~exp3_amd64.deb) ...
Selecting previously unselected package libfftw3-3:amd64.
Unpacking libfftw3-3:amd64 (from .../libfftw3-3_3.3.3-2ubuntu1_amd64.deb) ...
Selecting previously unselected package libamd2.2.0.
Unpacking libamd2.2.0 (from .../libamd2.2.0_1%3a4.2.0-juliadeps1~raring_amd64.deb) ...
Selecting previously unselected package libcolamd2.7.1.
Unpacking libcolamd2.7.1 (from .../libcolamd2.7.1_1%3a4.2.0-juliadeps1~raring_amd64.deb) ...
Selecting previously unselected package libcholmod1.7.1.
Unpacking libcholmod1.7.1 (from .../libcholmod1.7.1_1%3a4.2.0-juliadeps1~raring_amd64.deb) ...
Selecting previously unselected package libumfpack5.4.0.
Unpacking libumfpack5.4.0 (from .../libumfpack5.4.0_1%3a4.2.0-juliadeps1~raring_amd64.deb) ...
Selecting previously unselected package libarpack2.
Unpacking libarpack2 (from .../libarpack2_3.1.2-2~exp1_amd64.deb) ...
Selecting previously unselected package libopenlibm-base.
Unpacking libopenlibm-base (from .../libopenlibm-base_0.1~gitc9cf16d-juliadeps4~raring_amd64.deb) ...
Selecting previously unselected package librmath-base.
Unpacking librmath-base (from .../librmath-base_2.15.2-juliadeps2~raring_amd64.deb) ...
Selecting previously unselected package librmath-dev.
Unpacking librmath-dev (from .../librmath-dev_2.15.2-juliadeps2~raring_amd64.deb) ...
Selecting previously unselected package julia.
Unpacking julia (from .../julia_0.2.0+nightly20130531~raring1_amd64.deb) ...
Selecting previously unselected package liberror-perl.
Unpacking liberror-perl (from .../liberror-perl_0.17-1_all.deb) ...
Selecting previously unselected package git-man.
Unpacking git-man (from .../git-man_1%3a1.8.1.2-1_all.deb) ...
Selecting previously unselected package git.
Unpacking git (from .../git_1%3a1.8.1.2-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up libfftw3-double3:amd64 (3.3.3-2ubuntu1) ...
Setting up libfftw3-single3:amd64 (3.3.3-2ubuntu1) ...
Setting up libgfortran3:amd64 (4.7.3-1ubuntu1) ...
Setting up libunwind8 (1.0.1-4ubuntu2) ...
Setting up libdouble-conversion0:amd64 (1.1.1-1) ...
Setting up libopenblas-base (0.2.6-1~exp1) ...
update-alternatives: using /usr/lib/openblas-base/libopenblas.so.0 to provide /usr/lib/libblas.so.3 (libblas.so.3) in auto mode
Setting up liblapack3 (3.4.2-1~exp3) ...
update-alternatives: using /usr/lib/lapack/liblapack.so.3 to provide /usr/lib/liblapack.so.3 (liblapack.so.3) in auto mode
Setting up libfftw3-3:amd64 (3.3.3-2ubuntu1) ...
Setting up libamd2.2.0 (1:4.2.0-juliadeps1~raring) ...
Setting up libcolamd2.7.1 (1:4.2.0-juliadeps1~raring) ...
Setting up libcholmod1.7.1 (1:4.2.0-juliadeps1~raring) ...
Setting up libumfpack5.4.0 (1:4.2.0-juliadeps1~raring) ...
Setting up libarpack2 (3.1.2-2~exp1) ...
Setting up libopenlibm-base (0.1~gitc9cf16d-juliadeps4~raring) ...
Setting up librmath-base (2.15.2-juliadeps2~raring) ...
Setting up librmath-dev (2.15.2-juliadeps2~raring) ...
Setting up julia (0.2.0+nightly20130531~raring1) ...
Setting up liberror-perl (0.17-1) ...
Setting up git-man (1:1.8.1.2-1) ...
Setting up git (1:1.8.1.2-1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
ubuntu@ip-10-224-3-150:~$ julia
Illegal instruction (core dumped)
ubuntu@ip-10-224-3-150:~$ 

[ViralBShah]

That is odd. @amitmurthy and I have been using the nightlies on successfully on EC2, but it may have been on the small instances.

We will soon have an AMI and the ability to do addprocs_ec2(n).

[amitmurthy]

Yes, just to confirm, they have been 'small' instances - julia runs without any issues on them. As @staticfloat pointed out, maybe Amazon uses a different virtualization technology for the micro instances.

Also, apparently the micro instances do not come with any swap space configured. Which means that we have a total of 600MB RAM. I don't know how much memory the JIT compiler / LLVM requires at startup, but I wonder if this is just a out-of-memory situation.

[arnim]

Yes, I've been running julia on t1.micro (Amazon Linux AMI 2013.03.1) so it's possible. Seems to be related to Ubuntu.
Sadly I had to build there julia (in a very long build process) on my own / no apt-get availible just yum.
@ViralBShah If there would be an AMI ... that would be great ;)

[amitmurthy]

I just tried with the following AMIs on micro instances using the nightlies and and it worked fine. ami-d96015b0 (32 bit) and ami-9b691cf2(64 bit).

AWS region - N. Virginia (US East 1)

[ViralBShah]

Thanks @amitmurthy

I guess we are reaching a point where we should maintain AMIs going forward, especially since AWS.jl makes it so easy to use julia with EC2.

[staticfloat]

I'm going to try compiling from scratch on the default "free" AMI, and see if I can reproduce this error.

[cmey]

@arnim @amitmurthy I can't make it work on my default AMI micro instance OS (ami-ddc1d7a9, Ireland) by compiling from source. It's the same error. Thus it seems not related to Ubuntu.

[ec2-user@ip-172-31-0-220 julia]$ make
    LINK usr/bin/julia-release-readline
    JULIA usr/lib/julia/sys.ji
int.jl
/bin/sh: line 1:  8359 Instruction non permise /home/ec2-user/julia/usr/bin/julia-release-readline -bf sysimg.jl
*** This error is usually fixed by running 'make clean'. If the error persists, try 'make cleanall'. ***

I ended up with a french OS for some reason, but the error is the same "illegal instruction" and occurs at the same place (as my 1st report on Ubuntu).

[ec2-user@ip-172-31-0-220 julia]$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
stepping        : 7
microcode       : 0x70a
cpu MHz         : 1799.999
cache size      : 20480 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht nx constant_tsc up nonstop_tsc pni pclmulqdq ssse3 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes avx hypervisor
bogomips        : 3599.99
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Now trying on the exact same AMI ID as you guys report it worked.

[staticfloat]

The 0.1.2 binaries work fine for me on a 64-bit free tier Ubuntu machine I just instantiated. Similarly, my 0.2.0 nightlies work fine as well. I'm currently trying building from source, but I get the feeling that will probably work just fine as well.

My /proc/cpuinfo is also slightly different than yours:

$ cat /proc/cpuinfo                                                                                     [2/4]
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 44
model name      : Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
stepping        : 2
microcode       : 0x13
cpu MHz         : 2000.070
cache size      : 12288 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 36
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc re
p_good nopl nonstop_tsc pni ssse3 cx16 pcid sse4_1 sse4_2 popcnt hypervisor lahf_lm
bogomips        : 4000.14
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

The key thing to note is that @arnim's CPU advertises AVX, whereas mine does not. This blog post has a rather interesting view on this, their solution is to avoid a certain CPU family (The same CPU as reported by @cmey above). What would be ideal, is if we can somehow disable the CPU reporting AVX such that OpenBLAS chooses Nehalem kernels, instead of sandy bridge ones. I'll look into this.

[cmey]

@staticfloat I think you're on the right track!

So, it works in Virginia instances with this CPU :

ubuntu@ip-10-245-81-118:~$ cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 23
model name  : Intel(R) Xeon(R) CPU           E5430  @ 2.66GHz
stepping    : 10
microcode   : 0xa07
cpu MHz     : 2659.998
cache size  : 6144 KB
fdiv_bug    : no
hlt_bug     : no
f00f_bug    : no
coma_bug    : no
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht nx constant_tsc pni ssse3 sse4_1 hypervisor
bogomips    : 5319.99
clflush size    : 64
cache_alignment : 64
address sizes   : 38 bits physical, 48 bits virtual
power management:

But not in Ireland where I get my previously reported CPU.

Thanks for looking into this :-)

[staticfloat]

Got it! I had to start up a server in Ireland, but I finally got one with the same CPU type; a Xeon E5-2650, and the error occurs. I don't want to just recompile OpenBLAS without AVX instructions, because any other pieces of software that have this problem will then also start throwing errors like this. I'm investigating what I can do on the kernel side to get it to hide those avx flags.

[xianyi]

Hi @staticfloat ,

I think I fixed the AVX detection bug on VM in issue OpenMathLib/OpenBLAS#124 . So far, OpenBLAS will detect whether OS supports it.

Xianyi

[ViralBShah]

@pao @staticfloat I think it would be useful to start using the vagrant file to run various VMs on julia.mit.edu, which is quite beefy. Would you guys be able to set it up? If so, email me your preferred usernames.

I believe that @loladiro and @vtjnash are already running Windows VMs there.

[Keno]

Yeah, in fact I was already using Vagrant to run a bunch of Julia VMs ;). Every user on the system should be able to use the existing installation, so email me if you want to talk about that.

[staticfloat]

Well, I figured out that the standard Amazon-provided Ubuntu 13.04 kernel has a noxsave parameter you can pass that will also disable AVX, which does indeed remove the avx flag from /proc/cpuinfo. This does not, however, stop the illegal instruction signal, and manually compiling OpenBLAS with OPENBLAS_TARGET_ARCH=Nehalem confirms that this isn't an AVX/OpenBLAS problem. I also compiled a test C program that links with OpenBLAS, and with the noxsave kernel parameter, OpenBLAS does automatically detect the lack of AVX support, as I was expecting.

Making a debug Julia and trying to build the system image inside of gdb results in a bogus stack trace;

Program received signal SIGILL, Illegal instruction.
0x00007ffff46d581e in ?? ()
(gdb) bt
#0  0x00007ffff46d581e in ?? ()
#1  0x0000000000000008 in ?? ()
#2  0x00007fffffffc7e0 in ?? ()
#3  0x00000000006373ad in ?? ()
#4  0x00000000009f6090 in ?? ()
#5  0x00000000007ff620 in ?? ()
#6  0xce780d24fece1700 in ?? ()
#7  0xce780d24fece1700 in ?? ()
#8  0x00000000007ff620 in ?? ()
#9  0x0000000000000077 in ?? ()
#10 0x00000000000002f2 in ?? ()
#11 0x000000000000007a in ?? ()
#12 0x00000000006373ad in ?? ()
#13 0x00007fffffffc740 in ?? ()
#14 0x00007ffff62fb836 in jl_apply (f=0x2f2, args=0x77, nargs=0)
    at julia.h:1021
Backtrace stopped: frame did not save the PC
(gdb) 

I'm doubting that there was actually a stack frame at 0x08, so my guess is somewhere along the way we jump into the great unknown. I can't imagine why that is the case on one CPU and not another, but I'm still investigating. Right now I'm trying to figure out where things start to go south; it seems to happen during compilation of int.jl, although that may be inaccurate.

It'd help if I could do some printf debugging before things like print() are defined..... it looks like there's some old debug printing code at the top of sysimg.jl that was if false'ed out of normal execution. Bringing it back just throws out some errors, I'm assuming because noone has used it for a while. How do I throw out simple strings to STDOUT before print()?

[ViralBShah]

That's some pretty impressive work!

[staticfloat]

Bumping this issue to see if anyone can tell me how to print stuff out to the console during sys.ji creation

[ViralBShah]

@JeffBezanson @loladiro @vtjnash ?

[vtjnash]

I think this is a duplicate of #3634 and will be fixed when llvm is bumped to 3.3

[cmey]

Agreed

Le 28 juil. 2013 à 10:05, Jameson Nash [email protected] a écrit :

I think this is a duplicate of #3634 and will be fixed when llvm is bumped to 3.3

—
Reply to this email directly or view it on GitHub.

[staticfloat]

Aha! Thank you @vtjnash! Updating to llvm 3.3 does indeed solve the problem. Compiling julia completely from scratch (including LLVM 3.3, OpenBLAS, etc...) took the better part of two days, but I've finally got a Julia running on an Ireland instance with the "naughty" CPU. I'm closing this, although it would be nice to have this in the documentation somewhere, I just don't know where it should go.

[vtjnash]

It can probably go in the README on the front page, although we should be moving to llvm 3.3 soon.

[staticfloat]

6b0ccb4