Julia AdaGram process is giving bus error whiile loading large data in Docker container

[RamIdavalapaati]

Julia AdaGram process is giving bus error for more than 20MB data file in the docker container.

It is working good up to 20 MB. If I use more than 20 MB then it is giving bus error.

Error

signal (7): Bus error
while loading no file, in expression starting on line 0
macro expansion at ./cartesian.jl:62 [inlined]
macro expansion at ./multidimensional.jl:429 [inlined]
_unsafe_batchsetindex! at ./multidimensional.jl:421
_setindex! at ./multidimensional.jl:370 [inlined]
setindex! at ./abstractarray.jl:832 [inlined]
#9 at /root/.julia/v0.5/AdaGram/src/AdaGram.jl:64
#600 at ./multi.jl:1030
run_work_thunk at ./multi.jl:1001
run_work_thunk at ./multi.jl:1010 [inlined]
#597 at ./event.jl:68
unknown function (ip: 0x7fe1822db16f)
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1942
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1392 [inlined]
start_task at /home/centos/buildbot/slave/package_tarball64/build/src/task.c:253
unknown function (ip: 0xffffffffffffffff)
Allocations: 9661042 (Pool: 9659980; Big: 1062); GC: 16
Bus error (core dumped)

Steps to reproduce the behavior

Install docker-compose
$ curl -L https://github.com/docker/compose/releases/download/1.12.0/docker-compose-uname -s-uname -m > ./docker-compose
$ sudo mv ./docker-compose /usr/bin/docker-compose
$ sudo chmod +x /usr/bin/docker-compose

Dockerfile

FROM ubuntu:16.04

WORKDIR /julia

RUN apt-get -y update

RUN apt-get -y install unzip

RUN apt-get -y install cmake

RUN apt-get -y install clang

RUN apt-get -y install wget

RUN cd /tmp/

RUN wget "https://julialang.s3.amazonaws.com/bin/linux/x64/0.5/julia-0.5.0-linux-x86_64.tar.gz"

RUN tar -xzvf julia-0.5.0-linux-x86_64.tar.gz

RUN mv julia-3c9d75391c/ ~/julia

ENV PATH="/root/julia/bin:${PATH}"

RUN julia --eval 'Pkg.add("JSON")'

RUN julia --eval 'Pkg.add("HttpServer")'

RUN julia --eval 'Pkg.add("URIParser")'

RUN julia --eval 'Pkg.clone("https://github.com/deep-compute/AdaGram.jl.git")'

RUN julia --eval 'Pkg.build("AdaGram")'

CMD ["julia", "/tmp/adagram_server.jl", "80", "/julia/full.embed"]

Build docker image:
$ sudo docker build -t ramidavalapati/julia:v-1

docker-compose.yml

version: "3.1"
services:
    julia:
        image: ramidavalapati/julia:v-1
        volumes:
            - /home/ram/adagram_data/40MBfull.embed:/julia/40MBfull.embed
        ports:
            - 8080:80
        command: ["sleep", "1h"]

Next I am doing

sudo docker-compose up
sudo docker exec -it $(sudo docker-compose ps -q julia) bash
root@3748d5958f77:/julia# julia
julia> using AdaGram
julia> AdaGram.load_model("/julia/40MBfull.embed")

link to 40MBfull.embed file
link to 20MBfull.embed file

[vchuravy]

Please do not double post, https://stackoverflow.com/questions/47029360/julia-is-not-working-for-loading-large-data-in-docker-container

As the comment on SO said it is unlikely to be a problem in Julia and is more likely to be related to the configuration of the host your docker container is running on.

[yuyichao]

Dup of #19111 #19956 #17061 #18274 #17987. The fault is clearly raised on access of array that didn't fail allocation but doesn't have enough backing memory.