Enable runs on linux aarch64

[IanButterworth]

With help from @DilumAluthge

[DilumAluthge]

From a brief test locally, it looks like TOML will parse the . correctly as long as we put the artifact name in double quotes.

[gbaraldi]

Xref #97

[IanButterworth]

Can we get a AWS aarch64 instance started up and added as a repo self-hosted runner, so we can add that to CI here? Who's good to ping for that?

[DilumAluthge]

For CI here, we should just use one of the Buildkite aarch64-linux-gnu agents in the juliaecosystem queue. I've already created the Buildkite pipeline (https://buildkite.com/julialang/pkgeval-dot-jl). An example of the Buildkite pipeline.yml file for aarch64 in the juliaecosystem queue is available here: https://github.com/rr-debugger/rr/tree/master/.buildkite

When it's time to actually run the PkgEval job, then we can spin up the AWS instances.

[DilumAluthge]

@maleadt Do you need me to add xvfb inside the package_linux rootfs image?

[maleadt]

xvfb runs on the host, so we'd need Xvfb_jll.

[DilumAluthge]

xvfb runs on the host, so we'd need Xvfb_jll.

JuliaPackaging/Yggdrasil#5707

[DilumAluthge]

Alternatively, we can make a rootfs image for Buildkite that has xvfb in it.

[DilumAluthge]

Alternatively, we can make a rootfs image for Buildkite that has xvfb in it.

JuliaCI/rootfs-images#236

[maleadt]

Alternatively, we can make a rootfs image for Buildkite that has xvfb in it.

I'm not sure it's that simple. We should only launch a single xvfb instance, and it has to be accessible to each sandbox by mounting/exposing the necessary sockets and environment flags. Not impossible, just a bit of work. FWIW, running a virtual X server in each container is possible but probably generates quite a bit of overhead.

For aarch64 testing it's probably easier to disable xvfb for now and just fail tests of the (couple of) packages that require an active display server.

[IanButterworth]

For aarch64 testing it's probably easier to disable xvfb for now and just fail tests of the (couple of) packages that require an active display server.

Sounds good for a first run.

Following that, perhaps PkgEval could first check for the presence of xvfb in any .yml files in the repo under test, and if so run with it

[IanButterworth]

Probably good to LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 etc. too until that issue is fixed

[maleadt]

check for the presence of xvfb in any .yml files in the repo under test

I'm not sure I understand. The requirement is implicit because, e.g., libcairo.so accessing X11 libraries. No package currently explicitly requests a display server.

[IanButterworth]

No package currently explicitly requests a display server.

AFAIK quite a few do during CI testing

[IanButterworth]

See https://github.com/JuliaPlots/Plots.jl/blob/ff5d20d664e2b4bda8cf6097438bd35e6f8c768c/.github/workflows/ci.yml#L38

[maleadt]

Oh wait, was the rootfs change intended for the buildkite runners and not for use by PkgEval? I guess so, but was confused since PkgEval uses those images too. Let's try restarting the job.

[DilumAluthge]

Oh wait, was the rootfs change intended for the buildkite runners and not for use by PkgEval?

Yeah, sorry for the confusion. I was proposing to change the rootfs that the Buildkite agent is running inside. PkgEval will continue to use the package_linux rootfs, as it is doing currently.

[maleadt]

I was proposing to change the rootfs that the Buildkite agent is running inside.

Do the runners automatically pick up the new image? It's still reporting:

IOError: could not spawn Xvfb :1 -screen 0 1024x768x16: no such file or directory (ENOENT)

[DilumAluthge]

I was proposing to change the rootfs that the Buildkite agent is running inside.

Do the runners automatically pick up the new image? It's still reporting:

No, I have to change it manually. I'll push to this branch.

[DilumAluthge]

@staticfloat What have I done wrong here?

[DilumAluthge]

It works with a dummy command (e.g. ls), but fails if I stick another invocation of the julia plugin after I invoke the sandbox plugin.

[maleadt]

I've disabled the pipeline (emptied the steps) since it otherwise marks all builds, even outside of this PR, as broken.

[staticfloat]

It works with a dummy command (e.g. ls), but fails if I stick another invocation of the julia plugin after I invoke the sandbox plugin.

It's telling you, via the error at the top of the buildkite page:

You have two options:

1. Provide a download agent (e.g. curl, wget, etc...) within your rootfs image
2. Re-use the Julia installed as part of the outside environment by mounting /cache into the sandbox.