Skip to content

{CI} Use Ubuntu 24.04 on ARM agent #31158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bebound merged 4 commits into from
Apr 1, 2025
Merged

{CI} Use Ubuntu 24.04 on ARM agent #31158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cf3e2e2
Init
bebound Mar 31, 2025
f479d62
Init
bebound Mar 31, 2025
d23c3e5
Merge branch 'use-24.04' of https://github.com/bebound/azure-cli into…
bebound Apr 1, 2025
4263ae1
Minor fix
bebound Apr 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azure-pipelines/templates/variables.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ variables:
ubuntu_pool: 'pool-ubuntu-2204'
ubuntu_multi_core_pool: 'pool-ubuntu-latest-multi-core'
windows_pool: 'pool-windows-2019'
ubuntu_arm64_pool: 'ubuntu-arm64-2004-pool'
ubuntu_arm64_pool: 'pool-ubuntu-latest-arm64'
Copy link
Member

@jiasli jiasli Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember we decided not to use the name latest?

Copy link
Contributor Author

@bebound bebound Apr 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the latest version saves effort by avoiding changes to the pool reference in CI and release pipelines, but it loses the ability to revert back if we encounter severe issues.

We need to be cautious during the agent transition to eliminate the issue we met during the 20.04 deprecation. How about this? We create a temp pool with newer image when we need to update the agent, then test it in the main repository. If everything works fine, we'll update the latest pool with the newer image.

@wangzelin007 for awareness

macos_pool: 'macOS-14'
2 changes: 1 addition & 1 deletion azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ parameters:
pool: pool-ubuntu-latest-multi-core
- name: ARM64
value: arm64
pool: ubuntu-arm64-2004-pool
pool: pool-ubuntu-latest-arm64

jobs:
- job: CheckPullRequest
Expand Down
13 changes: 5 additions & 8 deletions scripts/ci/install_docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,8 @@ if [[ $(dpkg --print-architecture) == "amd64" ]]; then
echo "Docker is already installed on AMD64"
exit 0
fi
# https://docs.docker.com/engine/security/rootless/
Copy link
Contributor Author

@bebound bebound Mar 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not able to install rootless docker in 22.04 and 24.04.

Ubuntu use cgroup v2 after 22.04. Docker keeps raising docker: Error response from daemon: open /sys/fs/cgroup/user.slice/user-1000.slice/cgroup.controllers: no such file or directory when run docker, but /sys/fs/cgroup/user.slice/ and other files exist. I guess user-1000 is not logged in and this folder does not exist.

+ ls /sys/fs/cgroup/user.slice/
cgroup.controllers
cgroup.events
cgroup.freeze
cgroup.kill
cgroup.max.depth
cgroup.max.descendants
cgroup.pressure
cgroup.procs
cgroup.stat
cgroup.subtree_control
cgroup.threads
cgroup.type
cpu.idle
cpu.max
cpu.max.burst
cpu.pressure
cpu.stat
cpu.stat.local
cpu.uclamp.max
cpu.uclamp.min
cpu.weight
cpu.weight.nice
cpuset.cpus
cpuset.cpus.effective
cpuset.cpus.exclusive
cpuset.cpus.exclusive.effective
cpuset.cpus.partition
cpuset.mems
cpuset.mems.effective
io.max
io.pressure
io.prio.class
io.stat
io.weight
memory.current
memory.events
memory.events.local
memory.high
memory.low
memory.max
memory.min
memory.numa_stat
memory.oom.group
memory.peak
memory.pressure
memory.reclaim
memory.stat
memory.swap.current
memory.swap.events
memory.swap.high
memory.swap.max
memory.swap.peak
memory.zswap.current
memory.zswap.max
memory.zswap.writeback
pids.current
pids.events
pids.max
pids.peak

This issue is the same as this one, but there has been no response. https://forums.docker.com/t/docker-rootless-mode-issues-on-ubuntu-server-22-04-lts-in-aws/145115

Copy link
Member

@jiasli jiasli Mar 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The document https://docs.docker.com/engine/security/rootless/ still exists and is valid. Is user-1000.slice-not-exist an ADO-specific issue?

Copy link
Contributor Author

@bebound bebound Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Some say user-1000.slice is created when a user logs in or has a running process, but I can't find any reference.

/bin/bash -c "$(curl -fsSL https://get.docker.com)"
sudo apt-get install -y uidmap
dockerd-rootless-setuptool.sh install
export XDG_RUNTIME_DIR=/home/cloudtest/.docker/run
PATH=/usr/bin:/sbin:/usr/sbin:$PATH dockerd-rootless.sh &
sleep 5
docker context use rootless

# https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh
sudo chmod 666 /var/run/docker.sock
Copy link
Contributor Author

@bebound bebound Mar 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use chmod 666 to ensure docker can be used without sudo like runner-image agent.

The official way to do this is add user to docker group. In runner-image repo, this is achieved by adding user to docker group: https://github.com/actions/runner-images/blob/86ef9be8745af78c1e11cce2921532217216c90b/images/ubuntu/scripts/build/install-docker.sh#L49

But we can't re-login for it to take effect in the pipeline.

Copy link
Member

@jiasli jiasli Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runner-image is a general concept. I believe you are trying to mention Microsoft-hosted agents.

Copy link
Member

@jiasli jiasli Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we stick to the example at https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script, instead of piping to | sh in case something unexpected happens:

image

Copy link
Member

@jiasli jiasli Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use chmod 666 to ensure docker can be used without sudo like runner-image agent.

Any reference for this?

Copy link
Contributor Author

@bebound bebound Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use chmod 666 to ensure docker can be used without sudo like runner-image agent.

Any reference for this?

This error is raised when run docker run hello-world without sudo: docker: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Head "[http://%2Fvar%2Frun%2Fdocker.sock/_ping"](http://%2Fvar%2Frun%2Fdocker.sock/_ping%22): dial unix /var/run/docker.sock: connect: permission denied.

Change it to 666 is a general way to fix the permission issue.