Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Don't crash updating UID/GID when the image's platform is different from the native CPU arch #746

Merged
merged 14 commits into from
Feb 28, 2024

Conversation

trxcllnt
Copy link
Contributor

@trxcllnt trxcllnt commented Feb 6, 2024

If the user changes the platform to a non-native arch as suggested here, the devcontainers CLI fails to launch due to attempting to load the wrong platform during the "updateUID" image build.

Opening as a draft PR to show the behavior. Will follow up with a fix once I figure out where it should go.

@chrmarti
Copy link
Contributor

chrmarti commented Feb 7, 2024

CI appears to fail because no emulator is registered.

It works for me on an AMD64 host with an ARM64 image. Could you append the log from the failure?

I'm using the following for registering the emulator:

docker run --privileged --rm tonistiigi/binfmt --install all

@trxcllnt
Copy link
Contributor Author

trxcllnt commented Feb 7, 2024

I have the emulator installed locally and see an error building an ARM64 image from an AMD64 host:

  1) Dev Containers CLI
       updateUID
         should update UID and GID when the platform is linux/arm64:
     Error: the object {
  "error": "Error: Command failed: npx --prefix src/test/tmp devcontainer up --workspace-folder /home/ptaylor/dev/devcontainers/cli/src/test/configs/updateUIDarm64 --buildkit=never \n[2024-02-06T17:57:27.938Z] @devcontainers/cli 0.56.1. Node.js v20.10.0. linux 5.15.0-92-generic x64.\n[2024-02-06T17:57:29.626Z] Start: Run: docker build -f /tmp/devcontainercli-ptaylor/container-features/0.56.1-1707242249624/Dockerfile-with-features -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /home/ptaylor/dev/devcontainers/cli/src/test/configs/updateUIDarm64\n[2024-02-06T17:57:30.018Z] #0 building with \"default\" instance using docker driver\r\n\r\n#1 [internal] load build definition from Dockerfile-with-features\r\n#1 transferring dockerfile: 368B done\r\n#1 DONE 0.0s\r\n\n[2024-02-06T17:57:30.018Z] \r\n#2 [internal] load metadata for docker.io/library/debian:latest\r\n\n[2024-02-06T17:57:30.772Z] #2 DONE 0.9s\r\n\n[2024-02-06T17:57:30.959Z] \r\n#3 [internal] load .dockerignore\r\n#3 transferring context: 2B done\r\n#3 DONE 0.0s\r\n\r\n#4 [dev_container_auto_added_stage_label 1/3] FROM docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9\r\n#4 resolve docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 0.0s done\r\n#4 sha256:a250db2ed1e0cdcca7baf72084ba6c4578b69cd984367bd6616abe24c92d61a6 529B / 529B done\r\n\n[2024-02-06T17:57:30.959Z] #4 sha256:e244810c32b8849593705821bec8d7399b3feae991f2b9e8096e58496aa4d594 1.48kB / 1.48kB done\r\n#4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0B / 49.62MB 0.1s\r\n#4 sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 1.85kB / 1.85kB done\r\n\n[2024-02-06T17:57:31.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 3.15MB / 49.62MB 1.0s\r\n\n[2024-02-06T17:57:32.160Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 6.29MB / 49.62MB 1.3s\r\n\n[2024-02-06T17:57:32.559Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 9.44MB / 49.62MB 1.7s\r\n\n[2024-02-06T17:57:32.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 12.58MB / 49.62MB 1.9s\r\n\n[2024-02-06T17:57:32.959Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 15.73MB / 49.62MB 2.1s\r\n\n[2024-02-06T17:57:33.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 18.87MB / 49.62MB 2.4s\r\n\n[2024-02-06T17:57:33.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 23.07MB / 49.62MB 2.9s\r\n\n[2024-02-06T17:57:34.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 26.21MB / 49.62MB 3.4s\r\n\n[2024-02-06T17:57:34.458Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 29.36MB / 49.62MB 3.6s\r\n\n[2024-02-06T17:57:34.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 33.55MB / 49.62MB 3.9s\r\n\n[2024-02-06T17:57:35.059Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 36.70MB / 49.62MB 4.2s\r\n\n[2024-02-06T17:57:35.459Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 40.89MB / 49.62MB 4.6s\r\n\n[2024-02-06T17:57:35.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 45.09MB / 49.62MB 5.0s\r\n\n[2024-02-06T17:57:36.560Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s\r\n\n[2024-02-06T17:57:36.713Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s done\r\n#4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0.1s\r\n\n[2024-02-06T17:57:37.754Z] #4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 1.0s done\r\n#4 DONE 6.9s\r\n\n[2024-02-06T17:57:37.909Z] \r\n#5 [dev_container_auto_added_stage_label 2/3] RUN addgroup --gid 4321 foo\r\n\n[2024-02-06T17:57:38.334Z] #5 0.576 Adding group `foo' (GID 4321) ...\r\n\n[2024-02-06T17:57:38.456Z] #5 0.698 Done.\r\n\n[2024-02-06T17:57:38.563Z] #5 DONE 0.8s\r\n\n[2024-02-06T17:57:38.718Z] \r\n#6 [dev_container_auto_added_stage_label 3/3] RUN adduser --uid 1234 --gid 4321 foo\r\n\n[2024-02-06T17:57:39.226Z] #6 0.658 Adding user `foo' ...\r\n#6 0.658 Adding new user `foo' (1234) with group `foo (4321)' ...\r\n\n[2024-02-06T17:57:39.424Z] #6 0.857 Creating home directory `/home/foo' ...\r\n#6 0.857 Copying files from `/etc/skel' ...\r\n\n[2024-02-06T17:57:39.559Z] #6 0.948 New password: Password change has been aborted.\r\n#6 0.950 passwd: Authentication token manipulation error\r\n#6 0.950 passwd: password unchanged\r\n#6 0.955 Use of uninitialized value $answer in chop at /usr/sbin/adduser line 740.\r\n\n[2024-02-06T17:57:39.559Z] #6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 741.\r\n#6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 742.\r\n#6 0.956 Try again? [y/N] Changing the user information for foo\r\n#6 0.991 Enter the new value, or press ENTER for the default\r\n#6 0.991 \tFull Name []: \n[2024-02-06T17:57:39.748Z] \tRoom Number []: \tWork Phone []: \tHome Phone []: \tOther []: Use of uninitialized value $answer in chop at /usr/sbin/adduser line 766.\r\n#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 767.\r\n#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 768.\r\n#6 1.048 Is the information correct? [Y/n] Adding new user `foo' to supplemental / extra groups `users' ...\r\n#6 1.048 Adding user `foo' to group `users' ...\r\n#6 DONE 1.2s\r\n\n[2024-02-06T17:57:39.861Z] \r\n#7 exporting to image\r\n#7 exporting layers 0.1s done\r\n#7 writing image sha256:a4c57c263826803f0a70ad1c5e7bf6ae73a7f3ce838c834af648255959735832 done\r\n#7 naming to docker.io/library/vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 0.0s done\r\n\n[2024-02-06T17:57:39.861Z] #7 DONE 0.1s\r\n\nError: Command failed: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.1 -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3-uid --build-arg BASE_IMAGE=vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --build-arg REMOTE_USER=foo --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/devcontainercli-ptaylor/empty-folder\n    at O$ (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:1253)\n    at eK (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:997)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async gAA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:3660)\n    at async BC (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:4775)\n    at async xeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11265)\n    at async UeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11006)\n"
  "stderr": "[2024-02-06T17:57:27.938Z] @devcontainers/cli 0.56.1. Node.js v20.10.0. linux 5.15.0-92-generic x64.\n[2024-02-06T17:57:29.626Z] Start: Run: docker build -f /tmp/devcontainercli-ptaylor/container-features/0.56.1-1707242249624/Dockerfile-with-features -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /home/ptaylor/dev/devcontainers/cli/src/test/configs/updateUIDarm64\n[2024-02-06T17:57:30.018Z] #0 building with \"default\" instance using docker driver\r\n\r\n#1 [internal] load build definition from Dockerfile-with-features\r\n#1 transferring dockerfile: 368B done\r\n#1 DONE 0.0s\r\n\n[2024-02-06T17:57:30.018Z] \r\n#2 [internal] load metadata for docker.io/library/debian:latest\r\n\n[2024-02-06T17:57:30.772Z] #2 DONE 0.9s\r\n\n[2024-02-06T17:57:30.959Z] \r\n#3 [internal] load .dockerignore\r\n#3 transferring context: 2B done\r\n#3 DONE 0.0s\r\n\r\n#4 [dev_container_auto_added_stage_label 1/3] FROM docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9\r\n#4 resolve docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 0.0s done\r\n#4 sha256:a250db2ed1e0cdcca7baf72084ba6c4578b69cd984367bd6616abe24c92d61a6 529B / 529B done\r\n\n[2024-02-06T17:57:30.959Z] #4 sha256:e244810c32b8849593705821bec8d7399b3feae991f2b9e8096e58496aa4d594 1.48kB / 1.48kB done\r\n#4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0B / 49.62MB 0.1s\r\n#4 sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 1.85kB / 1.85kB done\r\n\n[2024-02-06T17:57:31.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 3.15MB / 49.62MB 1.0s\r\n\n[2024-02-06T17:57:32.160Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 6.29MB / 49.62MB 1.3s\r\n\n[2024-02-06T17:57:32.559Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 9.44MB / 49.62MB 1.7s\r\n\n[2024-02-06T17:57:32.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 12.58MB / 49.62MB 1.9s\r\n\n[2024-02-06T17:57:32.959Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 15.73MB / 49.62MB 2.1s\r\n\n[2024-02-06T17:57:33.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 18.87MB / 49.62MB 2.4s\r\n\n[2024-02-06T17:57:33.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 23.07MB / 49.62MB 2.9s\r\n\n[2024-02-06T17:57:34.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 26.21MB / 49.62MB 3.4s\r\n\n[2024-02-06T17:57:34.458Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 29.36MB / 49.62MB 3.6s\r\n\n[2024-02-06T17:57:34.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 33.55MB / 49.62MB 3.9s\r\n\n[2024-02-06T17:57:35.059Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 36.70MB / 49.62MB 4.2s\r\n\n[2024-02-06T17:57:35.459Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 40.89MB / 49.62MB 4.6s\r\n\n[2024-02-06T17:57:35.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 45.09MB / 49.62MB 5.0s\r\n\n[2024-02-06T17:57:36.560Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s\r\n\n[2024-02-06T17:57:36.713Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s done\r\n#4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0.1s\r\n\n[2024-02-06T17:57:37.754Z] #4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 1.0s done\r\n#4 DONE 6.9s\r\n\n[2024-02-06T17:57:37.909Z] \r\n#5 [dev_container_auto_added_stage_label 2/3] RUN addgroup --gid 4321 foo\r\n\n[2024-02-06T17:57:38.334Z] #5 0.576 Adding group `foo' (GID 4321) ...\r\n\n[2024-02-06T17:57:38.456Z] #5 0.698 Done.\r\n\n[2024-02-06T17:57:38.563Z] #5 DONE 0.8s\r\n\n[2024-02-06T17:57:38.718Z] \r\n#6 [dev_container_auto_added_stage_label 3/3] RUN adduser --uid 1234 --gid 4321 foo\r\n\n[2024-02-06T17:57:39.226Z] #6 0.658 Adding user `foo' ...\r\n#6 0.658 Adding new user `foo' (1234) with group `foo (4321)' ...\r\n\n[2024-02-06T17:57:39.424Z] #6 0.857 Creating home directory `/home/foo' ...\r\n#6 0.857 Copying files from `/etc/skel' ...\r\n\n[2024-02-06T17:57:39.559Z] #6 0.948 New password: Password change has been aborted.\r\n#6 0.950 passwd: Authentication token manipulation error\r\n#6 0.950 passwd: password unchanged\r\n#6 0.955 Use of uninitialized value $answer in chop at /usr/sbin/adduser line 740.\r\n\n[2024-02-06T17:57:39.559Z] #6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 741.\r\n#6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 742.\r\n#6 0.956 Try again? [y/N] Changing the user information for foo\r\n#6 0.991 Enter the new value, or press ENTER for the default\r\n#6 0.991 \tFull Name []: \n[2024-02-06T17:57:39.748Z] \tRoom Number []: \tWork Phone []: \tHome Phone []: \tOther []: Use of uninitialized value $answer in chop at /usr/sbin/adduser line 766.\r\n#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 767.\r\n#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 768.\r\n#6 1.048 Is the information correct? [Y/n] Adding new user `foo' to supplemental / extra groups `users' ...\r\n#6 1.048 Adding user `foo' to group `users' ...\r\n#6 DONE 1.2s\r\n\n[2024-02-06T17:57:39.861Z] \r\n#7 exporting to image\r\n#7 exporting layers 0.1s done\r\n#7 writing image sha256:a4c57c263826803f0a70ad1c5e7bf6ae73a7f3ce838c834af648255959735832 done\r\n#7 naming to docker.io/library/vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 0.0s done\r\n\n[2024-02-06T17:57:39.861Z] #7 DONE 0.1s\r\n\nError: Command failed: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.1 -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3-uid --build-arg BASE_IMAGE=vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --build-arg REMOTE_USER=foo --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/devcontainercli-ptaylor/empty-folder\n    at O$ (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:1253)\n    at eK (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:997)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async gAA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:3660)\n    at async BC (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:4775)\n    at async xeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11265)\n    at async UeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11006)\n"
  "stdout": "{\"outcome\":\"error\",\"message\":\"Command failed: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.1 -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3-uid --build-arg BASE_IMAGE=vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --build-arg REMOTE_USER=foo --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/devcontainercli-ptaylor/empty-folder\",\"description\":\"An error occurred setting up the container.\"}\n"
} was thrown, throw an Error :)

Here's stderr prettified:

[2024-02-06T17:57:27.938Z] @devcontainers/cli 0.56.1. Node.js v20.10.0. linux 5.15.0-92-generic x64.
[2024-02-06T17:57:29.626Z] Start: Run: docker build -f /tmp/devcontainercli-ptaylor/container-features/0.56.1-1707242249624/Dockerfile-with-features -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /home/ptaylor/dev/devcontainers/cli/src/test/configs/updateUIDarm64
[2024-02-06T17:57:30.018Z] #0 building with \"default\" instance using docker driver

#1 [internal] load build definition from Dockerfile-with-features
#1 transferring dockerfile: 368B done
#1 DONE 0.0s

[2024-02-06T17:57:30.018Z] 
#2 [internal] load metadata for docker.io/library/debian:latest

[2024-02-06T17:57:30.772Z] #2 DONE 0.9s

[2024-02-06T17:57:30.959Z] 
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [dev_container_auto_added_stage_label 1/3] FROM docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9
#4 resolve docker.io/library/debian:latest@sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 0.0s done
#4 sha256:a250db2ed1e0cdcca7baf72084ba6c4578b69cd984367bd6616abe24c92d61a6 529B / 529B done

[2024-02-06T17:57:30.959Z] #4 sha256:e244810c32b8849593705821bec8d7399b3feae991f2b9e8096e58496aa4d594 1.48kB / 1.48kB done
#4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0B / 49.62MB 0.1s
#4 sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9 1.85kB / 1.85kB done

[2024-02-06T17:57:31.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 3.15MB / 49.62MB 1.0s

[2024-02-06T17:57:32.160Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 6.29MB / 49.62MB 1.3s

[2024-02-06T17:57:32.559Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 9.44MB / 49.62MB 1.7s

[2024-02-06T17:57:32.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 12.58MB / 49.62MB 1.9s

[2024-02-06T17:57:32.959Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 15.73MB / 49.62MB 2.1s

[2024-02-06T17:57:33.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 18.87MB / 49.62MB 2.4s

[2024-02-06T17:57:33.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 23.07MB / 49.62MB 2.9s

[2024-02-06T17:57:34.259Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 26.21MB / 49.62MB 3.4s

[2024-02-06T17:57:34.458Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 29.36MB / 49.62MB 3.6s

[2024-02-06T17:57:34.759Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 33.55MB / 49.62MB 3.9s

[2024-02-06T17:57:35.059Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 36.70MB / 49.62MB 4.2s

[2024-02-06T17:57:35.459Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 40.89MB / 49.62MB 4.6s

[2024-02-06T17:57:35.859Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 45.09MB / 49.62MB 5.0s

[2024-02-06T17:57:36.560Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s

[2024-02-06T17:57:36.713Z] #4 sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 49.62MB / 49.62MB 5.7s done
#4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 0.1s

[2024-02-06T17:57:37.754Z] #4 extracting sha256:66932e2b787d33a94ee3eb8b489be6e6838b29f5c1d732262da306da9b1f2eed 1.0s done
#4 DONE 6.9s

[2024-02-06T17:57:37.909Z] 
#5 [dev_container_auto_added_stage_label 2/3] RUN addgroup --gid 4321 foo

[2024-02-06T17:57:38.334Z] #5 0.576 Adding group `foo' (GID 4321) ...

[2024-02-06T17:57:38.456Z] #5 0.698 Done.

[2024-02-06T17:57:38.563Z] #5 DONE 0.8s

[2024-02-06T17:57:38.718Z] 
#6 [dev_container_auto_added_stage_label 3/3] RUN adduser --uid 1234 --gid 4321 foo

[2024-02-06T17:57:39.226Z] #6 0.658 Adding user `foo' ...
#6 0.658 Adding new user `foo' (1234) with group `foo (4321)' ...

[2024-02-06T17:57:39.424Z] #6 0.857 Creating home directory `/home/foo' ...
#6 0.857 Copying files from `/etc/skel' ...

[2024-02-06T17:57:39.559Z] #6 0.948 New password: Password change has been aborted.
#6 0.950 passwd: Authentication token manipulation error
#6 0.950 passwd: password unchanged
#6 0.955 Use of uninitialized value $answer in chop at /usr/sbin/adduser line 740.

[2024-02-06T17:57:39.559Z] #6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 741.
#6 0.955 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 742.
#6 0.956 Try again? [y/N] Changing the user information for foo
#6 0.991 Enter the new value, or press ENTER for the default
#6 0.991 \tFull Name []: 
[2024-02-06T17:57:39.748Z] \tRoom Number []: \tWork Phone []: \tHome Phone []: \tOther []: Use of uninitialized value $answer in chop at /usr/sbin/adduser line 766.
#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 767.
#6 1.047 Use of uninitialized value $answer in pattern match (m//) at /usr/sbin/adduser line 768.
#6 1.048 Is the information correct? [Y/n] Adding new user `foo' to supplemental / extra groups `users' ...
#6 1.048 Adding user `foo' to group `users' ...
#6 DONE 1.2s

[2024-02-06T17:57:39.861Z] 
#7 exporting to image
#7 exporting layers 0.1s done
#7 writing image sha256:a4c57c263826803f0a70ad1c5e7bf6ae73a7f3ce838c834af648255959735832 done
#7 naming to docker.io/library/vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 0.0s done

[2024-02-06T17:57:39.861Z] #7 DONE 0.1s

Error: Command failed: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.1 -t vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3-uid --build-arg BASE_IMAGE=vsc-updateuidarm64-43db9ce0806ed8161fc09150530bdea4854e24b6a0719420c6cebc3074a816e3 --build-arg REMOTE_USER=foo --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/devcontainercli-ptaylor/empty-folder
    at O$ (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:1253)
    at eK (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:997)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async gAA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:3660)
    at async BC (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:4775)
    at async xeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11265)
    at async UeA (/home/ptaylor/dev/devcontainers/cli/src/test/tmp/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11006)

@trxcllnt
Copy link
Contributor Author

trxcllnt commented Feb 7, 2024

I'm also able to replicate this error in isolated CI environments where the --platform= arch matches the host arch (both x86 and ARM), though I can't seem to make it fail that way locally.

I get more info when I try to set --platform=linux/arm64 and open the devcontainer in VSCode. Here we can clearly see that it's failing due to docker attempting to pull an image that doesn't exist:

#27 preparing layers for inline cache
[2024-02-07T16:54:11.203Z] #27 DONE 2.1s

#28 exporting to image
#28 exporting layers done
[2024-02-07T16:54:11.311Z] #28 writing image sha256:fc3516c417d5ab3dd27b6db85c5f5e6abe3412a018b7b20d9a1de7a55b6d2bd4 done
#28 naming to docker.io/library/vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586 0.0s done
#28 DONE 0.0s
[2024-02-07T16:54:11.317Z] Stop (1181504 ms): Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /tmp/devcontainercli-ptaylor/container-features/0.56.0-1707323657417/Dockerfile-with-features -t vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586 --target dev_containers_target_stage --build-arg ARCH=arm64 --build-arg CUDA_VER=12.3 --build-arg RAPIDS_VER=24.02 --build-arg UBUNTU_VER=22.04 --build-context dev_containers_feature_content_source=/tmp/devcontainercli-ptaylor/container-features/0.56.0-1707323657417 --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp /home/ptaylor/dev/rapids/dlfw/24.02-cuda12.3-ubuntu22.04/arm64
[2024-02-07T16:54:11.320Z] Start: Run: docker inspect --type image vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586
[2024-02-07T16:54:11.341Z] Stop (21 ms): Run: docker inspect --type image vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586
[2024-02-07T16:54:11.343Z] Start: Run: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.0 -t vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586-uid --build-arg BASE_IMAGE=vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586 --build-arg REMOTE_USER=coder --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /home/ptaylor/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2024-02-07T16:54:11.687Z] #0 building with "default" instance using docker driver

#1 [internal] load build definition from updateUID.Dockerfile-0.56.0
#1 transferring dockerfile: 1.36kB done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586:latest
[2024-02-07T16:54:12.601Z] #2 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
[2024-02-07T16:54:12.662Z] ------
 > [internal] load metadata for docker.io/library/vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586:latest:
------
[2024-02-07T16:54:12.664Z] updateUID.Dockerfile-0.56.0:2
--------------------
   1 |     ARG BASE_IMAGE
   2 | >>> FROM $BASE_IMAGE
   3 |     
   4 |     USER root
--------------------
ERROR: failed to solve: vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
[2024-02-07T16:54:12.670Z] Stop (1327 ms): Run: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.0 -t vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586-uid --build-arg BASE_IMAGE=vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586 --build-arg REMOTE_USER=coder --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /home/ptaylor/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2024-02-07T16:54:12.671Z] Error: Command failed: docker build -f /tmp/devcontainercli-ptaylor/updateUID.Dockerfile-0.56.0 -t vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586-uid --build-arg BASE_IMAGE=vsc-dlfw-1161d4fc6643bc461b5a49850f1cf1dfe0541348d6472711c7e3d058d6fa0586 --build-arg REMOTE_USER=coder --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /home/ptaylor/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2024-02-07T16:54:12.671Z]     at J$ (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:464:1253)
[2024-02-07T16:54:12.671Z]     at $J (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:464:997)
[2024-02-07T16:54:12.671Z]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2024-02-07T16:54:12.671Z]     at async tAA (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:481:3660)
[2024-02-07T16:54:12.671Z]     at async CC (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:481:4775)
[2024-02-07T16:54:12.671Z]     at async NeA (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:614:11107)
[2024-02-07T16:54:12.672Z]     at async MeA (/home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:614:10848)
[2024-02-07T16:54:12.676Z] Stop (1198193 ms): Run: /usr/share/code/code /home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/ptaylor/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-dd990ee0-938e-47d7-9f14-8a6ee3959c5d1707323653642 --workspace-folder /home/ptaylor/dev/rapids/dlfw --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/ptaylor/dev/rapids/dlfw --id-label devcontainer.config_file=/home/ptaylor/dev/rapids/dlfw/.devcontainer/24.02-cuda12.3-dev-ubuntu22.04-arm64/devcontainer.json --log-level debug --log-format json --config /home/ptaylor/dev/rapids/dlfw/.devcontainer/24.02-cuda12.3-dev-ubuntu22.04-arm64/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-02-07T16:54:12.677Z] Exit code 1
[2024-02-07T16:54:12.678Z] Command failed: /usr/share/code/code /home/ptaylor/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/ptaylor/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-dd990ee0-938e-47d7-9f14-8a6ee3959c5d1707323653642 --workspace-folder /home/ptaylor/dev/rapids/dlfw --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/ptaylor/dev/rapids/dlfw --id-label devcontainer.config_file=/home/ptaylor/dev/rapids/dlfw/.devcontainer/24.02-cuda12.3-dev-ubuntu22.04-arm64/devcontainer.json --log-level debug --log-format json --config /home/ptaylor/dev/rapids/dlfw/.devcontainer/24.02-cuda12.3-dev-ubuntu22.04-arm64/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-02-07T16:54:12.678Z] Exit code 1

@trxcllnt
Copy link
Contributor Author

trxcllnt commented Feb 7, 2024

@chrmarti I just pushed a commit that fixes the test for me locally. The fix is a bit complicated, so I had to touch a few different places:

  1. Add a platformInfo field to DockerResolverParameters and DockerCLIParameters.
  2. Parse the buildxPlatform string into a PlatformInfo in createDockerParams.
    • Default to { os: cliHost.platform, arch: cliHost.arch } if buildxPlatform is undefined.
  3. Pass platformInfo into the inspectDockerImage -> inspectImageInRegistry call.
  4. Update inspectImageInRegistry to return the the platformInfo that it was called with.
    • Ideally we could read these values from the API calls, but those fields are only in the image index entry and not the manifest, so they aren't available if the CLI takes this conditional branch.
    • But since we're now passing the actual platformInfo to inspectImageInRegistry, the getImageIndexEntryForPlatform call should fail if there's no image for the input platform.
  5. Update the inspectDockerImage -> inspectImage call to return Os and Architecture fields
    • This ensures it aligns with the JSON shape of docker inspect ...
    • Add Os and Architecture fields to the ImageDetails interface
  6. And finally, add the --platform arg to updateRemoteUserUID so docker uses the correct base image.

I started at step 6 and worked my way backwards until my new tests passed, so I'm sure I didn't update everywhere that needs it.

@trxcllnt
Copy link
Contributor Author

The CI test failures seem to be temporary apt repo/network issues:

6.081 E: Failed to fetch https://packages.microsoft.com/repos/microsoft-debian-bullseye-prod/pool/main/m/moby-buildx/moby-buildx_0.12.1-debian11u1_amd64.deb  File has unexpected size (34140374 != 34084902). Mirror sync in progress? [IP: 104.42.185.173 443]

Hopefully re-running should resolve them.

@chrmarti chrmarti marked this pull request as ready for review February 15, 2024 21:10
@chrmarti chrmarti requested a review from a team as a code owner February 15, 2024 21:10
@chrmarti
Copy link
Contributor

Oops, didn't mean to bring it out of draft state. It looks promising on first glance, I have triggered CI now.

Copy link
Contributor

@chrmarti chrmarti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, left a few comments. Part of the complexity seems to come from the fact that we might not have pulled the base image yet, otherwise we could just always inspect that and wouldn't have to pass the platform around?

src/spec-node/devContainers.ts Outdated Show resolved Hide resolved
src/spec-node/devContainersSpecCLI.ts Outdated Show resolved Hide resolved
@trxcllnt
Copy link
Contributor Author

@chrmarti Part of the complexity seems to come from the fact that we might not have pulled the base image yet, otherwise we could just always inspect that and wouldn't have to pass the platform around?

We're inspecting the image first (step 3 below), but we were ignoring its Os and Architecture fields.

  1. updateRemoteUserUID needs to pass --platform, and it gets that info by calling getRemoteUserUIDUpdateDetails.
  2. getRemoteUserUIDUpdateDetails delegates to the imageDetails argument (which AFAIK is inspectDockerImage).
  3. inspectDockerImage calls inspectImage, which returns the platform in the docker inspect ... output
    • I believe this is always the path taken when updateRemoteUserUID gets the image details, since the CLI should have built the image locally right before this call.
    • The edits to inspectImageInRegistry ensure that when the image isn't in the local image cache, inspectDockerImage() fetches the image with the correct platform.

@trxcllnt
Copy link
Contributor Author

@chrmarti I just pushed a commit to also handle the variant component of platforms like linux/amd64/v3 and linux/arm64/v8.

I'm not sure how to get the variant of the host platform without exec'ing uname -m and parsing the output. That seems necessary to faithfully match the host arch in getImageIndexEntryForPlatform when --platform is omitted.

I also added tests that pass --platform as both build and run arguments. Users might want to do this so docker sets the multi-platform build arguments like TARGETARCH.

Copy link
Contributor

@chrmarti chrmarti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I'll ask for a second reviewer now.

Copy link
Member

@samruddhikhandale samruddhikhandale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@samruddhikhandale samruddhikhandale merged commit 8885d5a into devcontainers:main Feb 28, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants