devcontainer build fixups for macOS clients #874

bpkroth · 2024-10-16T20:58:11Z

Pull Request

Title

Fixups to the devcontainer build for macOS clients.

Description

Use a more portalable random number generator
Address some differences in docker.sock privileges.
Address differences in stat arguments.
Switch to wget mode for conda installation to workaround lack of arm64 apt repo.
When pulling base images to prime cache, use the appropriate architecture (for Windows we only support amd64 for now).
Remove :latest from cache-from args for podman compliance.
Address some differences in sed syntax.
Fixes DevContainer fails to start on VSCode + MacOS #873

Type of Change

🛠️ Bug fix

Testing

local MacBook testing
CI testing for Linux

Additional Notes

Doesn't currently do builds for arm64 platform in the pipeline.
Can work towards addressing that in the future.

motus · 2024-10-18T00:01:19Z

mmm, I am afraid it still does not work on Mac. Here's the full trace:

[17 ms] Dev Containers 0.388.0 in VS Code 1.94.2 (384ff7382de624fb94dbaf6da11977bba1ecd427).
[17 ms] Start: Resolving Remote
[24 ms] Setting up container for folder or workspace: /Users/sergiym/devel/MLOS/mlos.code-workspace
[24 ms] Context: desktop-linux
[25 ms] Start: Check Docker is running
[25 ms] Start: Run: docker version
[60 ms] Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:14:45 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

[60 ms] Server: Docker Desktop 4.34.3 (170107)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:41 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
[63 ms] Start: Run: docker volume ls -q
[86 ms] Start: Run: docker volume create vscode
[105 ms] Start: Run: docker ps -q -a --filter label=vsch.local.folder=/Users/sergiym/devel/MLOS --filter label=vsch.quality=stable
[127 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/sergiym/devel/MLOS --filter label=devcontainer.config_file=/Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json
[143 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/sergiym/devel/MLOS
[174 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/sergiym/devel/MLOS
[190 ms] Running Dev Containers CLI:   up --user-data-folder /Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-7f906bb4-53ac-4a04-bb62-7403d91cf5e31729209480694 --workspace-folder /Users/sergiym/devel/MLOS --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=/Users/sergiym/devel/MLOS --id-label devcontainer.config_file=/Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --include-configuration --include-merged-configuration
[293 ms] ProxyResolver#loadSystemCertificates count
[294 ms] ProxyResolver#loadSystemCertificates count filtered
[295 ms] Start: Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-7f906bb4-53ac-4a04-bb62-7403d91cf5e31729209480694 --workspace-folder /Users/sergiym/devel/MLOS --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=/Users/sergiym/devel/MLOS --id-label devcontainer.config_file=/Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --include-configuration --include-merged-configuration
[416 ms] @devcontainers/cli 0.71.0. Node.js v20.16.0. darwin 24.0.0 arm64.
[416 ms] Start: Run: docker buildx version
[594 ms] github.com/docker/buildx v0.16.2-desktop.1 081c21b9e461293ae243a1ff813a680a4f5f8fb9
[594 ms] 
[594 ms] Start: Run: docker -v
[605 ms] Start: Resolving Remote
[609 ms] Loading 29 extra certificates from /var/folders/kp/jd8fmmqd3clgld58441_g_v40000gn/T/vsch/certificates-4923716bdb9568154d418167e377b48d2637d62d665d3fbace955f17b92b86e0.pem.
Running the initializeCommand from devcontainer.json...

[666 ms] Start: Run: .devcontainer/scripts/prep-container-build 
+++ readlink -f .devcontainer/scripts/prep-container-build
++ dirname /Users/sergiym/devel/MLOS/.devcontainer/scripts/prep-container-build
+ scriptdir=/Users/sergiym/devel/MLOS/.devcontainer/scripts
+ cd /Users/sergiym/devel/MLOS/.devcontainer/scripts
+ cd ../../
+ '[' '!' -f .env ']'
+ '[' -e .devcontainer/.env ']'
+ egrep -q '^NGINX_PORT=[0-9]+$' .devcontainer/.env
+ '[' -d .devcontainer/tmp ']'
+ rm -rf .devcontainer/tmp
+ mkdir -p .devcontainer/tmp/
+ cp -v conda-envs/mlos.yml .devcontainer/tmp/mlos.yml
conda-envs/mlos.yml -> .devcontainer/tmp/mlos.yml
+ for pkg in mlos_core mlos_bench mlos_viz
+ mkdir -p .devcontainer/tmp/mlos_core/mlos_core
+ cp -v mlos_core/pyproject.toml .devcontainer/tmp/mlos_core/pyproject.toml
mlos_core/pyproject.toml -> .devcontainer/tmp/mlos_core/pyproject.toml
+ cp -v mlos_core/setup.py .devcontainer/tmp/mlos_core/setup.py
mlos_core/setup.py -> .devcontainer/tmp/mlos_core/setup.py
+ cp -v mlos_core/mlos_core/version.py .devcontainer/tmp/mlos_core/mlos_core/version.py
mlos_core/mlos_core/version.py -> .devcontainer/tmp/mlos_core/mlos_core/version.py
+ for pkg in mlos_core mlos_bench mlos_viz
+ mkdir -p .devcontainer/tmp/mlos_bench/mlos_bench
+ cp -v mlos_bench/pyproject.toml .devcontainer/tmp/mlos_bench/pyproject.toml
mlos_bench/pyproject.toml -> .devcontainer/tmp/mlos_bench/pyproject.toml
+ cp -v mlos_bench/setup.py .devcontainer/tmp/mlos_bench/setup.py
mlos_bench/setup.py -> .devcontainer/tmp/mlos_bench/setup.py
+ cp -v mlos_bench/mlos_bench/version.py .devcontainer/tmp/mlos_bench/mlos_bench/version.py
mlos_bench/mlos_bench/version.py -> .devcontainer/tmp/mlos_bench/mlos_bench/version.py
+ for pkg in mlos_core mlos_bench mlos_viz
+ mkdir -p .devcontainer/tmp/mlos_viz/mlos_viz
+ cp -v mlos_viz/pyproject.toml .devcontainer/tmp/mlos_viz/pyproject.toml
mlos_viz/pyproject.toml -> .devcontainer/tmp/mlos_viz/pyproject.toml
+ cp -v mlos_viz/setup.py .devcontainer/tmp/mlos_viz/setup.py
mlos_viz/setup.py -> .devcontainer/tmp/mlos_viz/setup.py
+ cp -v mlos_viz/mlos_viz/version.py .devcontainer/tmp/mlos_viz/mlos_viz/version.py
mlos_viz/mlos_viz/version.py -> .devcontainer/tmp/mlos_viz/mlos_viz/version.py
+ cp -v doc/requirements.txt .devcontainer/tmp/doc.requirements.txt
doc/requirements.txt -> .devcontainer/tmp/doc.requirements.txt
+ cp -v .devcontainer/scripts/common/prep-deps-files.sh .devcontainer/tmp/prep-deps-files.sh
.devcontainer/scripts/common/prep-deps-files.sh -> .devcontainer/tmp/prep-deps-files.sh
+ '[' '' '!=' true ']'
+ cacheFrom=mloscore.azurecr.io/mlos-devcontainer
+ echo 'Consider pulling image mloscore.azurecr.io/mlos-devcontainer for build caching.'
Consider pulling image mloscore.azurecr.io/mlos-devcontainer for build caching.

[742 ms] Start: Run: docker compose version --short
[890 ms] Docker Compose version: 2.29.2-desktop.2
[890 ms] Start: Run: docker compose -f /Users/sergiym/devel/MLOS/.devcontainer/docker-compose.yml --profile * config
[1006 ms] name: devcontainer
services:
  app:
    build:
      context: /Users/sergiym/devel/MLOS/.devcontainer/tmp
      dockerfile: ../Dockerfile
      args:
        BUILDKIT_INLINE_CACHE: "1"
        COMPOSE_DOCKER_CLI_BUILD: "1"
        DOCKER_BUILDKIT: "1"
        http_proxy: ""
        https_proxy: ""
        no_proxy: ""
      cache_from:
        - mloscore.azurecr.io/mlos-devcontainer:latest
        - mlos-devcontainer:latest
    command:
      - /bin/sh
      - -c
      - while true; do sleep 1000; done
    extra_hosts:
      - host.docker.internal=host-gateway
    networks:
      default: null
    volumes:
      - type: bind
        source: /Users/sergiym/devel
        target: /workspaces
        bind:
          create_host_path: true
  docs-www:
    build:
      context: /Users/sergiym/devel/MLOS/.devcontainer/tmp
      dockerfile: ../../doc/Dockerfile
      args:
        BUILDKIT_INLINE_CACHE: "1"
        COMPOSE_DOCKER_CLI_BUILD: "1"
        DOCKER_BUILDKIT: "1"
        NGINX_PORT: "3654"
        http_proxy: ""
        https_proxy: ""
        no_proxy: ""
    networks:
      default: null
    ports:
      - mode: ingress
        target: 3654
        protocol: tcp
    restart: unless-stopped
    volumes:
      - type: bind
        source: /Users/sergiym/devel/MLOS/doc
        target: /doc
        bind:
          create_host_path: true
      - type: bind
        source: /Users/sergiym/devel/MLOS/doc/nginx-default.conf
        target: /etc/nginx/templates/default.conf.template
        bind:
          create_host_path: true
networks:
  default:
    name: devcontainer_default
[1008 ms] Start: Run: docker ps -q -a --filter label=com.docker.compose.project=mlos_devcontainer --filter label=com.docker.compose.service=app
[1037 ms] Start: Run: docker events --format {{json .}} --filter event=start
[1037 ms] PersistedPath=/Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data, ContainerHasLabels=false
[1038 ms] Start: Run: docker compose -f /Users/sergiym/devel/MLOS/.devcontainer/docker-compose.yml --profile * config
[1120 ms] name: devcontainer
services:
  app:
    build:
      context: /Users/sergiym/devel/MLOS/.devcontainer/tmp
      dockerfile: ../Dockerfile
      args:
        BUILDKIT_INLINE_CACHE: "1"
        COMPOSE_DOCKER_CLI_BUILD: "1"
        DOCKER_BUILDKIT: "1"
        http_proxy: ""
        https_proxy: ""
        no_proxy: ""
      cache_from:
        - mloscore.azurecr.io/mlos-devcontainer:latest
        - mlos-devcontainer:latest
    command:
      - /bin/sh
      - -c
      - while true; do sleep 1000; done
    extra_hosts:
      - host.docker.internal=host-gateway
    networks:
      default: null
    volumes:
      - type: bind
        source: /Users/sergiym/devel
        target: /workspaces
        bind:
          create_host_path: true
  docs-www:
    build:
      context: /Users/sergiym/devel/MLOS/.devcontainer/tmp
      dockerfile: ../../doc/Dockerfile
      args:
        BUILDKIT_INLINE_CACHE: "1"
        COMPOSE_DOCKER_CLI_BUILD: "1"
        DOCKER_BUILDKIT: "1"
        NGINX_PORT: "3654"
        http_proxy: ""
        https_proxy: ""
        no_proxy: ""
    networks:
      default: null
    ports:
      - mode: ingress
        target: 3654
        protocol: tcp
    restart: unless-stopped
    volumes:
      - type: bind
        source: /Users/sergiym/devel/MLOS/doc
        target: /doc
        bind:
          create_host_path: true
      - type: bind
        source: /Users/sergiym/devel/MLOS/doc/nginx-default.conf
        target: /etc/nginx/templates/default.conf.template
        bind:
          create_host_path: true
networks:
  default:
    name: devcontainer_default
[1121 ms] Start: Run: docker inspect --type image mcr.microsoft.com/vscode/devcontainers/base
[1411 ms] Error fetching image details: No manifest found for mcr.microsoft.com/vscode/devcontainers/base.
[1412 ms] Start: Run: docker pull mcr.microsoft.com/vscode/devcontainers/base
Using default tag: latest
latest: Pulling from vscode/devcontainers/base
6d11c181ebb3: Already exists 
2b238499ec52: Already exists 
edb9092f6ed6: Pull complete 
4fab6b305bd9: Pull complete 
3f4bd65a2291: Pull complete 
e149adaf7d91: Pull complete 
d19027a11497: Pull complete 
9a2cc96877d4: Pull complete 
Digest: sha256:b3fd61ae07a1ebd211cac645c758f4018688a86e2c9b596a6c862ca89b0a668d
Status: Downloaded newer image for mcr.microsoft.com/vscode/devcontainers/base:latest
mcr.microsoft.com/vscode/devcontainers/base:latest

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview mcr.microsoft.com/vscode/devcontainers/base
[17606 ms] Start: Run: docker inspect --type image mcr.microsoft.com/vscode/devcontainers/base
[17626 ms] Resolving Feature dependencies for 'ghcr.io/devcontainers/features/azure-cli:1'...
[17627 ms] * Processing feature: ghcr.io/devcontainers/features/azure-cli:1
[17777 ms] Start: Run: docker-credential-desktop get
[18141 ms] * Processing feature: ghcr.io/devcontainers/features/common-utils
[18285 ms] Resolving Feature dependencies for 'ghcr.io/devcontainers/features/docker-outside-of-docker:1'...
[18285 ms] * Processing feature: ghcr.io/devcontainers/features/docker-outside-of-docker:1
[18438 ms] * Processing feature: ghcr.io/devcontainers/features/common-utils
[18576 ms] Resolving Feature dependencies for 'ghcr.io/devcontainers/features/github-cli:1'...
[18576 ms] * Processing feature: ghcr.io/devcontainers/features/github-cli:1
[18735 ms] * Processing feature: ghcr.io/devcontainers/features/common-utils
[18881 ms] * Processing feature: ghcr.io/devcontainers/features/git
[19021 ms] Resolving Feature dependencies for 'ghcr.io/stuartleeks/dev-container-features/azure-cli-persistence:0'...
[19021 ms] * Processing feature: ghcr.io/stuartleeks/dev-container-features/azure-cli-persistence:0
[19463 ms] Files to omit: ''
[19471 ms] * Processing feature: ghcr.io/devcontainers/features/azure-cli
[19629 ms] Resolving Feature dependencies for 'ghcr.io/stuartleeks/dev-container-features/shell-history:0'...
[19629 ms] * Processing feature: ghcr.io/stuartleeks/dev-container-features/shell-history:0
[20030 ms] Files to omit: ''
[20044 ms] * Processing feature: ghcr.io/devcontainers/features/common-utils
[20246 ms] * Processing feature: ghcr.io/meaningful-ooo/devcontainer-features/fish
[20415 ms] Soft-dependency 'ghcr.io/devcontainers/features/common-utils' is not required.  Removing from installation order...
[20415 ms] Soft-dependency 'ghcr.io/devcontainers/features/common-utils' is not required.  Removing from installation order...
[20415 ms] Soft-dependency 'ghcr.io/devcontainers/features/git' is not required.  Removing from installation order...
[20415 ms] Soft-dependency 'ghcr.io/devcontainers/features/common-utils' is not required.  Removing from installation order...
[20415 ms] Soft-dependency 'ghcr.io/meaningful-ooo/devcontainer-features/fish' is not required.  Removing from installation order...
[20415 ms] Soft-dependency 'ghcr.io/devcontainers/features/common-utils' is not required.  Removing from installation order...
[20415 ms] * Fetching feature: azure-cli_0_oci
[20585 ms] Files to omit: ''
[20593 ms] * Fetched feature: azure-cli_0_oci version 1.2.5
[20593 ms] * Fetching feature: docker-outside-of-docker_1_oci
[20808 ms] Files to omit: ''
[20815 ms] * Fetched feature: docker-outside-of-docker_1_oci version 1.6.0
[20815 ms] * Fetching feature: github-cli_2_oci
[21053 ms] Files to omit: ''
[21060 ms] * Fetched feature: github-cli_2_oci version 1.0.13
[21060 ms] * Fetching feature: shell-history_3_oci
[21232 ms] Files to omit: ''
[21236 ms] * Fetched feature: shell-history_3_oci version 0.0.3
[21236 ms] * Fetching feature: azure-cli-persistence_4_oci
[21406 ms] Files to omit: ''
[21409 ms] * Fetched feature: azure-cli-persistence_4_oci version 0.0.4
[21416 ms] Docker Compose override file for building image:
services:
  app:
    build:
      dockerfile: /var/folders/kp/jd8fmmqd3clgld58441_g_v40000gn/T/devcontainercli/container-features/0.71.0-1729209499734/Dockerfile-with-features
      args:
        - BUILDKIT_INLINE_CACHE=1
        - _DEV_CONTAINERS_BASE_IMAGE=conda
        - _DEV_CONTAINERS_IMAGE_USER=vscode:conda
        - _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp
      additional_contexts:
        - dev_containers_feature_content_source=/var/folders/kp/jd8fmmqd3clgld58441_g_v40000gn/T/devcontainercli/container-features/0.71.0-1729209499734

[21417 ms] Start: Run: docker compose --project-name mlos_devcontainer -f /Users/sergiym/devel/MLOS/.devcontainer/docker-compose.yml -f /Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/docker-compose/docker-compose.devcontainer.build-1729209503525.yml build
WARN[0000] The "http_proxy" variable is not set. Defaulting to a blank string. 
WARN[0000] The "https_proxy" variable is not set. Defaulting to a blank string. 
WARN[0000] The "no_proxy" variable is not set. Defaulting to a blank string. 
WARN[0000] The "http_proxy" variable is not set. Defaulting to a blank string. 
WARN[0000] The "https_proxy" variable is not set. Defaulting to a blank string. 
WARN[0000] The "no_proxy" variable is not set. Defaulting to a blank string. 
[+] Building 13.0s (28/38)                                 docker:desktop-linux
 => [app internal] load build definition from Dockerfile-with-features     0.0s
 => => transferring dockerfile: 12.62kB                                    0.0s
 => [docs-www internal] load build definition from Dockerfile              0.0s
 => => transferring dockerfile: 261B                                       0.0s
 => [docs-www internal] load metadata for docker.io/library/nginx:latest   0.7s
 => [app] resolve image config for docker-image://docker.io/docker/docker  0.7s
 => CACHED [app] docker-image://docker.io/docker/dockerfile:1.4@sha256:9b  0.0s
 => [docs-www internal] load .dockerignore                                 0.0s
 => => transferring context: 2B                                            0.0s
 => CACHED [docs-www 1/2] FROM docker.io/library/nginx:latest@sha256:2840  0.0s
 => [docs-www 2/2] RUN apt-get update && apt-get install -y --no-install-  3.2s
 => [app internal] load .dockerignore                                      0.0s
 => => transferring context: 2B                                            0.0s
 => [app context dev_containers_feature_content_source] load .dockerignor  0.0s
 => => transferring dev_containers_feature_content_source: 2B              0.0s
 => [app internal] load metadata for mcr.microsoft.com/vscode/devcontaine  0.0s
 => ERROR [app] importing cache manifest from mlos-devcontainer:latest     0.6s
 => [app] importing cache manifest from mloscore.azurecr.io/mlos-devconta  1.4s
 => => inferred cache manifest type: application/vnd.docker.distribution.  0.0s
 => [app context dev_containers_feature_content_source] load from client   0.0s
 => => transferring dev_containers_feature_content_source: 28.16kB         0.0s
 => [app internal] load build context                                      0.0s
 => => transferring context: 22.47kB                                       0.0s
 => [app deps-prep 1/4] FROM mcr.microsoft.com/vscode/devcontainers/base   0.0s
 => [app deps-prep 2/4] RUN apt-get update && export DEBIAN_FRONTEND=noni  4.7s
 => [app base 2/2] RUN apt-get update && export DEBIAN_FRONTEND=nonintera  7.4s
 => [docs-www] exporting to image                                          0.1s
 => => exporting layers                                                    0.1s
 => => preparing layers for inline cache                                   0.0s
 => => writing image sha256:2258217c331914e19a5a63a5d1f40dc9ae6047bfe3ff6  0.0s
 => => naming to docker.io/library/mlos_devcontainer-docs-www              0.0s
 => [docs-www] resolving provenance for metadata file                      0.0s
 => [app deps-prep 3/4] COPY --chown=vscode . /tmp/conda-tmp/              0.0s
 => [app deps-prep 4/4] RUN /tmp/conda-tmp/prep-deps-files.sh     && ls -  0.6s
 => [app conda 1/6] RUN addgroup conda     && adduser vscode conda         0.2s
 => [app conda 2/6] RUN mkdir -p /var/cache/pip     && chown -R vscode:co  0.1s
 => [app conda 3/6] RUN mkdir -p /opt/conda/pkgs     && chown -R vscode:c  0.1s
 => [app conda 4/6] RUN mkdir -p /home/vscode/.ssh     && (         grep   1.0s
 => [app conda 5/6] COPY --from=deps-prep --chown=vscode:conda /tmp/conda  0.0s
 => ERROR [app conda 6/6] RUN echo "Setup miniconda"     && curl -Ss http  1.8s
------
 > [app] importing cache manifest from mlos-devcontainer:latest:
------
------
 > [app conda 6/6] RUN echo "Setup miniconda"     && curl -Ss https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/conda.gpg > /dev/null     && gpg --keyring /etc/apt/trusted.gpg.d/conda.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806     && echo "deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/conda.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" | sudo tee /etc/apt/sources.list.d/conda.list     && sudo apt-get update     && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends conda     && sudo apt-get clean && sudo rm -rf /var/lib/apt/lists/*     && echo "# Adjust the conda installation to be user/group writable."     && sudo /opt/conda/bin/conda init --system     && sudo chgrp -R conda /opt/conda     && sudo chmod -R g+wX /opt/conda     && find /opt/conda -type d -print0 | xargs -0 sudo chmod -c g+s     && umask 0002     && echo "# Use conda-forge first to get the latest versions of packages "     && echo "# and reduce duplication with mlos env (which also uses conda-forge first)."     && echo "# Upgrade conda and use strict priorities"     && echo "# Use the mamba solver (necessary for some quality of life speedups due to required packages to support Windows)"     && /opt/conda/bin/conda init     && /opt/conda/bin/conda config --set channel_priority strict     && /opt/conda/bin/conda info     && /opt/conda/bin/conda update -v -y -n base -c conda-forge -c defaults --all     && /opt/conda/bin/conda list -n base     && /opt/conda/bin/conda install -v -y -n base -c conda-forge -c defaults conda-libmamba-solver     && /opt/conda/bin/conda config --system --set solver libmamba     && echo "# Install some additional editor packages for the base environment."     && /opt/conda/bin/conda run -n base pip install --no-cache-dir -U pynvim     && echo "# Clean up conda cache to save some space."     && /opt/conda/bin/conda list -n base     && /opt/conda/bin/conda clean -v -y -a     && /opt/conda/bin/conda run -n base pip cache purge     && echo "# Install some additional dependencies for the mlos environment."     && echo "# Make sure they have conda group ownership to make the devcontainer more"     && echo "# reliable useable across vscode uid changes."     && sg conda -c "/opt/conda/bin/conda env create -n mlos -v -f /tmp/conda-tmp/mlos_deps.yml"     && sg conda -c "/opt/conda/bin/conda run -n mlos pip install --no-cache-dir -U -r /tmp/conda-tmp/combined.requirements.txt"     && sg conda -c "/opt/conda/bin/conda run -n mlos pip cache purge"     && sg conda -c "/opt/conda/bin/conda clean -v -y -a"     && mkdir -p /opt/conda/pkgs/cache/ && chown -R vscode:conda /opt/conda/pkgs/cache/     && mkdir -p /home/vscode/.conda/envs     && ln -s /opt/conda/envs/mlos /home/vscode/.conda/envs/mlos:
0.115 Setup miniconda
0.198 gpg: directory '/home/vscode/.gnupg' created
0.198 gpg: /home/vscode/.gnupg/trustdb.gpg: trustdb created
0.199 pub   rsa2048 2018-11-02 [SC]
0.199       3416 1F5B F5EB 1D4B FBBB  8F0A 8AEB 4F8B 29D8 2806
0.199 uid           [ unknown] Distribution Team <[email protected]>
0.199 sub   rsa2048 2018-11-02 [E]
0.199 
0.201 deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/conda.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main
0.256 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
0.301 Get:2 https://repo.anaconda.com/pkgs/misc/debrepo/conda stable InRelease [3961 B]
0.305 Get:3 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
0.324 Get:4 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
0.345 Get:5 http://deb.debian.org/debian bookworm/main arm64 Packages [8689 kB]
0.376 Get:6 https://repo.anaconda.com/pkgs/misc/debrepo/conda stable/main amd64 Packages [4620 B]
0.583 Get:7 http://deb.debian.org/debian bookworm-updates/main arm64 Packages [2468 B]
0.583 Get:8 http://deb.debian.org/debian-security bookworm-security/main arm64 Packages [186 kB]
1.203 Fetched 9140 kB in 1s (9242 kB/s)
1.203 Reading package lists...
1.444 Reading package lists...
1.675 Building dependency tree...
1.751 Reading state information...
1.754 E: Unable to locate package conda
------
failed to solve: process "/bin/sh -c echo \"Setup miniconda\"     && curl -Ss https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/conda.gpg > /dev/null     && gpg --keyring /etc/apt/trusted.gpg.d/conda.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806     && echo \"deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/conda.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main\" | sudo tee /etc/apt/sources.list.d/conda.list     && sudo apt-get update     && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends conda     && sudo apt-get clean && sudo rm -rf /var/lib/apt/lists/*     && echo \"# Adjust the conda installation to be user/group writable.\"     && sudo /opt/conda/bin/conda init --system     && sudo chgrp -R conda /opt/conda     && sudo chmod -R g+wX /opt/conda     && find /opt/conda -type d -print0 | xargs -0 sudo chmod -c g+s     && umask 0002     && echo \"# Use conda-forge first to get the latest versions of packages \"     && echo \"# and reduce duplication with mlos env (which also uses conda-forge first).\"     && echo \"# Upgrade conda and use strict priorities\"     && echo \"# Use the mamba solver (necessary for some quality of life speedups due to required packages to support Windows)\"     && /opt/conda/bin/conda init     && /opt/conda/bin/conda config --set channel_priority strict     && /opt/conda/bin/conda info     && /opt/conda/bin/conda update -v -y -n base -c conda-forge -c defaults --all     && /opt/conda/bin/conda list -n base     && /opt/conda/bin/conda install -v -y -n base -c conda-forge -c defaults conda-libmamba-solver     && /opt/conda/bin/conda config --system --set solver libmamba     && echo \"# Install some additional editor packages for the base environment.\"     && /opt/conda/bin/conda run -n base pip install --no-cache-dir -U pynvim     && echo \"# Clean up conda cache to save some space.\"     && /opt/conda/bin/conda list -n base     && /opt/conda/bin/conda clean -v -y -a     && /opt/conda/bin/conda run -n base pip cache purge     && echo \"# Install some additional dependencies for the mlos environment.\"     && echo \"# Make sure they have conda group ownership to make the devcontainer more\"     && echo \"# reliable useable across vscode uid changes.\"     && sg conda -c \"/opt/conda/bin/conda env create -n mlos -v -f /tmp/conda-tmp/mlos_deps.yml\"     && sg conda -c \"/opt/conda/bin/conda run -n mlos pip install --no-cache-dir -U -r /tmp/conda-tmp/combined.requirements.txt\"     && sg conda -c \"/opt/conda/bin/conda run -n mlos pip cache purge\"     && sg conda -c \"/opt/conda/bin/conda clean -v -y -a\"     && mkdir -p /opt/conda/pkgs/cache/ && chown -R vscode:conda /opt/conda/pkgs/cache/     && mkdir -p /home/vscode/.conda/envs     && ln -s /opt/conda/envs/mlos /home/vscode/.conda/envs/mlos" did not complete successfully: exit code: 100
[34820 ms] Error: Command failed: docker compose --project-name mlos_devcontainer -f /Users/sergiym/devel/MLOS/.devcontainer/docker-compose.yml -f /Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/docker-compose/docker-compose.devcontainer.build-1729209503525.yml build
[34820 ms]     at Pm (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:430:525)
[34820 ms]     at async ftA (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:430:2476)
[34820 ms]     at async htA (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:410:3496)
[34820 ms]     at async TtA (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:482:4021)
[34820 ms]     at async iB (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:482:4963)
[34820 ms]     at async wrA (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:663:203)
[34820 ms]     at async DrA (/Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:662:14830)
[34820 ms]     at async /Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js:482:1190
[34827 ms] Exit code 1
[34833 ms] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Users/sergiym/.vscode/extensions/ms-vscode-remote.remote-containers-0.388.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/sergiym/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-7f906bb4-53ac-4a04-bb62-7403d91cf5e31729209480694 --workspace-folder /Users/sergiym/devel/MLOS --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=/Users/sergiym/devel/MLOS --id-label devcontainer.config_file=/Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/sergiym/devel/MLOS/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --include-configuration --include-merged-configuration
[34833 ms] Exit code 1

This reverts commit 2d88dfb.

MacOS sets the /var/run/docker.sock as a symlink to a spot in the user's home directory. The group id on the symlink is 1 (daemon) but the group id of /var/run/docker.sock inside the actual docker VM on MacOS is 0 (root).

…e isn't present in this version of sh

…alling conda

motus

It works on my Mac now! Will check it on Windows soon and post here

.devcontainer/build/build-devcontainer-cli.sh

bpkroth · 2024-10-21T15:17:32Z

Note: this enables the devcontainer for MacOS, but the tests don't pass. Will address this in a future PR (e.g., #875).

bpkroth added 9 commits October 16, 2024 15:55

Use a more portalable random number generator

a3dab39

Enable buildkit for mac os as well

7aaaf73

stat args differences

10075b6

podman vs. docker differences?

134e1be

fixup

5b6c8bd

more cross platform fixups

bbdcfa2

add devcontainer extension to recommendations

6f9f3bf

tweaks

20c73e6

fixup

0a5f6c4

bpkroth added 15 commits October 18, 2024 14:47

wip: fix docker socket perms

2d88dfb

bsd ism

5fe9927

Revert "wip: fix docker socket perms"

1397ee1

This reverts commit 2d88dfb.

Adjustments for the permissions to the docker.sock on MacOS

7911961

MacOS sets the /var/run/docker.sock as a symlink to a spot in the user's home directory. The group id on the symlink is 1 (daemon) but the group id of /var/run/docker.sock inside the actual docker VM on MacOS is 0 (root).

initialize RANDOM with a pseudo random pid in case the RANDOM variabl…

628fb04

…e isn't present in this version of sh

Allow more archs

81c8aee

licenseheaders

0ecab8e

specify the platform

d0cb8a4

tweaks

8d518c4

fixups

f5b3ee2

Use the wget method instead of the apt method of downloading and inst…

6a59eb7

…alling conda

fixu

051d67c

comments

8eff234

comments

e18e3bb

revert

50ed094

bpkroth marked this pull request as ready for review October 18, 2024 21:40

bpkroth requested a review from a team as a code owner October 18, 2024 21:40

motus approved these changes Oct 18, 2024

View reviewed changes

bpkroth commented Oct 21, 2024

View reviewed changes

.devcontainer/build/build-devcontainer-cli.sh Show resolved Hide resolved

bpkroth mentioned this pull request Oct 21, 2024

Add MacOS testing and arm64 devcontainer build to CI pipelines #875

Closed

bpkroth merged commit 72e10d1 into microsoft:main Oct 21, 2024
12 of 13 checks passed

bpkroth deleted the fixup-dev-container-for-mac-os branch October 21, 2024 15:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

devcontainer build fixups for macOS clients #874

devcontainer build fixups for macOS clients #874

bpkroth commented Oct 16, 2024 •

edited

Loading

motus commented Oct 18, 2024

motus left a comment

bpkroth commented Oct 21, 2024

devcontainer build fixups for macOS clients #874

devcontainer build fixups for macOS clients #874

Conversation

bpkroth commented Oct 16, 2024 • edited Loading

Pull Request

Title

Description

Type of Change

Testing

Additional Notes

motus commented Oct 18, 2024

motus left a comment

Choose a reason for hiding this comment

bpkroth commented Oct 21, 2024

bpkroth commented Oct 16, 2024 •

edited

Loading