Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
!az.completion
# Make an exception for the license file
!LICENSE.txt
# Make an exception for docker related files
!docker/*

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this to mount the rpm file, for

# Exclude everything by default
*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is confusing. We can consider either renaming it or adding comments to clarify its content.

# Exclude build droppings, as mentioned in .gitignore
src/build*
# Exclude tests
Expand Down
31 changes: 31 additions & 0 deletions Dockerfile.azure

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to use dockerfile as file extensions. See #21680

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#---------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
#---------------------------------------------------------------------------------------------

FROM mcr.microsoft.com/cbl-mariner/base/core:2.0

ARG CLI_VERSION

# Metadata as defined at http://label-schema.org
ARG BUILD_DATE

LABEL maintainer="Microsoft" \
org.label-schema.schema-version="1.0" \
org.label-schema.vendor="Microsoft" \
org.label-schema.name="Azure CLI" \
org.label-schema.version=$CLI_VERSION \
org.label-schema.license="MIT" \
org.label-schema.description="The Azure CLI is used for all Resource Manager deployments in Azure." \
org.label-schema.url="https://docs.microsoft.com/cli/azure/overview" \
org.label-schema.usage="https://learn.microsoft.com/en-us/cli/azure/run-azure-cli-docker" \
org.label-schema.build-date=$BUILD_DATE \
org.label-schema.vcs-url="https://github.com/Azure/azure-cli.git" \
org.label-schema.docker.cmd="docker run -v \${HOME}/.azure:/root/.azure -it mcr.microsoft.com/azure-cli:$CLI_VERSION-azure"


# Azure Linux does not contain Mozilla CA certificates, install ca-certificates package to prevent CERTIFICATE_VERIFY_FAILED errors, see https://github.com/Azure/azure-cli/issues/26026
RUN --mount=type=bind,target=/azure-cli.rpm,source=./docker/azure-cli.rpm pwd && ls && tdnf install ca-certificates /azure-cli.rpm -y && tdnf clean all

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use mount to get rid of copy and save image size.
Mounted file does not take up space in docker.


ENV AZ_INSTALLER=DOCKER

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we distinguish which docker image is used?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the values are PIP, MSI, RPM, HOMEBREW, DEB and DOCKER, which do not contains a specific OS version. So I use the same value as Alpine.

CMD bash
113 changes: 95 additions & 18 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -301,17 +301,13 @@ jobs:

- job: BuildDockerImage
displayName: Build Docker Image

dependsOn: ExtractMetadata
condition: succeeded()
strategy:
matrix:
AMD64:
pool: ${{ variables.ubuntu_pool }}
artifactName: docker-amd64
ARM64:
pool: ${{ variables.ubuntu_arm64_pool }}
artifactName: docker-arm64
${{ each arch in parameters.architectures }}:
Alpine ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-${{ arch.name }}
dockerfile: Dockerfile

@jiasli jiasli Sep 18, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe renaming the Dockerfile to make it more specific, such as alpine.dockerfile, given we already have another base - Azure Linux?

pool:
name: $(pool)
steps:
Expand All @@ -337,17 +333,16 @@ jobs:

- job: TestDockerImage
displayName: Test Docker Image

dependsOn: BuildDockerImage
dependsOn:
- BuildDockerImage

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Consider renaming it to BuildDockerImageAlpine.

- ExtractMetadata
condition: succeeded()
strategy:
matrix:
AMD64:
pool: ${{ variables.ubuntu_pool }}
artifactName: docker-amd64
ARM64:
pool: ${{ variables.ubuntu_arm64_pool }}
artifactName: docker-arm64
${{ each arch in parameters.architectures }}:
Alpine ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-${{ arch.name }}
pool:
name: $(pool)
steps:
Expand All @@ -357,7 +352,6 @@ jobs:
TargetPath: '$(Build.ArtifactStagingDirectory)/metadata'
artifactName: metadata


- task: DownloadPipelineArtifact@1
displayName: 'Download Docker Image'
inputs:
Expand All @@ -380,6 +374,89 @@ jobs:
docker run $IMAGE_NAME /bin/bash -c "time az self-test && time az --version && sleep 5"
displayName: 'Bash Script'

- job: BuildExtraDockerImage

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Extra" is too general. It's a good practice to be specific.

Suggested change
- job: BuildExtraDockerImage
- job: BuildAzureLinuxDockerImage

displayName: Build Extra Docker Image
dependsOn: BuildRpmPackageMariner

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job should be renamed:

Suggested change
dependsOn: BuildRpmPackageMariner
dependsOn: BuildRpmPackageAzureLinux

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mariner is still used in pipeline. Let's wait for Azure Linux team change the url of docker image.

strategy:
matrix:
${{ each arch in parameters.architectures }}:
Azure Linux ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-azure-${{ arch.value }}
dockerfile: Dockerfile.azure
packageArtifactName: rpm-mariner2.0-${{ arch.value }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar question here about mariner version: does it make sense to make the version a variable to allow for mariner 3.0 down the road?

pool:
name: $(pool)
steps:
- bash: ./scripts/ci/install_docker.sh
displayName: Install Docker
- task: DownloadPipelineArtifact@1
displayName: 'Download Build Artifacts'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/docker'
artifactName: $(packageArtifactName)
- bash: |
set -ex
mkdir docker
mv $(Build.ArtifactStagingDirectory)/docker/*.rpm ./docker/azure-cli.rpm

bash scripts/release/docker/pipeline.sh
displayName: 'Build Docker'

- task: AzureArtifacts.manifest-generator-task.manifest-generator-task.ManifestGeneratorTask@0
displayName: 'SBOM'
inputs:
BuildDropPath: $(Build.ArtifactStagingDirectory)
DockerImagesToScan: 'clibuild$BUILD_BUILDNUMBER:latest'

- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: docker image'

@jiasli jiasli Sep 18, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This displayName adds little value to the job. We may remove displayName everywhere if it is meaningless.

inputs:
TargetPath: $(Build.ArtifactStagingDirectory)
ArtifactName: $(artifactName)

- job: TestExtraDockerImage
displayName: Test Extra Docker Image
dependsOn:
- BuildExtraDockerImage
- ExtractMetadata
strategy:
matrix:
${{ each arch in parameters.architectures }}:
Azure Linux ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-azure-${{ arch.value }}
pool:
name: $(pool)
steps:
- task: DownloadPipelineArtifact@1
displayName: 'Download Metadata'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/metadata'
artifactName: metadata

- task: DownloadPipelineArtifact@1
displayName: 'Download Docker Image'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/docker'
artifactName: $(artifactName)

- bash: ./scripts/ci/install_docker.sh
displayName: Install Docker

- bash: |
set -exv

CLI_VERSION=`cat $SYSTEM_ARTIFACTSDIRECTORY/metadata/version`
IMAGE_NAME=clibuild$BUILD_BUILDNUMBER:latest
TAR_FILE=$SYSTEM_ARTIFACTSDIRECTORY/docker/docker-azure-cli-$CLI_VERSION.tar

echo "== Test docker image =="

docker load < $TAR_FILE
docker run $IMAGE_NAME /bin/bash -c "time az self-test && time az --version && sleep 5"
displayName: 'Bash Script'

- job: BuildPythonWheel
displayName: Build Python Wheels

Expand Down
1 change: 1 addition & 0 deletions scripts/release/docker/pipeline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ docker build --no-cache \
--build-arg BUILD_DATE="`date -u +"%Y-%m-%dT%H:%M:%SZ"`" \
--build-arg CLI_VERSION=$CLI_VERSION \
--tag $IMAGE_NAME:latest \
--file $DOCKERFILE \
$BUILD_SOURCESDIRECTORY

docker save -o "$BUILD_STAGINGDIRECTORY/docker-azure-cli-${CLI_VERSION}.tar" $IMAGE_NAME:latest