Skip to content

Conversation

@bebound
Copy link
Contributor

@bebound bebound commented Aug 21, 2023

Description

Create Azure Linux based image

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature


This checklist is used to make sure that common guidelines for a pull request are followed.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Aug 21, 2023

🔄AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
🔄containerapp
🔄latest
🔄3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Aug 21, 2023

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Aug 21, 2023

Packaging

Dockerfile.azure Outdated


# Azure Linux does not contain Mozilla CA certificates, install ca-certificates package to prevent CERTIFICATE_VERIFY_FAILED errors, see https://github.com/Azure/azure-cli/issues/26026
RUN --mount=type=bind,target=/azure-cli.rpm,source=./docker/azure-cli.rpm pwd && ls && tdnf install ca-certificates /azure-cli.rpm -y && tdnf clean all
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use mount to get rid of copy and save image size.
Mounted file does not take up space in docker.

@bebound bebound marked this pull request as ready for review August 21, 2023 03:29
.dockerignore Outdated
# Make an exception for the license file
!LICENSE.txt
# Make an exception for docker related files
!docker/*
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this to mount the rpm file, for

# Exclude everything by default
*

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is confusing. We can consider either renaming it or adding comments to clarify its content.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to use dockerfile as file extensions. See #21680

# Azure Linux does not contain Mozilla CA certificates, install ca-certificates package to prevent CERTIFICATE_VERIFY_FAILED errors, see https://github.com/Azure/azure-cli/issues/26026
RUN --mount=type=bind,target=/azure-cli.rpm,source=./docker/azure-cli.rpm tdnf install ca-certificates /azure-cli.rpm -y && tdnf clean all

ENV AZ_INSTALLER=DOCKER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we distinguish which docker image is used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the values are PIP, MSI, RPM, HOMEBREW, DEB and DOCKER, which do not contains a specific OS version. So I use the same value as Alpine.

Dockerfile.azure Outdated


# Azure Linux does not contain Mozilla CA certificates, install ca-certificates package to prevent CERTIFICATE_VERIFY_FAILED errors, see https://github.com/Azure/azure-cli/issues/26026
RUN --mount=type=bind,target=/azure-cli.rpm,source=./docker/azure-cli.rpm tdnf install ca-certificates /azure-cli.rpm -y && tdnf clean all
Copy link
Member

@jiasli jiasli Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if it will be good to follow the same installation method of Alpine-based docker image:

RUN ./scripts/install_full.sh && python ./scripts/trim_sdk.py \

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't create package for Alpine, there is no other choice except installing from pypi.

Installing from rpm is faster than install from pypi. Although it depends on build rpm, the overall time is not increased as it runs faster than test rpm package.

docker run $IMAGE_NAME /bin/bash -c "time az self-test && time az --version && sleep 5"
displayName: 'Bash Script'
- job: BuildExtraDockerImage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Extra" is too general. It's a good practice to be specific.

Suggested change
- job: BuildExtraDockerImage
- job: BuildAzureLinuxDockerImage

- job: BuildExtraDockerImage
displayName: Build Extra Docker Image
dependsOn: BuildRpmPackageMariner
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job should be renamed:

Suggested change
dependsOn: BuildRpmPackageMariner
dependsOn: BuildRpmPackageAzureLinux

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mariner is still used in pipeline. Let's wait for Azure Linux team change the url of docker image.

Copy link

@mandeepsplaha mandeepsplaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

${{ each arch in parameters.architectures }}:
Mariner 2.0 ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-mariner2.0-${{ arch.value }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The artifact name is docker-mariner2.0-amd64 (align with the RPM artifact rpm-mariner2.0-amd64), the docker image tag is cbl-mariner2.0.

This is the reply from Mariner team:

To keep things consistent with other images in MCR, can we make the tag "cbl-mariner2.0"? Once we release Mariner 3.0 next year, there will be a larger rebranding where we will be changing all container image tags to use "Azure Linux". However, for now we want to stay consistent with what already exists

Copy link
Member

@jiasli jiasli Sep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using the full name azure-linux. azure can easily cause confusion as it represents the Azure cloud.

Alpine ${{ arch.name }}:
pool: ${{ arch.pool }}
artifactName: docker-${{ arch.value }}
dockerfile: Dockerfile
Copy link
Member

@jiasli jiasli Sep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe renaming the Dockerfile to make it more specific, such as alpine.dockerfile, given we already have another base - Azure Linux?


dependsOn: BuildDockerImage
dependsOn:
- BuildDockerImage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Consider renaming it to BuildDockerImageAlpine.

docker run $IMAGE_NAME /bin/bash -c "time az self-test && time az --version && sleep 5"
displayName: 'Bash Script'
- job: BuildAzureDockerImage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe BuildDockerImageAzureLinux?

DockerImagesToScan: 'clibuild$BUILD_BUILDNUMBER:latest'

- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: docker image'
Copy link
Member

@jiasli jiasli Sep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This displayName adds little value to the job. We may remove displayName everywhere if it is meaningless.

azure.dockerfile Outdated
org.label-schema.name="Azure CLI" \
org.label-schema.version=$CLI_VERSION \
org.label-schema.license="MIT" \
org.label-schema.description="The Azure CLI is used for all Resource Manager deployments in Azure." \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description added 6 years ago by #4696 is no longer accurate, given we now support Microsoft Graph and other data-plane operations as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a new description in the repo A great cloud needs great tools; we're excited to introduce Azure CLI, our next generation multi-platform command line experience for Azure.. Do we have any alternative for this?



# Azure Linux base image does not contain Mozilla CA certificates, install ca-certificates package to prevent CERTIFICATE_VERIFY_FAILED errors, see https://github.com/Azure/azure-cli/issues/26026
RUN --mount=type=bind,target=/azure-cli.rpm,source=./docker-temp/azure-cli.rpm tdnf install ca-certificates /azure-cli.rpm -y && tdnf clean all && rm -rf /var/cache/tdnf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering why alpine.dockerfile uses COPY . /azure-cli instead of --mount=type=bind.

Copy link
Contributor Author

@bebound bebound Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--mount was added in dockerfile:1.2 on 2020-12-03.

@bebound bebound merged commit c59fef9 into Azure:dev Oct 17, 2023
@bebound bebound deleted the azure-dockerfile branch October 17, 2023 02:09
@Jordanwaslistening Jordanwaslistening mentioned this pull request Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot Packaging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants