Skip to content

[Packaging] Add Ubuntu 24.04 Noble Numbat support #28888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bebound merged 3 commits into from
May 13, 2024
Merged

[Packaging] Add Ubuntu 24.04 Noble Numbat support #28888

bebound merged 3 commits into from
May 13, 2024

Conversation

@bebound
Copy link
Contributor

@bebound bebound commented May 6, 2024 •
edited
Loading

Ubuntu 24.04 was released on April 25, 2024
Resolve #28872

We'll drop 18.04 in #28942

Ref: https://wiki.ubuntu.com/Releases

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented May 6, 2024 •
edited
Loading

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented May 6, 2024

Hi @bebound,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented May 6, 2024 •
edited
Loading

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented May 6, 2024

Packaging

@microsoft-github-policy-service microsoft-github-policy-service bot added the Auto-Assign Auto assign by bot label May 6, 2024
@bebound bebound marked this pull request as ready for review May 7, 2024 09:04
@bebound
Copy link
Contributor Author

bebound commented May 8, 2024 •
edited
Loading

Noble package is half the size of Jammy:

-rwxrwxrwx 1 root root 54M May  8 11:22 azure-cli_2.60.0-1~jammy_amd64.deb
-rwxrwxrwx 1 root root 27M May  8 10:59 azure-cli_2.60.0-1~noble_amd64.deb

Installed-Size: 697 MB vs Installed-Size: 359 MB

__pycache__ folders are missing in Noble, for example, /opt/az/lib/python3.11/site-packages/azure/keyvault/keys/__pycache__

@bebound
Copy link
Contributor Author

bebound commented May 8, 2024 •
edited
Loading

Noble uses debhelper 13.14, while Jammy uses 13.6.
The dh_clean step is slightly different while running dpkg-buildpackage.
-name __pycache__ is added in the find command and those folders are removed from final package.

This would significantly slow the az command from 2s to 11s for non-root user, as __pycache__ folder can't be saved in /opt/az folder

dh_clean: warning: Compatibility levels before 10 are deprecated (level 9 in use)
 	rm -fr -- debian/azure-cli/ debian/tmp/
 	find .  \( \( \
 		\( -path .\*/.git -o -path .\*/.svn -o -path .\*/.bzr -o -path .\*/.hg -o -path .\*/CVS -o -path .\*/.pc -o -path .\*/_darcs \) -prune -o -type f -a \
 	        \( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
 		 -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
 		 -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
 		 -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
 		\) -exec rm -f {} + \) -o \
 		\( -type d -a \( -name autom4te.cache -o -name __pycache__ \) -prune -exec rm -rf {} + \) \)

This change was introduced in debhelper 13.10.10.
Ref:
https://salsa.debian.org/debian/debhelper/-/blame/main/debian/changelog#L164
https://salsa.debian.org/debian/debhelper/-/merge_requests/120
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1048890

@bebound bebound requested review from calvinhzy and kairu-ms as code owners May 9, 2024 03:45
bebound
bebound commented May 9, 2024
View reviewed changes
scripts/release/debian/prepare.sh
${TAB}dh \$@ --sourcedirectory $source_dir
override_dh_clean:
${TAB}dh_clean --exclude=__pycache__
Copy link
Contributor Author

@bebound bebound May 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To ensure the pyc files are included in package, as they are excluded by default starting with debhelper version 13.10.10.
See #28888 (comment)

Ref: https://man7.org/linux/man-pages/man1/dh_clean.1.html

Copy link
Member

@jiasli jiasli May 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a more official doc: https://manpages.ubuntu.com/manpages/noble/en/man1/dh_clean.1.html

@bebound
Copy link
Contributor Author

bebound commented May 9, 2024

I've created a bug report for debhelper: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1070794

@bebound
Copy link
Contributor Author

bebound commented May 10, 2024 •
edited
Loading

As suggest by Niels from Debian bug report, we should move the byte-compile step in the post install script, which could help us to reduce the package size.

Debian Python Policy also suggests this:

If a binary package provides any binary-independent modules (foo.py files), the corresponding byte-compiled modules (foo.pyc files) and optimized modules (foo.pyo files) must not ship in the package. Instead, they should be generated in the package’s post-install script, and removed in the package’s pre-remove script. -- https://www.debian.org/doc/packaging-manuals/python-policy/#modules-byte-compilation

This is beyond the scope of this PR. I've simply omitted the __pycache__ file from dh_clean during packaging.

This was referenced May 13, 2024
Closed
Merged
@jiasli
Copy link
Member

jiasli commented May 13, 2024

Fascinating in-depth investigation!

@bebound bebound merged commit 833da42 into Azure:dev May 13, 2024
@bebound bebound deleted the support-24.04 branch May 13, 2024 06:33
@bebound bebound mentioned this pull request May 13, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jiasli jiasli jiasli approved these changes

@wangzelin007 wangzelin007 Awaiting requested review from wangzelin007 wangzelin007 is a code owner

@yonzhan yonzhan Awaiting requested review from yonzhan

@kairu-ms kairu-ms Awaiting requested review from kairu-ms kairu-ms is a code owner

@calvinhzy calvinhzy Awaiting requested review from calvinhzy calvinhzy is a code owner

Assignees

@bebound bebound

Labels

Auto-Assign Auto assign by bot Packaging

Projects

None yet

Milestone

June 2024 (2024-07-09)

Development

Successfully merging this pull request may close these issues.

Unable to install for Ubuntu 24.04

3 participants

@bebound @yonzhan @jiasli