Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

Merged
merged 5 commits into from
Jun 21, 2022
Merged

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d6cc8e6
Revert "Revert "Reduce image size for lazy installation packages (#10…
xumia May 26, 2022
96c8fba
Use the platform.tar.gz as a replacement for platform folder
xumia May 26, 2022
677c5bf
Reduce the requird disk size to unzip platform.tar.gz
xumia Jun 8, 2022
b6039a7
Fix a type issue
xumia Jun 8, 2022
dff8acb
Change to extract tar files in memory
xumia Jun 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build_debian.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -647,5 +647,5 @@ fi
pushd $FILESYSTEM_ROOT && sudo tar czf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd

## Compress together with /boot, /var/lib/docker and $PLATFORM_DIR as an installer payload zip file
pushd $FILESYSTEM_ROOT && sudo zip $OLDPWD/$ONIE_INSTALLER_PAYLOAD -r boot/ $PLATFORM_DIR/; popd
pushd $FILESYSTEM_ROOT && sudo tar czf platform.tar.gz -C $PLATFORM_DIR . && sudo zip -n .gz $OLDPWD/$ONIE_INSTALLER_PAYLOAD -r boot/ platform.tar.gz; popd
sudo zip -g -n .squashfs:.gz $ONIE_INSTALLER_PAYLOAD $FILESYSTEM_SQUASHFS $FILESYSTEM_DOCKERFS
7 changes: 6 additions & 1 deletion files/Aboot/boot0.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,12 @@ extract_image() {

info "Extracting swi content"
## Unzip the image except boot0 and dockerfs archive
unzip -oq "$swipath" -x boot0 "$dockerfs" -d "$image_path"
unzip -oq "$swipath" -x boot0 "$dockerfs" "platform.tar.gz" -d "$image_path"

## Extract the platform.tar.gz
info "Extracting platform.tar.gz"
mkdir -p "$image_path/platform"
unzip -oqp "$swipath" "platform.tar.gz" | tar xzf - -C "$image_path/platform" $TAR_EXTRA_OPTION

## detect rootfs type
local mountstr="$(grep " $target_path " /proc/mounts)"
Expand Down
4 changes: 3 additions & 1 deletion files/build_templates/sonic_debian_extension.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -614,7 +614,9 @@ sudo LANG=C chroot $FILESYSTEM_ROOT depmod -a {{kversion}}
sudo dpkg --root=$FILESYSTEM_ROOT -i {{deb}} || sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y install -f --download-only

sudo mkdir -p $FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}
sudo cp {{ deb }} $FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/
sudo mkdir -p $FILESYSTEM_ROOT/$PLATFORM_DIR/common
sudo cp {{ deb }} $FILESYSTEM_ROOT/$PLATFORM_DIR/common/
sudo ln -sf "../common/{{ debfilename }}" "$FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/{{ debfilename }}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we do the same thing (symbol link approach for deb in /var/cache/apt/archives?

Copy link
Collaborator Author

@xumia xumia May 31, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is to copy the packages to installer folder. The packages will be installed by the script (install.sh) by each of the boot loader. In the platform folder, it contains some sub folders, such as x86_64-8101_32fh_o-r0, x86_64-grub, the symbol links are created in the sub folders.
@lguohan , do you mean to copy the packages into /var/cache/apt/archives? The platform folder is created before the filesystem ready in boot loader (Aboot/Uboot/Grub), maybe the folder can not be used.

for f in $(find $FILESYSTEM_ROOT/var/cache/apt/archives -name "*.deb"); do
sudo mv $f $FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/
done
Expand Down
6 changes: 4 additions & 2 deletions installer/arm64/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,9 @@ fi
# Decompress the file for the file system directly to the partition
if [ x"$docker_inram" = x"on" ]; then
# when disk is small, keep dockerfs.tar.gz in disk, expand it into ramfs during initrd
unzip -o $ONIE_INSTALLER_PAYLOAD -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "platform.tar.gz" -d $demo_mnt/$image_dir
else
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" "platform.tar.gz" -d $demo_mnt/$image_dir

if [ "$install_env" = "onie" ]; then
TAR_EXTRA_OPTION="--numeric-owner"
Expand All @@ -168,6 +168,8 @@ else
unzip -op $ONIE_INSTALLER_PAYLOAD "$FILESYSTEM_DOCKERFS" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/$DOCKERFS_DIR
fi

mkdir -p $demo_mnt/$image_dir/platform
unzip -op $ONIE_INSTALLER_PAYLOAD "platform.tar.gz" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/platform

if [ "$install_env" = "onie" ]; then
# Store machine description in target file system
Expand Down
6 changes: 4 additions & 2 deletions installer/armhf/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,9 @@ fi
# Decompress the file for the file system directly to the partition
if [ x"$docker_inram" = x"on" ]; then
# when disk is small, keep dockerfs.tar.gz in disk, expand it into ramfs during initrd
unzip -o $ONIE_INSTALLER_PAYLOAD -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "platform.tar.gz" -d $demo_mnt/$image_dir
else
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" "platform.tar.gz" -d $demo_mnt/$image_dir

if [ "$install_env" = "onie" ]; then
TAR_EXTRA_OPTION="--numeric-owner"
Expand All @@ -168,6 +168,8 @@ else
unzip -op $ONIE_INSTALLER_PAYLOAD "$FILESYSTEM_DOCKERFS" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/$DOCKERFS_DIR
fi

mkdir -p $demo_mnt/$image_dir/platform
unzip -op $ONIE_INSTALLER_PAYLOAD "platform.tar.gz" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/platform

if [ "$install_env" = "onie" ]; then
# Store machine description in target file system
Expand Down
7 changes: 5 additions & 2 deletions installer/x86_64/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,9 +536,9 @@ fi
# Decompress the file for the file system directly to the partition
if [ x"$docker_inram" = x"on" ]; then
# when disk is small, keep dockerfs.tar.gz in disk, expand it into ramfs during initrd
unzip -o $ONIE_INSTALLER_PAYLOAD -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "platform.tar.gz" -d $demo_mnt/$image_dir
else
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" -d $demo_mnt/$image_dir
unzip -o $ONIE_INSTALLER_PAYLOAD -x "$FILESYSTEM_DOCKERFS" "platform.tar.gz" -d $demo_mnt/$image_dir

if [ "$install_env" = "onie" ]; then
TAR_EXTRA_OPTION="--numeric-owner"
Expand All @@ -549,6 +549,9 @@ else
unzip -op $ONIE_INSTALLER_PAYLOAD "$FILESYSTEM_DOCKERFS" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/$DOCKERFS_DIR
fi

mkdir -p $demo_mnt/$image_dir/platform
unzip -op $ONIE_INSTALLER_PAYLOAD "platform.tar.gz" | tar xz $TAR_EXTRA_OPTION -f - -C $demo_mnt/$image_dir/platform

if [ "$install_env" = "onie" ]; then
# Store machine description in target file system
if [ -f /etc/machine-build.conf ]; then
Expand Down