failure during install_opte.sh left corrupt file that breaks pkg (not fixed by re-run) #1322
Description
I tried to update prerequisites in my clone:
$ ./tools/install_prerequisites.sh
Install (or update) [pkg:/package/pkg build-essential library/postgresql-13 pkg-config brand/omicron1/tools library/libxmlsec1 pkg:/ooce/developer/clang-120]? (y/n): y
No updates necessary for this image.
FMRI IFO
pkg://helios-dev/developer/[email protected]:20210409T072026Z im-
pkg://helios-dev/developer/[email protected]:20201015T092246Z im-
pkg://helios-dev/library/[email protected]:20220506T083154Z im-
pkg://helios-dev/ooce/developer/[email protected]:20210507T211404Z im-
pkg://helios-dev/ooce/library/[email protected]:20201030T111312Z im-
pkg://helios-dev/package/[email protected]:20220522T114219Z im-
pkg://helios-dev/system/zones/brand/omicron1/[email protected]:20220523T144242Z im-
+ set -o errexit
+++ dirname ./tools/ci_download_cockroachdb
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron-fixes/tools
++ basename ./tools/ci_download_cockroachdb
+ ARG0=ci_download_cockroachdb
++ cat /home/dap/omicron-fixes/tools/cockroachdb_version
+ CIDL_VERSION=v21.2.9
+ source /home/dap/omicron-fixes/tools/cockroachdb_checksums
++ CIDL_MD5_DARWIN=1074a0946f57b65331a3a5723e068645
++ CIDL_MD5_LINUX=50348e9a4e48d4a6ef973650a9edf09d
++ CIDL_MD5_ILLUMOS=7efc03b988983e3f1829814c48553822
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/cockroachdb
+ CIDL_URL_COCKROACH=https://binaries.cockroachdb.com
+ CIDL_URL_ILLUMOS=https://illumos.org/downloads
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron-fixes'
current directory: /home/dap/omicron-fixes
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ CIDL_BUILD=illumos
+ CIDL_SUFFIX=tar.gz
+ CIDL_MD5=7efc03b988983e3f1829814c48553822
+ CIDL_MD5FUNC=do_md5sum
+ CIDL_URL_BASE=https://illumos.org/downloads
+ CIDL_ASSEMBLE=do_assemble_illumos
+ TARBALL_DIRNAME=cockroach-v21.2.9.illumos
+ TARBALL_FILENAME=cockroach-v21.2.9.illumos.tar.gz
+ TARBALL_FILE=out/downloads/cockroach-v21.2.9.illumos.tar.gz
+ TARBALL_DIR=out/downloads/cockroach-v21.2.9.illumos
+ CIDL_URL=https://illumos.org/downloads/cockroach-v21.2.9.illumos.tar.gz
+ echo 'URL: https://illumos.org/downloads/cockroach-v21.2.9.illumos.tar.gz'
URL: https://illumos.org/downloads/cockroach-v21.2.9.illumos.tar.gz
+ echo 'Local file: out/downloads/cockroach-v21.2.9.illumos.tar.gz'
Local file: out/downloads/cockroach-v21.2.9.illumos.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/cockroachdb
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/cockroach-v21.2.9.illumos.tar.gz ]]
++ do_md5sum out/downloads/cockroach-v21.2.9.illumos.tar.gz
++ md5sum
++ awk '{print $1}'
+ calculated_md5=7efc03b988983e3f1829814c48553822
+ [[ 7efc03b988983e3f1829814c48553822 == \7\e\f\c\0\3\b\9\8\8\9\8\3\e\3\f\1\8\2\9\8\1\4\c\4\8\5\5\3\8\2\2 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/cockroach-v21.2.9.illumos.tar.gz
+ tar xzf out/downloads/cockroach-v21.2.9.illumos.tar.gz -C out/downloads
+ do_assemble_illumos
+ rm -r ./out/cockroachdb
+ cp -r out/downloads/cockroach-v21.2.9 ./out/cockroachdb
+ ./out/cockroachdb/bin/cockroach version
Build Tag: v21.2.9
Build Time: 2022/04/28 04:02:42
Distribution: OSS
Platform: illumos amd64 (x86_64-pc-solaris2.11)
Go Version: go1.16.10
C Compiler: gcc 10.3.0
Build Commit ID: 11787edfcfc157a0df951abc34684e4e18b3ef20
Build Type: release
+ set -o errexit
+++ dirname ./tools/ci_download_clickhouse
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron-fixes/tools
++ basename ./tools/ci_download_clickhouse
+ ARG0=ci_download_clickhouse
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/clickhouse
+ CIDL_VERSION=v21.7
+ source /home/dap/omicron-fixes/tools/clickhouse_checksums
++ CIDL_MD5_DARWIN=7e22bd0b0108dee55605dd563a5698df
++ CIDL_MD5_LINUX=f821613422cda8bb3c7f281eeb87f145
++ CIDL_MD5_ILLUMOS=2870157143dc78b1ebe1e3dfabe5fd61
+ CIDL_URL_BASE=https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron-fixes'
current directory: /home/dap/omicron-fixes
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ CIDL_PLATFORM=illumos
+ CIDL_MD5=2870157143dc78b1ebe1e3dfabe5fd61
+ CIDL_MD5FUNC=do_md5sum
+ TARBALL_DIRNAME=clickhouse-v21.7
+ TARBALL_FILENAME=clickhouse-v21.7.illumos.tar.gz
+ TARBALL_FILE=out/downloads/clickhouse-v21.7.illumos.tar.gz
+ TARBALL_DIR=out/downloads/clickhouse-v21.7
+ CIDL_URL=https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v21.7.illumos.tar.gz
+ echo 'URL: https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v21.7.illumos.tar.gz'
URL: https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v21.7.illumos.tar.gz
+ echo 'Local file: out/downloads/clickhouse-v21.7.illumos.tar.gz'
Local file: out/downloads/clickhouse-v21.7.illumos.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/clickhouse
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/clickhouse-v21.7.illumos.tar.gz ]]
++ do_md5sum out/downloads/clickhouse-v21.7.illumos.tar.gz
++ md5sum
++ awk '{print $1}'
+ calculated_md5=2870157143dc78b1ebe1e3dfabe5fd61
+ [[ 2870157143dc78b1ebe1e3dfabe5fd61 == \2\8\7\0\1\5\7\1\4\3\d\c\7\8\b\1\e\b\e\1\e\3\d\f\a\b\e\5\f\d\6\1 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/clickhouse-v21.7.illumos.tar.gz ./out/clickhouse
+ mkdir -p ./out/clickhouse
+ tar xzf out/downloads/clickhouse-v21.7.illumos.tar.gz -C ./out/clickhouse
+ ./out/clickhouse/clickhouse server --version
ClickHouse server version 21.7.1.1.
+ set -o errexit
+++ dirname ./tools/ci_download_console
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron-fixes/tools
++ basename ./tools/ci_download_console
+ ARG0=ci_download_console
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/console-assets
+ URL_BASE=https://dl.oxide.computer/releases/console
+ source /home/dap/omicron-fixes/tools/console_version
++ COMMIT=a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9
++ SHA2=5f5cf28e245fd9b252f7610a36a74ce68305e91544ca6213ba30517aa5834b00
+ URL=https://dl.oxide.computer/releases/console/a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9.tar.gz
+ TARBALL_FILE=out/downloads/console.tar.gz
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://dl.oxide.computer/releases/console/a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9.tar.gz'
URL: https://dl.oxide.computer/releases/console/a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9.tar.gz
+ echo 'Local file: out/downloads/console.tar.gz'
Local file: out/downloads/console.tar.gz
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/console.tar.gz ]]
++ do_sha256sum out/downloads/console.tar.gz
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=10c0b7b50f55e8f47e2419c218e435d44268576608e7535a61f728a5ecf53d19
+ [[ 10c0b7b50f55e8f47e2419c218e435d44268576608e7535a61f728a5ecf53d19 == \5\f\5\c\f\2\8\e\2\4\5\f\d\9\b\2\5\2\f\7\6\1\0\a\3\6\a\7\4\c\e\6\8\3\0\5\e\9\1\5\4\4\c\a\6\2\1\3\b\a\3\0\5\1\7\a\a\5\8\3\4\b\0\0 ]]
+ mkdir -p out/downloads
+ mkdir -p ./out/console-assets
+ '[' true == true ']'
+ echo Downloading...
Downloading...
+ do_download_curl https://dl.oxide.computer/releases/console/a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9.tar.gz out/downloads/console.tar.gz
+ curl --silent --show-error --fail --location --output out/downloads/console.tar.gz https://dl.oxide.computer/releases/console/a4b7e4bcc88a9f131c0dd486add7ffd3f838bfd9.tar.gz
++ do_sha256sum out/downloads/console.tar.gz
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=5f5cf28e245fd9b252f7610a36a74ce68305e91544ca6213ba30517aa5834b00
+ [[ 5f5cf28e245fd9b252f7610a36a74ce68305e91544ca6213ba30517aa5834b00 != \5\f\5\c\f\2\8\e\2\4\5\f\d\9\b\2\5\2\f\7\6\1\0\a\3\6\a\7\4\c\e\6\8\3\0\5\e\9\1\5\4\4\c\a\6\2\1\3\b\a\3\0\5\1\7\a\a\5\8\3\4\b\0\0 ]]
+ do_untar out/downloads/console.tar.gz ./out/console-assets
+ mkdir -p ./out/console-assets
+ tar xzf out/downloads/console.tar.gz -C ./out/console-assets
+ set -o errexit
+++ dirname ./tools/ci_download_maghemite_openapi
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron-fixes/tools
++ basename ./tools/ci_download_maghemite_openapi
+ ARG0=ci_download_maghemite_openapi
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ source /home/dap/omicron-fixes/tools/maghemite_openapi_version
++ COMMIT=2be097ddd1d3fd8e7f56bc0a4bfd696253b11454
++ SHA2=003110ea572aea8c578ac606cbd31818d51f820f5cde65ec92700d99de2d9fa5
+ URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/2be097ddd1d3fd8e7f56bc0a4bfd696253b11454/ddm-admin.json
+ LOCAL_FILE=out/downloads/ddm-admin-2be097ddd1d3fd8e7f56bc0a4bfd696253b11454.json
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/2be097ddd1d3fd8e7f56bc0a4bfd696253b11454/ddm-admin.json'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/2be097ddd1d3fd8e7f56bc0a4bfd696253b11454/ddm-admin.json
+ echo 'Local file: out/downloads/ddm-admin-2be097ddd1d3fd8e7f56bc0a4bfd696253b11454.json'
Local file: out/downloads/ddm-admin-2be097ddd1d3fd8e7f56bc0a4bfd696253b11454.json
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/ddm-admin-2be097ddd1d3fd8e7f56bc0a4bfd696253b11454.json ]]
++ do_sha256sum out/downloads/ddm-admin-2be097ddd1d3fd8e7f56bc0a4bfd696253b11454.json
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=003110ea572aea8c578ac606cbd31818d51f820f5cde65ec92700d99de2d9fa5
+ [[ 003110ea572aea8c578ac606cbd31818d51f820f5cde65ec92700d99de2d9fa5 == \0\0\3\1\1\0\e\a\5\7\2\a\e\a\8\c\5\7\8\a\c\6\0\6\c\b\d\3\1\8\1\8\d\5\1\f\8\2\0\f\5\c\d\e\6\5\e\c\9\2\7\0\0\d\9\9\d\e\2\d\9\f\a\5 ]]
+ DO_DOWNLOAD=false
+ mkdir -p out/downloads
+ '[' false == true ']'
+ MARKER=/etc/opt/oxide/NO_INSTALL
+ [[ -f /etc/opt/oxide/NO_INSTALL ]]
++ uname
+ [[ SunOS != \S\u\n\O\S ]]
++ id -u
+ [[ 0 -ne 0 ]]
+++ dirname ./tools/install_opte.sh
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron-fixes/tools
+ cd /home/dap/omicron-fixes/tools/..
+ OMICRON_TOP=/home/dap/omicron-fixes
+ OUT_DIR=/home/dap/omicron-fixes/out
+ XDE_DIR=/home/dap/omicron-fixes/out/xde
+ mkdir -p /home/dap/omicron-fixes/out/xde
+ HELIOS_NETDEV_BASE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo
+ HELIOS_NETDEV_COMMIT=5764c0732a26c6d0e096ce1b8e871b44d8dc79e7
+ HELIOS_NETDEV_REPO_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p
+ HELIOS_NETDEV_REPO_SHA_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p.sha256
++ basename https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p
+ HELIOS_NETDEV_REPO_PATH=/home/dap/omicron-fixes/out/xde/opte.p5p
+ XDE_REPO_BASE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde
+ XDE_REPO_COMMIT=bd79a6eb03c6622760297887458d0601ebc188eb
+ XDE_REPO_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p
+ XDE_REPO_SHA_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p.sha256
++ basename https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p
+ XDE_REPO_PATH=/home/dap/omicron-fixes/out/xde/repo.p5p
++ sha_from_url https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p.sha256
++ local SHA_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p.sha256
++ curl -L https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p.sha256
++ cut -d ' ' -f 1
+ download_and_check_sha https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p 3a2a739ff3ee30789faf1d0a4b1e3bb167009ba930e5181cdd2afe255ff7d350
+ local URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p
++ basename https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/repo/5764c0732a26c6d0e096ce1b8e871b44d8dc79e7/opte.p5p
+ local FILENAME=opte.p5p
+ local OUT_PATH=/home/dap/omicron-fixes/out/xde/opte.p5p
+ local SHA=3a2a739ff3ee30789faf1d0a4b1e3bb167009ba930e5181cdd2afe255ff7d350
+ [[ -f /home/dap/omicron-fixes/out/xde/opte.p5p ]]
++ file_sha /home/dap/omicron-fixes/out/xde/opte.p5p
++ sha256sum /home/dap/omicron-fixes/out/xde/opte.p5p
++ cut -d ' ' -f 1
+ [[ 3a2a739ff3ee30789faf1d0a4b1e3bb167009ba930e5181cdd2afe255ff7d350 != \3\a\2\a\7\3\9\f\f\3\e\e\3\0\7\8\9\f\a\f\1\d\0\a\4\b\1\e\3\b\b\1\6\7\0\0\9\b\a\9\3\0\e\5\1\8\1\c\d\d\2\a\f\e\2\5\5\f\f\7\d\3\5\0 ]]
+ echo '"/home/dap/omicron-fixes/out/xde/opte.p5p" already exists with correct SHA'
"/home/dap/omicron-fixes/out/xde/opte.p5p" already exists with correct SHA
++ sha_from_url https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p.sha256
++ local SHA_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p.sha256
++ curl -L https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p.sha256
++ cut -d ' ' -f 1
+ download_and_check_sha https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p 3f9138ee3dc8651a8823373cd8f3c575c83d8fb7f305c1857898c24b94b2bbce
+ local URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p
++ basename https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p
+ local FILENAME=repo.p5p
+ local OUT_PATH=/home/dap/omicron-fixes/out/xde/repo.p5p
+ local SHA=3f9138ee3dc8651a8823373cd8f3c575c83d8fb7f305c1857898c24b94b2bbce
+ [[ -f /home/dap/omicron-fixes/out/xde/repo.p5p ]]
++ file_sha /home/dap/omicron-fixes/out/xde/repo.p5p
++ sha256sum /home/dap/omicron-fixes/out/xde/repo.p5p
++ cut -d ' ' -f 1
+ [[ 3f9138ee3dc8651a8823373cd8f3c575c83d8fb7f305c1857898c24b94b2bbce != \0\3\5\e\8\2\7\8\e\2\b\0\c\6\c\a\0\f\c\8\d\e\3\4\1\1\6\1\3\2\5\b\0\8\b\c\c\3\3\8\c\8\b\f\b\1\5\9\e\a\3\a\7\4\1\6\6\5\d\b\a\d\4\e ]]
+ curl -L -o /home/dap/omicron-fixes/out/xde/repo.p5p https://buildomat.eng.oxide.computer/public/file/oxidecomputer/os-build/xde/bd79a6eb03c6622760297887458d0601ebc188eb/repo.p5p
Something went wrong, but this script is idempotent - If you can fix the issue, try re-running
Status code is:
$ echo $?
56
From curl(1), this means:
56 Failure in receiving network data.
Okay, maybe a transient network issue. The problem is the script is supposed to be idempotent, but it keeps failing with:
$ ./tools/install_prerequisites.sh
Install (or update) [pkg:/package/pkg build-essential library/postgresql-13 pkg-config brand/omicron1/tools library/libxmlsec1 pkg:/ooce/developer/clang-120]? (y/n): y
pkg: 2/3 catalogs successfully updated:
file protocol error: code: E_TFTP_ILLEGAL (71) reason: Archive /home/dap/omicron-fixes/out/xde/repo.p5p is missing, unsupported, or corrupt.
Repository URL: 'file:///home/dap/omicron-fixes/out/xde/repo.p5p'. (happened 4 times)
Indeed, that file exists:
$ ls out/xde/repo.p5p
-rw-r--r-- 1 root root 71515985 Jun 30 09:18 out/xde/repo.p5p
but it makes sense that it would be incomplete (and so corrupted) given that curl(1) failed to finish downloading it.
It looks like there's code in ./install_opte.sh that will check the SHA and redownload it if it's wrong, but I don't think we get that far. A previous successful invocation of install_prerequisites.sh appears to have registered this repo with pkg(1) which is now quite unhappy. A regular pkg refresh (having nothing to do with any omicron stuff) also fails so this has broken pkg(1) on this system.
Some ideas for possible improvements:
- Download the file to a temporary file, then rename it into place? That way if we have a good one we won't clobber it unless the download completes and the SHA checks out.
- Remove the publisher early in the script, then re-add it later. I'm not sure if this has other side effects we wouldn't want.