-
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Fix nightly/release client bundling and VM CI regressions #12006
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
edac1a4
Guard the cmux-tui client commit resolution in nightly and release
austinywang fcc7a1c
Resolve the bundled cmux-tui client commit from shallow CI checkouts
austinywang eb39a93
Merge remote-tracking branch 'origin/main' into fix/nightly-cmux-tui-…
austinywang 9888d1c
Address review: decimal --max-fallback, no fixed retry delay in the m…
austinywang 8215b46
test: determinize the two tests the determinism gate flags on main
austinywang dae6b05
Merge remote-tracking branch 'origin/main' into fix/nightly-cmux-tui-…
austinywang 991e3ed
fix(cloud): clear the two new Swift warnings that exceed the warning …
austinywang 4e572a2
Merge branch 'main' of https://github.com/manaflow-ai/cmux into fix/n…
austinywang fd228aa
test: reproduce client installer failure under macOS Bash
austinywang b22bf35
fix(ci): unblock client packaging, warning checks, and CLI help probes
austinywang 04b399f
Merge branch 'main' into fix/nightly-cmux-tui-client-resolution
austinywang 748c648
test(vms): cover resolved allowances across paused VM access paths
austinywang 192fef6
fix(vms): preserve resolved allowances and retryable Base reset confl…
austinywang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| #!/usr/bin/env bash | ||
| # resolve-cmux-tui-client-commit.sh — print the cmux-tui commit whose published client | ||
| # an app bundle should carry. | ||
| # | ||
| # The `cmux-tui artifacts` workflow publishes one client build per main commit that | ||
| # touches cmux-tui or its build inputs, at | ||
| # https://files.cmux.com/cmux-tui/<commit>/manifest.json. The app workflows bundle the | ||
| # client of the newest such commit in the checked-out history. | ||
| # | ||
| # A bare `git log -1 -- <paths>` is wrong on CI: actions/checkout clones with depth 1, | ||
| # and in a one-commit history the grafted root shows every file as added, so the answer | ||
| # is always HEAD. HEAD has a published client only when it touched cmux-tui itself, so | ||
| # the download 404s on almost every push. This deepens a shallow clone until real | ||
| # cmux-tui history is visible, ignores shallow boundary commits, and walks candidates | ||
| # newest first until one has a published manifest. | ||
| # | ||
| # Usage: scripts/ci/resolve-cmux-tui-client-commit.sh [--max-fallback <n>] [--head <rev>] | ||
| # --max-fallback <n> older published commits that may stand in when newer candidates | ||
| # have no manifest yet (artifacts run failed or still running). | ||
| # Default 0: the newest candidate must be published, or this fails. | ||
| # --head <rev> history to search (default HEAD). | ||
| # Env: CMUX_TUI_CLIENT_MANIFEST_BASE (default https://files.cmux.com/cmux-tui), | ||
| # CMUX_TUI_CLIENT_REMOTE (default origin; where a shallow clone deepens from). | ||
| # The chosen 40-hex commit is the only stdout line; diagnostics go to stderr. | ||
| set -euo pipefail | ||
|
|
||
| PATHS=(cmux-tui ghostty .github/workflows/cmux-tui-artifacts.yml .github/workflows/cmux-tui-build-package.yml) | ||
| BASE="${CMUX_TUI_CLIENT_MANIFEST_BASE:-https://files.cmux.com/cmux-tui}" | ||
| REMOTE="${CMUX_TUI_CLIENT_REMOTE:-origin}" | ||
| MAX_FALLBACK=0 | ||
| HEAD_REV=HEAD | ||
|
|
||
| log() { echo "resolve-cmux-tui-client-commit: $*" >&2; } | ||
| usage() { sed -n '2,24p' "$0"; } | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --max-fallback) shift; MAX_FALLBACK="${1:?--max-fallback needs a value}" ;; | ||
| --head) shift; HEAD_REV="${1:?--head needs a value}" ;; | ||
| -h|--help) usage; exit 0 ;; | ||
| *) echo "unknown option: $1" >&2; usage >&2; exit 64 ;; | ||
| esac | ||
| shift | ||
| done | ||
| case "$MAX_FALLBACK" in | ||
| ''|*[!0-9]*) echo "error: --max-fallback must be a non-negative integer" >&2; exit 64 ;; | ||
| esac | ||
| # Decimal, so a value with a leading zero (08) is not read as octal by the arithmetic below. | ||
| MAX_FALLBACK=$((10#$MAX_FALLBACK)) | ||
|
|
||
| head_sha="$(git rev-parse --verify "${HEAD_REV}^{commit}")" | ||
| shallow_file="$(git rev-parse --git-path shallow)" | ||
| want=$((MAX_FALLBACK + 1)) | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| is_shallow_boundary() { | ||
| [[ -f "$shallow_file" ]] && grep -qx "$1" "$shallow_file" | ||
| } | ||
|
|
||
| # Candidates: commits in the history that touch the client inputs, newest first. A | ||
| # shallow boundary commit is skipped: with its parents missing, git shows it as adding | ||
| # every file, so it would match whether or not it touched cmux-tui. | ||
| CANDIDATES=() | ||
| collect_candidates() { | ||
| CANDIDATES=() | ||
| local sha | ||
| while IFS= read -r sha; do | ||
| [[ -n "$sha" ]] || continue | ||
| if is_shallow_boundary "$sha"; then continue; fi | ||
| CANDIDATES[${#CANDIDATES[@]}]="$sha" | ||
| done < <(git log -n $((want + 8)) --format=%H "$head_sha" -- "${PATHS[@]}") | ||
| } | ||
|
|
||
| deepen=200 | ||
| rounds=0 | ||
| while :; do | ||
| collect_candidates | ||
| if [[ ${#CANDIDATES[@]} -ge $want ]]; then break; fi | ||
| # No shallow file means the history is complete: what we have is all there is. | ||
| if [[ ! -f "$shallow_file" ]]; then break; fi | ||
| if [[ $rounds -ge 6 ]]; then | ||
| log "gave up deepening after $rounds rounds with ${#CANDIDATES[@]} usable candidate(s)" | ||
| break | ||
| fi | ||
| log "shallow clone shows ${#CANDIDATES[@]} usable cmux-tui commit(s); deepening by $deepen from $REMOTE" | ||
| if ! git fetch --quiet --deepen="$deepen" "$REMOTE" "$head_sha" 2>/dev/null \ | ||
| && ! git fetch --quiet --deepen="$deepen" "$REMOTE"; then | ||
| log "could not deepen the clone from $REMOTE" | ||
| break | ||
| fi | ||
| rounds=$((rounds + 1)) | ||
| deepen=$((deepen * 2)) | ||
| done | ||
|
|
||
| if [[ ${#CANDIDATES[@]} -eq 0 ]]; then | ||
| echo "error: no commit touching ${PATHS[*]} is visible from $head_sha" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| chosen="" | ||
| skipped=0 | ||
| for ((i = 0; i < ${#CANDIDATES[@]}; i++)); do | ||
| sha="${CANDIDATES[$i]}" | ||
| url="$BASE/$sha/manifest.json" | ||
| # One probe per candidate, no retry loop: a transient failure just moves on to the | ||
| # next candidate (or fails exact mode, which a re-run covers) instead of waiting. | ||
| if curl --proto '=https,file' --tlsv1.2 -fsS -o /dev/null "$url" 2>/dev/null; then | ||
| chosen="$sha" | ||
| break | ||
| fi | ||
| log "no published cmux-tui client for $sha ($url)" | ||
| skipped=$((skipped + 1)) | ||
| if [[ $skipped -gt $MAX_FALLBACK ]]; then break; fi | ||
| done | ||
|
|
||
| if [[ -z "$chosen" ]]; then | ||
| echo "error: the newest cmux-tui commit ${CANDIDATES[0]} has no published client at $BASE/${CANDIDATES[0]}/manifest.json" >&2 | ||
| echo " check the 'cmux-tui artifacts' run for that commit (--max-fallback $MAX_FALLBACK)" >&2 | ||
| exit 1 | ||
| fi | ||
| if [[ $skipped -gt 0 ]]; then | ||
| echo "::warning title=cmux-tui client fallback::bundling the client of $chosen; $skipped newer cmux-tui commit(s) have no published artifacts (newest: ${CANDIDATES[0]})" >&2 | ||
| fi | ||
| log "using cmux-tui commit $chosen" | ||
| printf '%s\n' "$chosen" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| #!/usr/bin/env bash | ||
| # Guards scripts/ci/resolve-cmux-tui-client-commit.sh, which the nightly and release | ||
| # workflows use to pick the cmux-tui client build to bundle. | ||
| # | ||
| # Regression: actions/checkout clones with depth 1. In a one-commit history the grafted | ||
| # root shows every file as added, so `git log -1 -- cmux-tui` answers HEAD whether or | ||
| # not HEAD touched cmux-tui. HEAD only has a published client when it touched cmux-tui, | ||
| # so the manifest download returned 404 on almost every push (nightly runs 33941558929 | ||
| # and 33943122606 died in "Bundle the cmux-tui client" with `curl: (56) ... 404`). | ||
| set -euo pipefail | ||
|
|
||
| ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| RESOLVER="$ROOT_DIR/scripts/ci/resolve-cmux-tui-client-commit.sh" | ||
| if [[ ! -x "$RESOLVER" ]]; then | ||
| echo "FAIL: missing executable $RESOLVER" | ||
| exit 1 | ||
| fi | ||
|
|
||
| TMP="$(mktemp -d)" | ||
| trap 'rm -rf "$TMP"' EXIT | ||
| export GIT_AUTHOR_NAME=cmux-test GIT_AUTHOR_EMAIL=test@example.com | ||
| export GIT_COMMITTER_NAME=cmux-test GIT_COMMITTER_EMAIL=test@example.com | ||
| export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 | ||
|
|
||
| git init -q "$TMP/src" | ||
| git -C "$TMP/src" checkout -q -b main | ||
| commit_touching() { | ||
| mkdir -p "$(dirname "$TMP/src/$2")" | ||
| echo "$1" >"$TMP/src/$2" | ||
| git -C "$TMP/src" add -A | ||
| git -C "$TMP/src" commit -q -m "$1" | ||
| git -C "$TMP/src" rev-parse HEAD | ||
| } | ||
| C1="$(commit_touching "tui one" cmux-tui/src/main.rs)" | ||
| C2="$(commit_touching "app change" Sources/App.swift)" | ||
| C3="$(commit_touching "tui two" cmux-tui/src/lib.rs)" | ||
| C4="$(commit_touching "docs" docs/notes.md)" | ||
| C5="$(commit_touching "web" web/app.ts)" | ||
| : "$C2" "$C4" | ||
|
|
||
| # The artifact store: only C1 and C3 have a published client, like main after an | ||
| # artifacts run that only exists for commits touching cmux-tui. | ||
| STORE="$TMP/store" | ||
| mkdir -p "$STORE/$C1" "$STORE/$C3" | ||
| printf '{"commit":"%s"}\n' "$C1" >"$STORE/$C1/manifest.json" | ||
| printf '{"commit":"%s"}\n' "$C3" >"$STORE/$C3/manifest.json" | ||
| export CMUX_TUI_CLIENT_MANIFEST_BASE="file://$STORE" | ||
|
|
||
| git clone -q --depth 1 "file://$TMP/src" "$TMP/work" | ||
|
|
||
| # This is the bug the resolver exists for: a depth-1 clone answers HEAD. | ||
| naive="$(git -C "$TMP/work" log -1 --format=%H -- cmux-tui)" | ||
| if [[ "$naive" != "$C5" ]]; then | ||
| echo "FAIL: expected the depth-1 clone to answer HEAD ($C5) for the naive query, got $naive" | ||
| exit 1 | ||
| fi | ||
|
|
||
| got="$(cd "$TMP/work" && "$RESOLVER")" | ||
| if [[ "$got" != "$C3" ]]; then | ||
| echo "FAIL: shallow clone must resolve the newest published cmux-tui commit $C3, got '$got'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # The newest cmux-tui commit lost its artifacts (failed or still-running artifacts run). | ||
| rm "$STORE/$C3/manifest.json" | ||
| if (cd "$TMP/work" && "$RESOLVER" >/dev/null 2>&1); then | ||
| echo "FAIL: exact mode must fail when the newest cmux-tui commit has no published client" | ||
| exit 1 | ||
| fi | ||
| got="$(cd "$TMP/work" && "$RESOLVER" --max-fallback 3 2>"$TMP/fallback.err")" | ||
| if [[ "$got" != "$C1" ]]; then | ||
| echo "FAIL: fallback must pick the previous published commit $C1, got '$got'" | ||
| exit 1 | ||
| fi | ||
| if ! grep -q '^::warning' "$TMP/fallback.err"; then | ||
| echo "FAIL: a fallback must annotate the run with a ::warning" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # A leading zero is decimal, not octal: 08 means eight, not a Bash arithmetic error. | ||
| got="$(cd "$TMP/work" && "$RESOLVER" --max-fallback 08 2>/dev/null)" | ||
| if [[ "$got" != "$C1" ]]; then | ||
| echo "FAIL: --max-fallback 08 must be read as decimal 8 and resolve $C1, got '$got'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # A full clone takes the same decision without deepening. | ||
| git clone -q "file://$TMP/src" "$TMP/full" | ||
| got="$(cd "$TMP/full" && "$RESOLVER" --max-fallback 3 2>/dev/null)" | ||
| if [[ "$got" != "$C1" ]]; then | ||
| echo "FAIL: full clone must resolve $C1, got '$got'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "PASS: resolve-cmux-tui-client-commit picks the newest published cmux-tui commit, shallow or not" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.