-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade actions that use node16 to versions that use node20 #1779
Upgrade actions that use node16 to versions that use node20 #1779
Conversation
last time I tried that it failed because of our old 18.04 images |
…can use the devcontainers/ci action
@miscco yeah, I'm testing whether it's possible to move the node20 actions outside of the 18.04 images. |
This reverts commit 7c4f44a.
🟨 CI finished in 5h 33m: Pass: 99%/365 | Total: 6d 00h | Avg: 23m 45s | Max: 55m 50s | Hits: 72%/522045
|
Project | |
---|---|
+/- | CCCL Infrastructure |
libcu++ | |
CUB | |
Thrust | |
CUDA Experimental |
Modifications in project or dependencies?
Project | |
---|---|
+/- | CCCL Infrastructure |
+/- | libcu++ |
+/- | CUB |
+/- | Thrust |
+/- | CUDA Experimental |
🏃 Runner counts (total jobs: 365)
# | Runner |
---|---|
264 | linux-amd64-cpu16 |
56 | linux-amd64-gpu-v100-latest-1 |
24 | linux-arm64-cpu16 |
21 | windows-amd64-cpu16 |
🟨 CI finished in 7h 29m: Pass: 0%/365 | Total: 5d 23h | Avg: 23m 40s | Max: 55m 50s
|
Project | |
---|---|
+/- | CCCL Infrastructure |
libcu++ | |
CUB | |
Thrust | |
CUDA Experimental |
Modifications in project or dependencies?
Project | |
---|---|
+/- | CCCL Infrastructure |
+/- | libcu++ |
+/- | CUB |
+/- | Thrust |
+/- | CUDA Experimental |
🏃 Runner counts (total jobs: 365)
# | Runner |
---|---|
264 | linux-amd64-cpu16 |
56 | linux-amd64-gpu-v100-latest-1 |
24 | linux-arm64-cpu16 |
21 | windows-amd64-cpu16 |
…o download after re-running individual failing jobs
🟨 CI finished in 2h 03m: Pass: 99%/365 | Total: 2d 14h | Avg: 10m 19s | Max: 1h 19m | Hits: 88%/520864
|
Project | |
---|---|
+/- | CCCL Infrastructure |
libcu++ | |
CUB | |
Thrust | |
CUDA Experimental |
Modifications in project or dependencies?
Project | |
---|---|
+/- | CCCL Infrastructure |
+/- | libcu++ |
+/- | CUB |
+/- | Thrust |
+/- | CUDA Experimental |
🏃 Runner counts (total jobs: 365)
# | Runner |
---|---|
264 | linux-amd64-cpu16 |
56 | linux-amd64-gpu-v100-latest-1 |
24 | linux-arm64-cpu16 |
21 | windows-amd64-cpu16 |
.devcontainer/launch.sh
Outdated
tabs_to_spaces() { | ||
sed $'s/\t/ /g' | ||
} | ||
|
||
no_empty_lines() { | ||
grep -v -e '^$' || [ "$?" == "1" ] | ||
} | ||
|
||
trim_leading_whitespace() { | ||
sed 's/^[[:space:]]*//' | ||
} | ||
|
||
trim_trailing_whitespace() { | ||
sed 's/[[:space:]]*$//' | ||
} | ||
|
||
remove_leading_char() { | ||
cut -d"$1" -f1 --complement | ||
} | ||
|
||
remove_trailing_char() { | ||
rev | cut -d"$1" -f1 --complement | rev | ||
} | ||
|
||
strip_enclosing_chars() { | ||
remove_leading_char "$1" \ | ||
| remove_trailing_char "$2" | ||
} | ||
|
||
read_docker_image_metadata() { | ||
docker inspect --type image --format '{{json .Config.Labels}}' "$DOCKER_IMAGE" \ | ||
| sed -r 's/.*"devcontainer.metadata":[ ]*"(.*[]\)])",?.*/\1/g' \ | ||
| sed -r 's/\\\"/"/g' | ||
} | ||
|
||
json_map_key_to_shell_syntax() { | ||
sed -r 's/^"(.*)":[ ]*?([^,]*),?$/\1=\2/g' | ||
} | ||
|
||
translate_local_envvars_to_shell_syntax() { | ||
sed -r 's/\$\{localEnv:([^\:]*):?(.*)\}/${\1:-\2}/g' | ||
} | ||
|
||
inline_local_workspace_folder() { | ||
sed "s@\${localWorkspaceFolder}@$(pwd)@g" | ||
} | ||
|
||
inline_container_workspace_folder() { | ||
sed "s@\${containerWorkspaceFolder}@${WORKSPACE_FOLDER:-}@g" | ||
} | ||
|
||
inline_local_workspace_folder_basename() { | ||
sed "s@\${localWorkspaceFolderBasename}@$(basename "$(pwd)")@g" | ||
} | ||
|
||
transform_to_one_line() { | ||
tabs_to_spaces \ | ||
| trim_leading_whitespace \ | ||
| trim_trailing_whitespace \ | ||
| inline_local_workspace_folder \ | ||
| inline_container_workspace_folder \ | ||
| inline_local_workspace_folder_basename \ | ||
| translate_local_envvars_to_shell_syntax \ | ||
| tr -s '\n' '\t' | ||
} | ||
|
||
json_string() { | ||
transform_to_one_line \ | ||
| grep -Po "\"$1\":\s*\"(.*)\"" \ | ||
| sed -r "s/.*\"$1\":[ ]*\"([^\"]*)\",?.*/\1/g" \ | ||
| no_empty_lines | ||
} | ||
|
||
json_map() { | ||
transform_to_one_line \ | ||
| grep -Po "\"$1\":\s*{(.*?)\s*?}[^\"]" \ | ||
| strip_enclosing_chars '{' '}' \ | ||
| tr -s '\t' '\n' \ | ||
| json_map_key_to_shell_syntax \ | ||
| no_empty_lines | ||
} | ||
|
||
json_array() { | ||
transform_to_one_line \ | ||
| grep -Po "\"$1\":\s*\[(.*?)\s*?\][^\"]" \ | ||
| strip_enclosing_chars '[' ']' \ | ||
| tr -s '\t' '\n' \ | ||
| sed -r 's/", "/"\n"/g' \ | ||
| sed -r 's/^(.*"),$/\1/' \ | ||
| no_empty_lines | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jrhemstad I'd like your thoughts on this. @trxcllnt implemented this set of bash functions to parse JSON without needing jq
as a dependency for users of the launch.sh
script.
Do you have any concerns about requiring jq
here? We could simplify this implementation quite a bit with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with adding modes to the script that have as many dependencies as we want, so long as there's a mode that has no additional dependencies than what we had before for the manual docker mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to use python -c
to parse JSON and select the fields:
Lines 109 to 179 in 63cc99b
inline_vars() { | |
cat - \ | |
`# inline local workspace folder` \ | |
| sed "s@\${localWorkspaceFolder}@$(pwd)@g" \ | |
`# inline local workspace folder basename` \ | |
| sed "s@\${localWorkspaceFolderBasename}@$(basename "$(pwd)")@g" \ | |
`# inline container workspace folder` \ | |
| sed "s@\${containerWorkspaceFolder}@${WORKSPACE_FOLDER:-}@g" \ | |
`# inline container workspace folder basename` \ | |
| sed "s@\${containerWorkspaceFolderBasename}@$(basename "${WORKSPACE_FOLDER:-}")@g" \ | |
`# translate local envvars to shell syntax` \ | |
| sed -r 's/\$\{localEnv:([^\:]*):?(.*)\}/${\1:-\2}/g' | |
} | |
args_to_path() { | |
local -a keys=("${@}") | |
keys=("${keys[@]/#/[}") | |
keys=("${keys[@]/%/]}") | |
echo "$(IFS=; echo "${keys[*]}")" | |
} | |
json_string() { | |
python3 -c "import json,sys; print(json.load(sys.stdin)$(args_to_path "${@}"))" 2>/dev/null | inline_vars | |
} | |
json_array() { | |
python3 -c "import json,sys; [print(f'\"{x}\"') for x in json.load(sys.stdin)$(args_to_path "${@}")]" 2>/dev/null | inline_vars | |
} | |
json_map() { | |
python3 -c "import json,sys; [print(f'{k}=\"{v}\"') for k,v in json.load(sys.stdin)$(args_to_path "${@}").items()]" 2>/dev/null | inline_vars | |
} | |
devcontainer_metadata_json() { | |
docker inspect --type image --format '{{json .Config.Labels}}' "$DOCKER_IMAGE" \ | |
| json_string '"devcontainer.metadata"' | |
} | |
### | |
# Read relevant values from devcontainer.json | |
### | |
local devcontainer_json="${path}/devcontainer.json"; | |
# Read image | |
local DOCKER_IMAGE="$(json_string '"image"' < "${devcontainer_json}")" | |
# Always pull the latest copy of the image | |
docker pull "$DOCKER_IMAGE" | |
# Read workspaceFolder | |
local WORKSPACE_FOLDER="$(json_string '"workspaceFolder"' < "${devcontainer_json}")" | |
# Read remoteUser | |
local REMOTE_USER="$(json_string '"remoteUser"' < "${devcontainer_json}")" | |
# If remoteUser isn't in our devcontainer.json, read it from the image's "devcontainer.metadata" label | |
if test -z "${REMOTE_USER:-}"; then | |
REMOTE_USER="$(devcontainer_metadata_json | json_string "-1" '"remoteUser"')" | |
fi | |
# Read runArgs | |
local -a RUN_ARGS="($(json_array '"runArgs"' < "${devcontainer_json}"))" | |
# Read initializeCommand | |
local -a INITIALIZE_COMMAND="($(json_array '"initializeCommand"' < "${devcontainer_json}"))" | |
# Read containerEnv | |
local -a ENV_VARS="($(json_map '"containerEnv"' < "${devcontainer_json}" | sed -r 's/(.*)=(.*)/--env \1=\2/'))" | |
# Read mounts | |
local -a MOUNTS="($( | |
tee < "${devcontainer_json}" \ | |
1>/dev/null \ | |
>(json_array '"mounts"') \ | |
>(json_string '"workspaceMount"') \ | |
| xargs -r -I% echo --mount '%' | |
))" |
🟩 CI finished in 7h 19m: Pass: 100%/365 | Total: 3d 03h | Avg: 12m 21s | Max: 54m 06s | Hits: 77%/522045
|
Project | |
---|---|
+/- | CCCL Infrastructure |
libcu++ | |
CUB | |
Thrust | |
CUDA Experimental |
Modifications in project or dependencies?
Project | |
---|---|
+/- | CCCL Infrastructure |
+/- | libcu++ |
+/- | CUB |
+/- | Thrust |
+/- | CUDA Experimental |
🏃 Runner counts (total jobs: 365)
# | Runner |
---|---|
264 | linux-amd64-cpu16 |
56 | linux-amd64-gpu-v100-latest-1 |
24 | linux-arm64-cpu16 |
21 | windows-amd64-cpu16 |
See NVIDIA/cccl#1779 Signed-off-by: Jordan Jacobelli <[email protected]>
See NVIDIA/cccl#1779 Signed-off-by: Jordan Jacobelli <[email protected]>
See NVIDIA/cccl#1779 Signed-off-by: Jordan Jacobelli <[email protected]>
* devcontainer: replace VAULT_HOST with AWS_ROLE_ARN Signed-off-by: Jordan Jacobelli <[email protected]> * Update devcontainers base image to support AWS_ROLE_ARN Signed-off-by: Jordan Jacobelli <[email protected]> * Bump cuda latest version to 12.6 Signed-off-by: Jordan Jacobelli <[email protected]> * Replace ubuntu18.04 with ubuntu20.04 Ubuntu 18.04 is not supported anymore Signed-off-by: Jordan Jacobelli <[email protected]> * Use DOOD stategy to keep supporting ubuntu18.04 See NVIDIA/cccl#1779 Signed-off-by: Jordan Jacobelli <[email protected]> --------- Signed-off-by: Jordan Jacobelli <[email protected]>
Description
Update the following actions:
actions/checkout@v3
->v4
aws-actions/configure-aws-credentials@v2
->v4
actions/upload-artifact@v3
->v4
actions/download-artifact@v3
->v4
Closes #1778