-
Notifications
You must be signed in to change notification settings - Fork 65
feat: Migrate vwa workflows clean #625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: notebooks-v1
Are you sure you want to change the base?
Changes from all commits
920944f
861cd57
616d1a8
93bccd1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,167 @@ | ||
name: VWA Frontend Tests | ||
on: | ||
pull_request: | ||
paths: | ||
- components/crud-web-apps/volumes/frontend/** | ||
- releasing/version/VERSION | ||
branches: | ||
- main | ||
- notebooks-v1 | ||
|
||
jobs: | ||
frontend-format-lint-check: | ||
name: Check code format and lint | ||
runs-on: ubuntu-22.04 | ||
steps: | ||
- name: Check out code | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup Node | ||
uses: actions/setup-node@v4 | ||
with: | ||
node-version: 16 | ||
|
||
- name: Check frontend code formatting | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm i | ||
npm run format:check | ||
|
||
- name: Check frontend code linting | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm i | ||
npm run lint-check | ||
Comment on lines
+12
to
+34
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please see this comment I made on a PR from @yehudit1987 that talks about how we can streamline this particular job (given its isolation/simplicity) |
||
|
||
frontend-unit-tests: | ||
runs-on: ubuntu-22.04 | ||
name: Unit tests | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup node version to 16 | ||
uses: actions/setup-node@v4 | ||
with: | ||
node-version: 16 | ||
|
||
- name: Install Kubeflow common library dependecies | ||
run: | | ||
cd components/crud-web-apps/common/frontend/kubeflow-common-lib | ||
npm i | ||
npm run build | ||
npm link ./dist/kubeflow | ||
|
||
- name: Install VWA dependencies | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm i | ||
npm link kubeflow | ||
|
||
- name: Run unit tests | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm run test:prod | ||
|
||
run-tests-in-chrome: | ||
name: UI tests in chrome | ||
runs-on: ubuntu-22.04 | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup node version to 16 | ||
uses: actions/setup-node@v4 | ||
with: | ||
node-version: 16 | ||
|
||
- name: Install Kubeflow common library dependecies | ||
run: | | ||
cd components/crud-web-apps/common/frontend/kubeflow-common-lib | ||
npm i | ||
npm run build | ||
npm link ./dist/kubeflow | ||
|
||
- name: Install VWA dependencies | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm i | ||
npm link kubeflow | ||
|
||
- name: Serve UI & run Cypress tests in Chrome | ||
uses: cypress-io/github-action@v6 | ||
with: | ||
working-directory: components/crud-web-apps/volumes/frontend | ||
start: npm run serve | ||
install: false | ||
browser: chrome | ||
wait-on: "http://localhost:4200" | ||
record: false | ||
|
||
run-tests-in-firefox: | ||
name: UI tests in firefox | ||
runs-on: ubuntu-22.04 | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup node version to 16 | ||
uses: actions/setup-node@v4 | ||
with: | ||
node-version: 16 | ||
|
||
- name: Install Firefox dependencies | ||
run: | | ||
sudo apt-get update | ||
# Install Firefox from snap (more reliable for CI) | ||
sudo snap install firefox --classic | ||
# Install xvfb for headless mode | ||
sudo apt-get install -y xvfb | ||
# Create multiple symlinks to ensure Firefox is found | ||
sudo ln -sf /snap/bin/firefox /usr/local/bin/firefox | ||
sudo ln -sf /snap/bin/firefox /usr/bin/firefox | ||
# Add snap bin to PATH for this session | ||
echo 'export PATH="/snap/bin:$PATH"' >> ~/.bashrc | ||
export PATH="/snap/bin:$PATH" | ||
# Verify Firefox installation and location | ||
which firefox | ||
firefox --version | ||
# Set up display for headless mode | ||
export DISPLAY=:99 | ||
Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & | ||
# Wait a moment for Xvfb to start | ||
sleep 2 | ||
Comment on lines
+113
to
+133
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to a Slack DM I got from @yehudit1987 - it seems Cypress/Firefox issues can be resolved by upgrading Cypress to Can you give that a try to see if it avoid having to do this complicated configuration step? Additionally - these types of (legitimate) changes to the codebase should be called out in the commit message for posterity sake! |
||
|
||
- name: Install Kubeflow common library dependecies | ||
run: | | ||
cd components/crud-web-apps/common/frontend/kubeflow-common-lib | ||
npm i | ||
npm run build | ||
npm link ./dist/kubeflow | ||
|
||
- name: Install VWA dependencies | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
npm i | ||
npm link kubeflow | ||
|
||
- name: Verify Firefox detection | ||
run: | | ||
cd components/crud-web-apps/volumes/frontend | ||
# List available browsers for Cypress | ||
npx cypress info | ||
# Try to detect Firefox specifically | ||
npx cypress run --browser firefox --dry-run || echo "Firefox detection failed, trying alternative approach" | ||
|
||
- name: Serve UI & run Cypress tests in Firefox | ||
uses: cypress-io/github-action@v6 | ||
with: | ||
working-directory: components/crud-web-apps/volumes/frontend | ||
start: npm run serve | ||
install: false | ||
browser: firefox | ||
wait-on: "http://localhost:4200" | ||
record: false | ||
env: | ||
DISPLAY: :99 | ||
PATH: /snap/bin:/usr/local/bin:/usr/bin:/bin |
Original file line number | Diff line number | Diff line change | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,67 @@ | ||||||||||||||||
name: VWA Integration Test | ||||||||||||||||
on: | ||||||||||||||||
pull_request: | ||||||||||||||||
paths: | ||||||||||||||||
- components/crud-web-apps/volumes/** | ||||||||||||||||
- components/crud-web-apps/common/** | ||||||||||||||||
- releasing/version/VERSION | ||||||||||||||||
branches: | ||||||||||||||||
- main | ||||||||||||||||
- notebooks-v1 | ||||||||||||||||
Comment on lines
+8
to
+10
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Need to also include the "release branch" to trigger this workflow - just like its defined in the original
Suggested change
|
||||||||||||||||
|
||||||||||||||||
concurrency: | ||||||||||||||||
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.number }} | ||||||||||||||||
cancel-in-progress: true | ||||||||||||||||
|
||||||||||||||||
env: | ||||||||||||||||
IMG: ghcr.io/kubeflow/notebooks/volumes-web-app | ||||||||||||||||
TAG: integration-test | ||||||||||||||||
|
||||||||||||||||
jobs: | ||||||||||||||||
build: | ||||||||||||||||
runs-on: ubuntu-22.04 | ||||||||||||||||
steps: | ||||||||||||||||
- name: Checkout | ||||||||||||||||
uses: actions/checkout@v4 | ||||||||||||||||
|
||||||||||||||||
- name: Setup Docker Buildx | ||||||||||||||||
uses: docker/setup-buildx-action@v3 | ||||||||||||||||
|
||||||||||||||||
- name: Build VWA Image | ||||||||||||||||
run: | | ||||||||||||||||
cd components/crud-web-apps/volumes | ||||||||||||||||
make docker-build-multi-arch | ||||||||||||||||
|
||||||||||||||||
- name: Install KinD | ||||||||||||||||
run: ./components/testing/gh-actions/install_kind.sh | ||||||||||||||||
|
||||||||||||||||
- name: Create KinD Cluster | ||||||||||||||||
run: kind create cluster --config components/testing/gh-actions/kind-1-25.yaml | ||||||||||||||||
|
||||||||||||||||
- name: Load Image into KinD Cluster | ||||||||||||||||
run: | | ||||||||||||||||
kind load docker-image "${IMG}:${TAG}" | ||||||||||||||||
|
||||||||||||||||
- name: Install kustomize | ||||||||||||||||
run: ./components/testing/gh-actions/install_kustomize.sh | ||||||||||||||||
|
||||||||||||||||
- name: Install Istio | ||||||||||||||||
run: ./components/testing/gh-actions/install_istio.sh | ||||||||||||||||
|
||||||||||||||||
- name: Build & Apply manifests | ||||||||||||||||
run: | | ||||||||||||||||
cd components/crud-web-apps/volumes/manifests | ||||||||||||||||
kubectl create ns kubeflow | ||||||||||||||||
|
||||||||||||||||
export CURRENT_IMAGE="${IMG}" | ||||||||||||||||
export PR_IMAGE="${IMG}:${TAG}" | ||||||||||||||||
|
||||||||||||||||
# escape "." in the image names, as it is a special characters in sed | ||||||||||||||||
export CURRENT_IMAGE=$(echo "$CURRENT_IMAGE" | sed 's|\.|\\\.|g') | ||||||||||||||||
export PR_IMAGE=$(echo "$PR_IMAGE" | sed 's|\.|\\\.|g') | ||||||||||||||||
|
||||||||||||||||
kustomize build overlays/istio \ | ||||||||||||||||
| sed "s|${CURRENT_IMAGE}:[a-zA-Z0-9_.-]*|${PR_IMAGE}|g" \ | ||||||||||||||||
| kubectl apply -f - | ||||||||||||||||
|
||||||||||||||||
kubectl wait pods -n kubeflow -l app=volumes-web-app --for=condition=Ready --timeout=300s |
Original file line number | Diff line number | Diff line change | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,45 @@ | ||||||||||||||||
name: VWA Multi-Arch Build Test | ||||||||||||||||
on: | ||||||||||||||||
pull_request: | ||||||||||||||||
paths: | ||||||||||||||||
- components/crud-web-apps/volumes/** | ||||||||||||||||
- components/crud-web-apps/common/** | ||||||||||||||||
- releasing/version/VERSION | ||||||||||||||||
branches: | ||||||||||||||||
- main | ||||||||||||||||
- notebooks-v1 | ||||||||||||||||
Comment on lines
+8
to
+10
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Need to also include the "release branch" to trigger this workflow - just like its defined in the original
Suggested change
|
||||||||||||||||
|
||||||||||||||||
concurrency: | ||||||||||||||||
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.number }} | ||||||||||||||||
cancel-in-progress: true | ||||||||||||||||
|
||||||||||||||||
env: | ||||||||||||||||
IMG: ghcr.io/kubeflow/notebooks/volumes-web-app | ||||||||||||||||
PLATFORMS: linux/amd64,linux/ppc64le,linux/arm64/v8 | ||||||||||||||||
|
||||||||||||||||
jobs: | ||||||||||||||||
build: | ||||||||||||||||
runs-on: ubuntu-22.04 | ||||||||||||||||
steps: | ||||||||||||||||
- name: Checkout | ||||||||||||||||
uses: actions/checkout@v4 | ||||||||||||||||
|
||||||||||||||||
- name: Setup QEMU | ||||||||||||||||
uses: docker/setup-qemu-action@v3 | ||||||||||||||||
|
||||||||||||||||
- name: Setup Docker Buildx | ||||||||||||||||
uses: docker/setup-buildx-action@v3 | ||||||||||||||||
|
||||||||||||||||
- name: Build multi-arch images | ||||||||||||||||
uses: docker/build-push-action@v5 | ||||||||||||||||
with: | ||||||||||||||||
context: components/crud-web-apps | ||||||||||||||||
file: components/crud-web-apps/volumes/Dockerfile | ||||||||||||||||
platforms: ${{ env.PLATFORMS }} | ||||||||||||||||
push: false | ||||||||||||||||
load: false | ||||||||||||||||
tags: | | ||||||||||||||||
${{ env.IMG }}:${{ github.sha }} | ||||||||||||||||
${{ env.IMG }}:latest | ||||||||||||||||
cache-from: type=gha | ||||||||||||||||
cache-to: type=gha,mode=max | ||||||||||||||||
Comment on lines
+33
to
+45
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please see related comments I have made on @yehudit1987 's PR that includes a similar
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
#!/bin/bash | ||
|
||
set -euo pipefail | ||
|
||
ISTIO_VERSION="1.17.8" | ||
ISTIO_URL="https://istio.io/downloadIstio" | ||
|
||
echo "Installing Istio ${ISTIO_VERSION} ..." | ||
mkdir istio_tmp | ||
pushd istio_tmp >/dev/null | ||
curl -sL "$ISTIO_URL" | ISTIO_VERSION=${ISTIO_VERSION} sh - | ||
cd istio-${ISTIO_VERSION} | ||
export PATH=$PWD/bin:$PATH | ||
|
||
# Install Istio with default profile | ||
istioctl install --set values.defaultRevision=default -y | ||
|
||
# Wait for Istio control plane to be ready | ||
kubectl wait --for=condition=ready pod -l app=istiod -n istio-system --timeout=300s | ||
|
||
# Verify CRDs are installed | ||
kubectl get crd | grep istio | ||
popd |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
#!/bin/bash | ||
|
||
set -euo pipefail | ||
|
||
KIND_VERSION="0.22.0" | ||
KIND_URL="https://kind.sigs.k8s.io/dl/v${KIND_VERSION}/kind-linux-amd64" | ||
|
||
echo "Setting up kind environment..." | ||
sudo swapoff -a | ||
sudo rm -f /swapfile | ||
sudo mkdir -p /tmp/etcd | ||
sudo mount -t tmpfs tmpfs /tmp/etcd | ||
|
||
echo "Installing kind ${KIND_VERSION} ..." | ||
curl -sL -o kind "$KIND_URL" | ||
chmod +x ./kind | ||
sudo mv kind /usr/local/bin |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
#!/bin/bash | ||
|
||
set -euo pipefail | ||
|
||
KUSTOMIZE_VERSION="5.4.1" | ||
KUSTOMIZE_URL="https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize/v${KUSTOMIZE_VERSION}/kustomize_v${KUSTOMIZE_VERSION}_linux_amd64.tar.gz" | ||
|
||
echo "Installing kustomize ${KUSTOMIZE_VERSION} ..." | ||
curl -sL -o kustomize.tar.gz "$KUSTOMIZE_URL" | ||
tar -xzf kustomize.tar.gz | ||
chmod +x kustomize | ||
sudo mv kustomize /usr/local/bin |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
apiVersion: kind.x-k8s.io/v1alpha4 | ||
kind: Cluster | ||
# Configure registry for KinD. | ||
containerdConfigPatches: | ||
- |- | ||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."$REGISTRY_NAME:$REGISTRY_PORT"] | ||
endpoint = ["http://$REGISTRY_NAME:$REGISTRY_PORT"] | ||
# This is needed in order to support projected volumes with service account tokens. | ||
# See: https://kubernetes.slack.com/archives/CEKK1KTN2/p1600268272383600 | ||
kubeadmConfigPatches: | ||
- | | ||
apiVersion: kubeadm.k8s.io/v1beta3 | ||
kind: ClusterConfiguration | ||
metadata: | ||
name: config | ||
apiServer: | ||
extraArgs: | ||
"service-account-issuer": "kubernetes.default.svc" | ||
"service-account-signing-key-file": "/etc/kubernetes/pki/sa.key" | ||
nodes: | ||
- role: control-plane | ||
image: kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1 | ||
- role: worker | ||
image: kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
latest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to also include the "release branch" to trigger this workflow - just like its defined in the original
kubeflow/kubeflow
source of truth: