add OpenPERouter baremetal E2E deploy-verify lane #83156
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -111,3 +111,86 @@ presubmits: | |
| secret: | ||
| secretName: result-aggregator | ||
| trigger: (?m)^/test( | .* )images,?($|\s.*) | ||
| - agent: kubernetes | ||
| always_run: true | ||
| branches: | ||
| - ^main$ | ||
| - ^main- | ||
| cluster: build05 | ||
| context: ci/prow/openperouter-e2e-metal | ||
| decorate: true | ||
| decoration_config: {} | ||
| labels: | ||
| ci-operator.openshift.io/cloud: equinix-ocp-metal | ||
| ci-operator.openshift.io/cloud-cluster-profile: equinix-ocp-metal | ||
| ci-operator.openshift.io/cluster: build05 | ||
| ci.openshift.io/generator: prowgen | ||
| pj-rehearse.openshift.io/can-be-rehearsed: "true" | ||
| name: pull-ci-openshift-kni-openperouter-main-openperouter-e2e-metal | ||
| path_alias: github.com/openperouter/openperouter | ||
| rerun_command: /test openperouter-e2e-metal | ||
| spec: | ||
| containers: | ||
| - args: | ||
| - --gcs-upload-secret=/secrets/gcs/service-account.json | ||
| - --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson | ||
| - --lease-server-credentials-file=/etc/boskos/credentials | ||
| - --report-credentials-file=/etc/report/credentials | ||
| - --secret-dir=/secrets/ci-pull-credentials | ||
| - --target=openperouter-e2e-metal | ||
| command: | ||
| - ci-operator | ||
| env: | ||
| - name: HTTP_SERVER_IP | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: status.podIP | ||
| image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest | ||
| imagePullPolicy: Always | ||
| name: "" | ||
| ports: | ||
| - containerPort: 8080 | ||
| name: http | ||
| resources: | ||
| requests: | ||
| cpu: 10m | ||
| volumeMounts: | ||
| - mountPath: /etc/boskos | ||
| name: boskos | ||
| readOnly: true | ||
| - mountPath: /secrets/ci-pull-credentials | ||
| name: ci-pull-credentials | ||
| readOnly: true | ||
| - mountPath: /secrets/gcs | ||
| name: gcs-credentials | ||
| readOnly: true | ||
| - mountPath: /secrets/manifest-tool | ||
| name: manifest-tool-local-pusher | ||
| readOnly: true | ||
| - mountPath: /etc/pull-secret | ||
| name: pull-secret | ||
| readOnly: true | ||
| - mountPath: /etc/report | ||
| name: result-aggregator | ||
| readOnly: true | ||
| serviceAccountName: ci-operator | ||
| volumes: | ||
| - name: boskos | ||
| secret: | ||
| items: | ||
| - key: credentials | ||
| path: credentials | ||
| secretName: boskos-credentials | ||
| - name: ci-pull-credentials | ||
| secret: | ||
| secretName: ci-pull-credentials | ||
| - name: manifest-tool-local-pusher | ||
| secret: | ||
| secretName: manifest-tool-local-pusher | ||
| - name: pull-secret | ||
| secret: | ||
| secretName: registry-pull-credentials | ||
| - name: result-aggregator | ||
| secret: | ||
| secretName: result-aggregator | ||
| trigger: (?m)^/test( | .* )openperouter-e2e-metal,?($|\s.*) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| approvers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke | ||
| reviewers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| { | ||
| "path": "baremetalds/openperouter-e2e/baremetalds-openperouter-e2e-workflow.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "fedepaol", | ||
| "maiqueb", | ||
| "oribon", | ||
| "zeeke" | ||
| ], | ||
| "reviewers": [ | ||
| "fedepaol", | ||
| "maiqueb", | ||
| "oribon", | ||
| "zeeke" | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| workflow: | ||
| as: baremetalds-openperouter-e2e | ||
| steps: | ||
| pre: | ||
| - ref: ofcir-acquire | ||
| - ref: ipi-install-rbac | ||
| - ref: baremetalds-devscripts-ibm | ||
| - ref: baremetalds-devscripts-proxy | ||
| - ref: ipi-install-hosted-loki | ||
| - ref: rhcos-conf-osstream | ||
| - ref: baremetalds-openperouter-e2e-iptables | ||
| - ref: baremetalds-devscripts-setup | ||
| test: | ||
| - ref: optional-operators-operator-sdk | ||
| - ref: baremetalds-openperouter-e2e-test | ||
| post: | ||
| - chain: baremetalds-ofcir-post | ||
| documentation: |- | ||
| Deploy OpenPerOuter on a baremetal OCP cluster and verify all components | ||
| come up healthy. Uses firewalld REJECT (tcp-reset) rules on the dev-scripts | ||
| host to keep the bootstrap gather from stalling on routed toswitch IPv6 | ||
| addresses. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| approvers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke | ||
| reviewers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -o nounset | ||
| set -o errexit | ||
| set -o pipefail | ||
|
|
||
| echo "************ openperouter firewalld fix for bootstrap gather ************" | ||
|
|
||
| # shellcheck source=/dev/null | ||
| source "${SHARED_DIR}/packet-conf.sh" | ||
|
|
||
| # The installer's bootstrap gather runs on the bootstrap node and SSHes to every | ||
| # NIC address of every cluster node. The extra-network (toswitch1/toswitch2) | ||
| # addresses are unreachable, and the IPv6 ones in particular stall for ~7min each | ||
| # (TCP half-opens, then the SSH key exchange is reset), adding well over an hour | ||
| # to the gather. bootstrap->toswitch traffic is *routed* between libvirt bridges | ||
| # through the dev-scripts host, so it traverses this host's FORWARD chain. We add | ||
| # REJECT --reject-with tcp-reset rules there so those SYNs get an immediate RST | ||
| # instead of stalling, turning each ~7min stall into an instant failure. | ||
| # | ||
| # Notes: | ||
| # - Rules go in FORWARD (routed node-to-node gather traffic) and OUTPUT | ||
| # (host-originated `openshift-install gather bootstrap` legs). | ||
| # - We use firewalld --permanent rules, NOT raw `iptables -I`. dev-scripts runs | ||
| # `firewall-cmd --reload` during setup, which flushes manually-inserted rules; | ||
| # permanent firewalld rules survive the reload. | ||
| # - Only tcp/22 to the toswitch subnets is rejected; the EVPN/SRv6 fabric uses | ||
| # BGP/BFD/VXLAN, not SSH, so the fabric E2E is unaffected. | ||
| # - Link-local (fe80::) gather attempts are on-link and never routed through this | ||
| # host, so they cannot be intercepted here. | ||
|
|
||
| # Parse subnet vars from DEVSCRIPTS_CONFIG | ||
| eval "${DEVSCRIPTS_CONFIG}" | ||
|
|
||
| ssh "${SSHOPTS[@]}" "root@${IP}" bash -s -- \ | ||
| "${TOSWITCH1_NETWORK_SUBNET_V4}" "${TOSWITCH2_NETWORK_SUBNET_V4}" \ | ||
| "${TOSWITCH1_NETWORK_SUBNET_V6}" "${TOSWITCH2_NETWORK_SUBNET_V6}" << 'EOFFIREWALL' | ||
| set -euo pipefail | ||
|
|
||
| # dev-scripts depends on firewalld; make sure it is running before we add rules. | ||
| systemctl is-active --quiet firewalld || systemctl start firewalld | ||
|
|
||
| # add_reject <ipv4|ipv6> <subnet>: reject tcp/22 to <subnet> on FORWARD + OUTPUT | ||
| add_reject() { | ||
| local family="$1" subnet="$2" chain | ||
| for chain in FORWARD OUTPUT; do | ||
| firewall-cmd --permanent --direct --add-rule "${family}" filter "${chain}" 0 \ | ||
| -p tcp -d "${subnet}" --dport 22 -j REJECT --reject-with tcp-reset | ||
| done | ||
| } | ||
|
|
||
| add_reject ipv4 "$1" | ||
| add_reject ipv4 "$2" | ||
| add_reject ipv6 "$3" | ||
| add_reject ipv6 "$4" | ||
|
|
||
| # Apply permanent -> runtime. | ||
| firewall-cmd --reload | ||
|
|
||
| echo "Added firewalld REJECT(tcp-reset) rules for toswitch subnets on FORWARD+OUTPUT" | ||
| firewall-cmd --direct --get-all-rules | ||
| EOFFIREWALL |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| { | ||
| "path": "baremetalds/openperouter-e2e/iptables/baremetalds-openperouter-e2e-iptables-ref.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "fedepaol", | ||
| "maiqueb", | ||
| "oribon", | ||
| "zeeke" | ||
| ], | ||
| "reviewers": [ | ||
| "fedepaol", | ||
| "maiqueb", | ||
| "oribon", | ||
| "zeeke" | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| ref: | ||
| as: baremetalds-openperouter-e2e-iptables | ||
| from: dev-scripts | ||
| commands: baremetalds-openperouter-e2e-iptables-commands.sh | ||
| timeout: 300s | ||
| env: | ||
| - name: DEVSCRIPTS_CONFIG | ||
| default: "" | ||
| documentation: Dev-scripts config containing TOSWITCH*_NETWORK_SUBNET_V4/V6 vars | ||
| resources: | ||
| requests: | ||
| cpu: 100m | ||
| memory: 200Mi | ||
| documentation: |- | ||
| Add firewalld REJECT (tcp-reset) rules on the dev-scripts host FORWARD and | ||
| OUTPUT chains for tcp/22 to the toswitch1/toswitch2 subnets. The installer's | ||
| bootstrap gather SSHes to every node NIC address; the routed, unreachable | ||
| toswitch IPv6 addresses otherwise stall the gather ~7min each. An immediate | ||
| RST turns each stall into an instant failure. Uses permanent firewalld rules | ||
| so they survive the `firewall-cmd --reload` that dev-scripts runs during | ||
| setup. Link-local (fe80::) attempts are on-link and cannot be handled here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| approvers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke | ||
| reviewers: | ||
| - fedepaol | ||
| - maiqueb | ||
| - oribon | ||
| - zeeke | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since the current PR is not really executing tests, can we deploy the
Dockerfile.openshiftimage? Does it have deployment issues too?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, let's use the openshift dockerfile, so we have a clear signal of the bits that will land downstream.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are using Dockerfile.openshift for the build itself, and that is what is mentioned in the
substitutions(so that the end result is that we do use the openshift build from the given PR).as for using bundle.Dockerfile vs bundle.Dockerfile.openshift (which wrap the olm manifests), bundle.Dockerfile.openshift did some problems, and given that they are practically the same (manifests wise, minus the overlay pins that konflux does that I think aren't related to a given PR) I flipped to bundle.Dockerfile.this is the same as the other projects under telco5g-konflux e.g oran-o2ims and lifecycle-agent, although they don't have bundle.Dockerfile.openshift (but their bundle.Dockerfile is pretty much the same as ours)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand.
I don't like testing a bundle that is not the one that goes to production, but the konflux build for the bundle is so complicated that using the
operator/bundle.Dockerfile.openshiftfile for Prow jobs has a risk of not testing the right code (the PR code).I'm ok on keeping this.
Unfortunately, we can't add a comment to these file