Skip to content

MGMT-18579: Inject nmpolicy captures into the provided YAML and place it with INI file under the host-specific path - #6695

Merged
openshift-merge-bot[bot] merged 5 commits into
openshift:masterfrom
linoyaslan:MGMT-18579.nmpolicy_nmstate_service_replace_pre-generated_nmconnection_files.linoy
Sep 17, 2024
Merged

openshift-merge-bot[bot] merged 5 commits into
openshift:masterfrom
linoyaslan:MGMT-18579.nmpolicy_nmstate_service_replace_pre-generated_nmconnection_files.linoy

Conversation

@linoyaslan

Copy link
Copy Markdown
Contributor

The purpose of this PR is to change the current method of creating nmconnection files from the nmstate YAML provided by users. Currently, we rely on pre-generated nmconnection files and a complex script running on the node to replace temporary interface names with actual ones. These modifications aim to simplify the process by using nmpolicy and the nmstate service, offering a more straightforward approach to generating nmconnection files. For more detailed information, please refer to the documentation here - Doc

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 23, 2024
@linoyaslan
linoyaslan marked this pull request as draft August 23, 2024 13:15
@openshift-ci openshift-ci Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 23, 2024
@openshift-ci
openshift-ci Bot requested review from eranco74 and gamli75 August 23, 2024 13:15
@openshift-ci openshift-ci Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 23, 2024
@linoyaslan

Copy link
Copy Markdown
Contributor Author

Comment thread internal/bminventory/inventory.go Outdated
Comment thread internal/bminventory/inventory.go Outdated
Comment thread internal/constants/files.go Outdated
Comment thread internal/constants/script-with-nmstatectl.go Outdated
Comment thread internal/ignition/discovery.go Outdated
Comment thread internal/ignition/templates/discovery.ign Outdated
@linoyaslan
linoyaslan force-pushed the MGMT-18579.nmpolicy_nmstate_service_replace_pre-generated_nmconnection_files.linoy branch 3 times, most recently from 7446faf to 81c53bb Compare August 28, 2024 18:48
@openshift-ci openshift-ci Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 28, 2024
@linoyaslan
linoyaslan force-pushed the MGMT-18579.nmpolicy_nmstate_service_replace_pre-generated_nmconnection_files.linoy branch 5 times, most recently from 906f5df to d7623d4 Compare August 28, 2024 19:40
@linoyaslan
linoyaslan requested a review from carbonin August 28, 2024 19:41
@openshift-ci openshift-ci Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 29, 2024
Comment thread internal/bminventory/inventory.go Outdated
@linoyaslan

Copy link
Copy Markdown
Contributor Author

/retest

@linoyaslan

Copy link
Copy Markdown
Contributor Author

/test edge-e2e-metal-assisted-static-ip-suite

Comment thread internal/bminventory/inventory.go Outdated
Comment thread internal/isoeditor/rhcos_test.go Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does fetching the binary from the rootfs have to do with the max version?
It seems to me that we would support versions over 4.17 regardless of which approach we use.

The arch is what matters for the rootfs approach or not.

I just worry that we're going to test out a 4.18 release before this gets updated and things won't work as we expect and I don't see a reason to let that happen.

@linoyaslan

Copy link
Copy Markdown
Contributor Author

/test ?

@openshift-ci

openshift-ci Bot commented Sep 12, 2024

Copy link
Copy Markdown
Contributor

@linoyaslan: The following commands are available to trigger required jobs:

  • /test e2e-agent-compact-ipv4
  • /test edge-assisted-operator-catalog-publish-verify
  • /test edge-ci-index
  • /test edge-e2e-ai-operator-ztp
  • /test edge-e2e-ai-operator-ztp-sno-day2-workers
  • /test edge-e2e-ai-operator-ztp-sno-day2-workers-late-binding
  • /test edge-e2e-metal-assisted
  • /test edge-e2e-metal-assisted-4-12
  • /test edge-e2e-metal-assisted-cnv-4-16
  • /test edge-e2e-metal-assisted-lvm
  • /test edge-e2e-metal-assisted-odf-4-16
  • /test edge-images
  • /test edge-lint
  • /test edge-subsystem-aws
  • /test edge-subsystem-kubeapi-aws
  • /test edge-unit-test
  • /test edge-verify-generated-code
  • /test images
  • /test mce-images

The following commands are available to trigger optional jobs:

  • /test e2e-agent-ha-dualstack
  • /test e2e-agent-sno-ipv6
  • /test edge-e2e-ai-operator-disconnected-capi
  • /test edge-e2e-ai-operator-ztp-3masters
  • /test edge-e2e-ai-operator-ztp-capi
  • /test edge-e2e-ai-operator-ztp-compact-day2-masters
  • /test edge-e2e-ai-operator-ztp-compact-day2-workers
  • /test edge-e2e-ai-operator-ztp-disconnected
  • /test edge-e2e-ai-operator-ztp-hypershift-zero-nodes
  • /test edge-e2e-ai-operator-ztp-multiarch-3masters-ocp
  • /test edge-e2e-ai-operator-ztp-multiarch-sno-ocp
  • /test edge-e2e-ai-operator-ztp-node-labels
  • /test edge-e2e-ai-operator-ztp-remove-node
  • /test edge-e2e-ai-operator-ztp-sno-day2-masters
  • /test edge-e2e-ai-operator-ztp-sno-day2-workers-ignitionoverride
  • /test edge-e2e-metal-assisted-4-13
  • /test edge-e2e-metal-assisted-4-14
  • /test edge-e2e-metal-assisted-4-15
  • /test edge-e2e-metal-assisted-4-16
  • /test edge-e2e-metal-assisted-bond
  • /test edge-e2e-metal-assisted-bond-4-12
  • /test edge-e2e-metal-assisted-bond-4-13
  • /test edge-e2e-metal-assisted-bond-4-14
  • /test edge-e2e-metal-assisted-bond-4-15
  • /test edge-e2e-metal-assisted-bond-4-16
  • /test edge-e2e-metal-assisted-day2
  • /test edge-e2e-metal-assisted-day2-arm-workers
  • /test edge-e2e-metal-assisted-day2-single-node
  • /test edge-e2e-metal-assisted-external
  • /test edge-e2e-metal-assisted-external-4-14
  • /test edge-e2e-metal-assisted-ipv4v6
  • /test edge-e2e-metal-assisted-ipv6
  • /test edge-e2e-metal-assisted-kube-api-late-binding-single-node
  • /test edge-e2e-metal-assisted-kube-api-late-unbinding-ipv4-single-node
  • /test edge-e2e-metal-assisted-kube-api-net-suite
  • /test edge-e2e-metal-assisted-mce-4-16
  • /test edge-e2e-metal-assisted-mce-sno-4-16
  • /test edge-e2e-metal-assisted-metallb
  • /test edge-e2e-metal-assisted-none
  • /test edge-e2e-metal-assisted-onprem
  • /test edge-e2e-metal-assisted-single-node
  • /test edge-e2e-metal-assisted-static-ip-suite
  • /test edge-e2e-metal-assisted-static-ip-suite-4-12
  • /test edge-e2e-metal-assisted-static-ip-suite-4-13
  • /test edge-e2e-metal-assisted-static-ip-suite-4-14
  • /test edge-e2e-metal-assisted-static-ip-suite-4-15
  • /test edge-e2e-metal-assisted-static-ip-suite-4-16
  • /test edge-e2e-metal-assisted-tang
  • /test edge-e2e-metal-assisted-tpmv2
  • /test edge-e2e-metal-assisted-upgrade-agent
  • /test edge-e2e-nutanix-assisted
  • /test edge-e2e-nutanix-assisted-2workers
  • /test edge-e2e-nutanix-assisted-4-14
  • /test edge-e2e-oci-assisted
  • /test edge-e2e-oci-assisted-4-14
  • /test edge-e2e-oci-assisted-iscsi
  • /test edge-e2e-vsphere-assisted
  • /test edge-e2e-vsphere-assisted-4-14
  • /test edge-e2e-vsphere-assisted-4-15
  • /test edge-e2e-vsphere-assisted-4-16
  • /test edge-e2e-vsphere-assisted-umn
  • /test okd-scos-images
  • /test push-pr-image

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-openshift-assisted-service-master-e2e-agent-compact-ipv4
  • pull-ci-openshift-assisted-service-master-edge-ci-index
  • pull-ci-openshift-assisted-service-master-edge-e2e-ai-operator-ztp
  • pull-ci-openshift-assisted-service-master-edge-e2e-metal-assisted
  • pull-ci-openshift-assisted-service-master-edge-images
  • pull-ci-openshift-assisted-service-master-edge-lint
  • pull-ci-openshift-assisted-service-master-edge-subsystem-aws
  • pull-ci-openshift-assisted-service-master-edge-subsystem-kubeapi-aws
  • pull-ci-openshift-assisted-service-master-edge-unit-test
  • pull-ci-openshift-assisted-service-master-edge-verify-generated-code
  • pull-ci-openshift-assisted-service-master-images
  • pull-ci-openshift-assisted-service-master-mce-images
Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@linoyaslan

Copy link
Copy Markdown
Contributor Author

/test edge-e2e-metal-assisted-static-ip-suite
/test edge-e2e-metal-assisted-bond

@linoyaslan

Copy link
Copy Markdown
Contributor Author

/hold

@carbonin

Copy link
Copy Markdown
Member

/test edge-e2e-metal-assisted-static-ip-suite

@carbonin carbonin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Unhold when you've got all the feedback from others you want.

@openshift-ci

openshift-ci Bot commented Sep 12, 2024

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: carbonin, linoyaslan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Comment thread internal/bminventory/inventory.go Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange to me that we'd maintain two different ways of validating the same input data, based on what transformation we are applying to it before putting it in the ISO.

Validating the input directly involves a lot more .(map[interface{}]interface{}) but ultimately validates the same thing as in the keyfiles that come out of nmstatectl gc. I can't see a reason to maintain two implementations.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I'll keep the validation directly on the YAML since we don't want our code handling both YAML and nmconnection formats. The whole point of using nmstate is that we don't need to worry about its internal translation. Right now, we're using the generator for validation, but we're aware that the nmstate team is working on Go structures and adding validation, so that aspect will change in a later phase.

Comment thread pkg/staticnetworkconfig/generator.go Outdated
Comment thread pkg/staticnetworkconfig/generator.go Outdated
Comment thread pkg/staticnetworkconfig/generator.go Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to ignore the missing name when the interface name appears as an interface in the NMState itself.

@AlonaKaplan AlonaKaplan Sep 14, 2024 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that the following won't work when using nmstate.service in case the MAC map doesn't contain the ports.

 ---
 interfaces:
 - name: eth0
   type: ethernet
   state: up
   identifier: mac-address
   mac-address: 00:23:45:67:89:1a
 - name: eth1
   type: ethernet
   state: up
   identifier: mac-address
   mac-address: 00:23:45:67:89:1b
 - name: bond0
   type: bond
   state: up
   link-aggregation:
     mode: balance-rr
     port:
       - eth0
       - eth1

If it is indeed the case, the bond validation should remain as is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlonaKaplan As discussed offline, the above example indeed doesn't work with nmstate, but it also fails with a VerificationError when the ports are physical interface names. I've reached out to Gris to check if this is a known issue with nmstate.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gris said here that it would work for nmstatectl apply. Is there something different about nmstatectl service that would make it not work there? It seems to me like an issue with nmstate if this doesn't work. Forbidding this config would certainly be a regression for ABI.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cathay4t would the above config work using nmstate apply?

@linoyaslan linoyaslan Sep 16, 2024 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zaneb I’ve added a temporary condition to prevent regressions for ABI while blocking non-ABI users from using the mac-identifier

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YAML at #6695 (comment) will not works as expected, the bond still refer the bond port by name eth1 and eth2.

You may use interface.controller property for now:

---
interfaces:
  - name: bond0
    type: bond
    link-aggregation:
      mode: balance-rr
  - name: bond0-port1
    type: ethernet
    state: up
    identifier: mac-address
    mac-address: 00:23:45:67:89:1a
    controller: bond0

Another approach would be wait nmstate/nmstate#2710

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zaneb as Gris approved, the yaml is not supported by nmstate and not the current/planned API to refer bond ports with mac-identifier.
I understand it works with nmstate generator, but it is kind of a hack.
I think both ABI and AI should be aligned with the official nmstate API and not support this format.

Comment thread pkg/staticnetworkconfig/generator.go Outdated
Comment thread pkg/staticnetworkconfig/generator.go Outdated
Comment thread pkg/staticnetworkconfig/generator.go Outdated
@openshift-ci

openshift-ci Bot commented Sep 16, 2024 •

Copy link
Copy Markdown
Contributor

@linoyaslan: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/edge-e2e-metal-assisted-static-ip-suite-4-15 5f0cd9da09d94953875ada27d39a19864efdac31 link false /test edge-e2e-metal-assisted-static-ip-suite-4-15
ci/prow/edge-e2e-metal-assisted-bond-4-15 5f0cd9da09d94953875ada27d39a19864efdac31 link false /test edge-e2e-metal-assisted-bond-4-15
ci/prow/edge-e2e-metal-assisted-static-ip-suite-4-16 5f0cd9da09d94953875ada27d39a19864efdac31 link false /test edge-e2e-metal-assisted-static-ip-suite-4-16
ci/prow/edge-e2e-metal-assisted-bond-4-16 5f0cd9da09d94953875ada27d39a19864efdac31 link false /test edge-e2e-metal-assisted-bond-4-16

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Comment thread pkg/staticnetworkconfig/generator.go Outdated
@zaneb

zaneb commented Sep 16, 2024

Copy link
Copy Markdown
Member

As far as I can see this should be OK for ABI now.

Inject nmpolicy captures into the provided YAML and place it with INI
file under the host-specific path
Script for the new flow using nmstate service, along with the service configurations to execute the script for both minimal and full ISO
Modify the initrd in minimal ISO to include nmpolicy files along with the script and relevant service, while maintaining backward compatibility with the current flow for versions earlier than 4.13
Modify static network flow for full ISO along with maintaining backward compatibility with the current flow for versions earlier than 4.13
@linoyaslan

Copy link
Copy Markdown
Contributor Author

/test edge-e2e-metal-assisted-bond
/test edge-e2e-metal-assisted-static-ip-suite

@danielerez

Copy link
Copy Markdown
Contributor

/lgtm

@linoyaslan

Copy link
Copy Markdown
Contributor Author

/unhold

@linoyaslan

linoyaslan commented Sep 17, 2024 •

Copy link
Copy Markdown
Contributor Author

/retitle MGMT-18579: Inject nmpolicy captures into the provided YAML and place it with INI file under the host-specific path

@openshift-ci-robot

openshift-ci-robot commented Sep 17, 2024 •

Copy link
Copy Markdown

@linoyaslan: This pull request references MGMT-18579 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.18.0" version, but no target version was set.

Details

In response to this:

The purpose of this PR is to change the current method of creating nmconnection files from the nmstate YAML provided by users. Currently, we rely on pre-generated nmconnection files and a complex script running on the node to replace temporary interface names with actual ones. These modifications aim to simplify the process by using nmpolicy and the nmstate service, offering a more straightforward approach to generating nmconnection files. For more detailed information, please refer to the documentation here - Doc

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-bot

Copy link
Copy Markdown
Contributor

[ART PR BUILD NOTIFIER]

Distgit: ose-agent-installer-api-server
This PR has been included in build ose-agent-installer-api-server-container-v4.18.0-202409171339.p0.g7402f8f.assembly.stream.el9.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants