Skip to content

UDN workloads improvements#399

Merged
rsevilla87 merged 6 commits into
kube-burner:mainfrom
rsevilla87:refactor-udn-workloads
Mar 12, 2026
Merged

UDN workloads improvements#399
rsevilla87 merged 6 commits into
kube-burner:mainfrom
rsevilla87:refactor-udn-workloads

Conversation

@rsevilla87

Copy link
Copy Markdown
Member

Type of change

  • Refactor
  • Optimization

Description

This refactoring improves the reliability of UDN workload testing by ensuring proper network setup validation and standardizing configuration patterns across different UDN workload types.

  • Configuration cleanup: Removed duplicate preLoadPeriod, jobPause, and namespace label configurations across UDN workload files
  • Improved wait logic: Added custom status path validation for NetworkAttachmentDefinitionCreated and NetworkCreated conditions to ensure proper UDN setup
  • File reorganization: Renamed virt-density-udn.go → virt-udn-density.go for consistent naming convention
  • Enhanced documentation: Updated command descriptions and flag help text for better clarity (e.g., "One UDN will be created per iteration")
  • Enhanced wait conditions for network attachment definitions with custom status path validation rathern than an arbitrary pause
  • Fixed duplicate flag: Removed duplicate --iteration flag in virt-udn-density workload

Related Tickets & Documents

  • Related Issue #
  • Closes #

@rsevilla87 rsevilla87 requested review from a team as code owners February 19, 2026 10:39
@rsevilla87 rsevilla87 force-pushed the refactor-udn-workloads branch 3 times, most recently from 9b9e5ff to 452d8e1 Compare February 19, 2026 10:47
- objectTemplate: udn_l2.yml
replicas: 1
{{ end }}
waitOptions:

@rsevilla87 rsevilla87 Feb 23, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @mohit-sheth, this check is not valid to verify that a UDN is ready

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rsevilla87 @mohit-sheth @jtaleric I thik you can use NetworkAllocationSucceeded instead of NetworkCreated. In my 72 CUDNs, I have observed, I have observed some UDNs taking 30 seconds to report NetworkAllocationSucceeded status, check this gist https://gist.github.com/venkataanil/9a046c3e86400a7a4248fd3b01c32fb5

Here P99 podreadylatency is closely matching with that
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: Initialized 99th: 1000 ms max: 1000 ms avg: 16 ms" file="base_measurement.go:127"
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: Ready 99th: 34000 ms max: 36000 ms avg: 23252 ms" file="base_measurement.go:127"
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: PodReadyToStartContainers 99th: 34000 ms max: 36000 ms avg: 23252 ms" file="base_measurement.go:127"
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: ContainersStarted 99th: 34404 ms max: 36086 ms avg: 23311 ms" file="base_measurement.go:127"
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: PodScheduled 99th: 469 ms max: 1000 ms avg: 12 ms" file="base_measurement.go:127"
time="2026-02-23 19:00:18" level=info msg="udn-density-l3-pods: ContainersReady 99th: 34000 ms max: 36000 ms avg: 23252 ms" file="base_measurement.go:127"

So I think we can use NetworkAllocationSucceeded I assume this means OVNK has done with creating reasources in NBDB. At least this can be a confimation for us to proceed creating pods though OVS flows might not be programmed.

I have also tried a go program (https://gist.github.com/venkataanil/ddc254653d95863d1d60bb0dc827212a which is generated with AI assistnace) which watches for logical switches and routers in all nodes NBDBs for the corresponding UDN, during this testing, and observed similar latency of 34 sec, see this output of the go program run https://gist.github.com/venkataanil/d5d513be46d6377f7a8751f669ce2490

To run this go program, you need to run
oc get pods -n openshift-ovn-kubernetes -l app=ovnkube-node -o name |
xargs -I{} oc exec -n openshift-ovn-kubernetes -c nbdb {} --
ovn-nbctl set-connection ptcp:6641:0.0.0.0 punix:/var/run/ovn/ovnnb_db.sock

and then run the go program like
go run ./testudnstatus2.go --kubeconfig=/root/mno/kubeconfig

Comment thread cmd/config/udn-density-pods/udn-density-pods.yml Outdated
Signed-off-by: Raul Sevilla <rsevilla@redhat.com>
Signed-off-by: Raul Sevilla <rsevilla@redhat.com>
Signed-off-by: Raul Sevilla <rsevilla@redhat.com>
Signed-off-by: Raul Sevilla <rsevilla@redhat.com>
Signed-off-by: Raul Sevilla <rsevilla@redhat.com>
@rsevilla87 rsevilla87 force-pushed the refactor-udn-workloads branch from e4cfeb1 to bb339aa Compare March 11, 2026 12:31

@jtaleric jtaleric left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
tested locally..

@rsevilla87 rsevilla87 merged commit 0bb000e into kube-burner:main Mar 12, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants