Skip to content

fix: e2e test with kind#6162

Merged
k8s-ci-robot merged 6 commits intokubernetes-sigs:masterfrom
Raffo:fix-e2e
Mar 7, 2026
Merged

fix: e2e test with kind#6162
k8s-ci-robot merged 6 commits intokubernetes-sigs:masterfrom
Raffo:fix-e2e

Conversation

@Raffo
Copy link
Copy Markdown
Contributor

@Raffo Raffo commented Feb 1, 2026

What does it do ?

Closes #6149

Looks like the end to end tested that I pushed in #5933 were so so wrong. I am not sure if I screwed up something upon push or if I really tested them wrong, but nonetheless, they didn't work.

This PR should fix them, I tested breaking it and then restoring it a couple of times, both locally and in action. I still would appreciate a careful review to make sure that things make sense.

Aaand of course they don't pass after many successful passes both locally and in CI. I guess they are flaky as hell. Moving this as draft and going to the drawing board again. Le sigh

I think I fixed it.

More

  • Yes, this PR title follows Conventional Commits
  • Yes, I added unit tests
  • Yes, I updated end user documentation accordingly

@k8s-ci-robot k8s-ci-robot added the scripts Issues or PRs related to internal scripts label Feb 1, 2026
@k8s-ci-robot k8s-ci-robot requested a review from vflaux February 1, 2026 14:14
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 1, 2026
@coveralls
Copy link
Copy Markdown

coveralls commented Feb 1, 2026

Pull Request Test Coverage Report for Build 22314723358

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 554 unchanged lines in 19 files lost coverage.
  • Overall coverage increased (+0.1%) to 79.258%

Files with Coverage Reduction New Missed Lines %
utils.go 2 97.1%
crd.go 2 67.3%
f5_transportserver.go 5 78.65%
openshift_route.go 11 83.59%
ingress.go 12 88.3%
f5_virtualserver.go 13 79.79%
istio_gateway.go 13 88.55%
crypto.go 15 67.9%
pdns/pdns.go 15 71.19%
contour_httpproxy.go 16 84.33%
Totals Coverage Status
Change from base Build 21559764040: 0.1%
Covered Lines: 16041
Relevant Lines: 20239

💛 - Coveralls

@Raffo Raffo marked this pull request as draft February 1, 2026 14:40
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 1, 2026
@Raffo Raffo changed the title fix: e2e test with kind [WIP] fix: e2e test with kind Feb 1, 2026
@Raffo Raffo marked this pull request as ready for review February 2, 2026 11:21
@Raffo Raffo changed the title [WIP] fix: e2e test with kind fix: e2e test with kind Feb 2, 2026
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 2, 2026
@ivankatliarchuk
Copy link
Copy Markdown
Member

It's hard to run this tests in bash. Maybe worth to have a python dsl? I'm not sure where it is still green or actually failed

Testing DNS server with dig (with retries)...
job.batch/dns-server-test-job created
Waiting for DNS server test job to complete...
job.batch/dns-server-test-job condition met
DNS server test job results:
Testing DNS server at 172.18.0.2:5353
=== Testing DNS server with dig (retrying for up to 180s) ===
Attempt 1/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 2/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 3/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 4/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 5/18: Querying externaldns-e2e.external.dns A record
DNS query successful: ;; communications error to 172.18.0.2#5353: timed out
SUCCESS: DNS server test completed successfully
job.batch "dns-server-test-job" deleted from default namespace
End-to-end test completed!

@Raffo
Copy link
Copy Markdown
Contributor Author

Raffo commented Feb 4, 2026

@ivankatliarchuk hard to read you mean? I definitely don’t want a python wrapper, but I agree the output isn’t the best. I think I can make the output better, but running it locally wasn’t an issue for me. Just delete the cluster with kind if you have any, run the script. I do run everything in a vm though, not sure if this complicates people’s local dev, so I’d be eager to hear more from you.

@ivankatliarchuk
Copy link
Copy Markdown
Member

Have a look at the output

DNS query returned empty result, retrying in 10s...
Attempt 5/18: Querying externaldns-e2e.external.dns A record
DNS query successful: ;; communications error to 172.18.0.2#5353: timed out <<<< FAIL
SUCCESS: DNS server test completed successfully <<<  SUCCESS
job.batch "dns-server-test-job" deleted from default n

Bash with log regexes can work for individual tests, but this approach doesn’t scale well if we decide to add more tests later.

@Raffo
Copy link
Copy Markdown
Contributor Author

Raffo commented Feb 5, 2026

Yeah I get that but I took this approach exactly because I didn’t want to write a mega wrapper in go like I did for the end to end tests that I have in my private repo and that for reasons I didn’t want to opensource. Do you think it would make sense to get this in at least to have the tests fixed (even with the crappy output) and then work in another pr to convert them in go?

@ivankatliarchuk
Copy link
Copy Markdown
Member

For sure we should fix the test first.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 23, 2026
@Raffo
Copy link
Copy Markdown
Contributor Author

Raffo commented Feb 26, 2026

@ivankatliarchuk I think this works? Is it terrible?

@ivankatliarchuk
Copy link
Copy Markdown
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 26, 2026
@ivankatliarchuk
Copy link
Copy Markdown
Member

ivankatliarchuk commented Feb 26, 2026

Results, seems like working. I was initially confused, that is trying so many attempts. But this is expected due to DNS propagation

Testing DNS server functionality...
Node IP: 172.18.0.2
Testing DNS server with dig (with retries)...
job.batch/dns-server-test-job created
Waiting for DNS server test job to complete...
job.batch/dns-server-test-job condition met
DNS server test job results:
Testing DNS server at 172.18.0.2:5353
=== Testing DNS server with dig (retrying for up to 180s) ===
Attempt 1/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 2/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 3/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 4/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 5/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 6/18: Querying externaldns-e2e.external.dns A record
DNS query returned empty result, retrying in 10s...
Attempt 7/18: Querying externaldns-e2e.external.dns A record
DNS query successful: 10.244.0.7

@ivankatliarchuk
Copy link
Copy Markdown
Member

ivankatliarchuk commented Feb 26, 2026

DNS is working, is just takes some time to propagate to a node

ATTEMPT=1
while [ \$ATTEMPT -le \$MAX_ATTEMPTS ]; do
echo "Attempt \$ATTEMPT/\$MAX_ATTEMPTS: Querying externaldns-e2e.external.dns A record"
RESULT=\$(dig @$NODE_IP -p 5353 externaldns-e2e.external.dns A +short +timeout=5 2>/dev/null)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe something like

dig @coredns.default.svc.cluster.local

If we are using coredns

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could validate, where the records added to etcd, something like

kubectl exec -it etcd-0 -- etcdctl get /skydns/dns/external --prefix --keys-only

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how this would add? Maybe you say it to reduce possible flakyness? I did it just with dig originally to make a test that was as close as possible to users 🤔

Copy link
Copy Markdown
Member

@ivankatliarchuk ivankatliarchuk Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, not a big difference. Is goot too go as is.

For dig @coredns.default.svc.cluster.local (service DNS) the only difference with NODE_IP

  • Tests the actual path -> reflects how real workloads inside the cluster resolve DNS, which is what external-dns is supposed to serve

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok cool, I think I understand your point of view now, but the change would also add a bit of complexity. I’ll merge like this, so we have working tests again.

@Raffo
Copy link
Copy Markdown
Contributor Author

Raffo commented Mar 7, 2026

/approve

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Raffo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 7, 2026
@k8s-ci-robot k8s-ci-robot merged commit 0507bb5 into kubernetes-sigs:master Mar 7, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. scripts Issues or PRs related to internal scripts size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

end to end tests sometime reported as successful even though they actually failed

4 participants