[Test] Split E2E nightly operator tests into RayCluster/GCS and RayJob runners #3932

Future-Outlier · 2025-08-10T04:15:09Z

Why are these changes needed?

We have recently noticed that the E2E nightly operator tests in CI have become flaky.
https://buildkite.com/ray-project/ray-ecosystem-ci-kuberay-ci/builds/10289#01988d2f-1b10-46c7-825e-826f7c895dfb

It appears that the “Test E2E (nightly operator)” step takes more than 30 minutes, causing it to time out.

To address this, we can split the nightly operator tests into two runners:
• Runner 1: RayCluster / GCS tests
• Runner 2: RayJob tests
The execution times for these two groups are similar, and splitting them will make debugging easier in the future.

Related issue number

#3930

Checks

I've made sure the tests are passing.
Testing Strategy
- Unit tests
- Manual tests
- This PR is not tested :(

…b runners Signed-off-by: Future-Outlier <[email protected]>

Signed-off-by: Future-Outlier <[email protected]>

machichima · 2025-08-10T06:28:33Z

.buildkite/test-e2e.yml

@@ -1,4 +1,4 @@
- label: 'Test E2E (nightly operator)'
+- label: 'Test E2E (nightly operator) - RayCluster & GCS'


Suggested change

- label: 'Test E2E (nightly operator) - RayCluster & GCS'

- label: 'Test RayCluster and GCS E2E (nightly operator)'

nit: align naming format with others

machichima · 2025-08-10T06:29:43Z

.buildkite/test-e2e.yml

-    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
-    - echo "--- END:e2e (nightly operator) tests finished"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayCluster|TestGcs" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
+    - echo "--- END:e2e (nightly operator) RayCluster & GCS tests finished"


RayCluster and GCS E2E

Suggested change

- echo "--- END:e2e (nightly operator) RayCluster & GCS tests finished"

- echo "--- END:RayCluster and GCS E2E (nightly operator) tests finished"

nit

Do we need to also align naming format under other e2e tests in the same file?
For example,

kuberay/.buildkite/test-e2e.yml

Line 33 in d1e750d

- echo "--- START:Running e2e rayservice (nightly operator) tests"

WDYT, @machichima @Future-Outlier

Good question, I think we can open a follow-up PR after this PR is merged

machichima · 2025-08-10T06:30:13Z

.buildkite/test-e2e.yml

+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayCluster|TestGcs" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
+    - echo "--- END:e2e (nightly operator) RayCluster & GCS tests finished"
+
+- label: 'Test E2E (nightly operator) - RayJob'


Suggested change

- label: 'Test E2E (nightly operator) - RayJob'

- label: 'Test RayJob E2E (nightly operator)'

nit: align naming format with others

machichima · 2025-08-10T06:30:46Z

.buildkite/test-e2e.yml

+    - mkdir -p "$(pwd)/tmp" && export KUBERAY_TEST_OUTPUT_DIR=$(pwd)/tmp
+    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
+    - echo "--- END:e2e (nightly operator) RayJob tests finished"


Suggested change

- echo "--- END:e2e (nightly operator) RayJob tests finished"

- echo "--- END:RayJob E2E (nightly operator) tests finished"

nit

owenowenisme · 2025-08-10T07:49:58Z

.buildkite/test-e2e.yml

+    - set -o pipefail
+    - mkdir -p "$(pwd)/tmp" && export KUBERAY_TEST_OUTPUT_DIR=$(pwd)/tmp
+    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)


Instead of using go test -timeout 30m -v ./test/e2e -run "TestRayJob" (run flag), I think it would be better to split rayJob tests into new folder e.g. e2e-rayjob

Thank you, @owenowenisme!
I think this is a great idea, and the current structure also works well. Perhaps we could leave it to the maintainer or committer to decide?

cc @kevin85421 @rueian @MortalHappiness

Thanks, @Future-Outlier ！

The reasons that I suggested split into files is that maybe not all tests in the future follows the convention of testRayJobxxx.

And also, rayServiceE2E and other e2e test are splited into files, so we might want to follow this pattern?

I tried this solution, but I found that I need to make some changes to this file:
https://github.com/ray-project/kuberay/blob/master/ray-operator/test/e2e/support.go
Maybe we can address this in a separate PR?

update: I split rayJob tests into new folder e2erayjob, thank you all!

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: machichima <[email protected]>

win5923

Thanks!

400Ping

LGTM!

fscnick · 2025-08-10T15:42:25Z

.buildkite/test-e2e.yml

    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
-    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
-    - echo "--- END:e2e (nightly operator) tests finished"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayCluster|TestGcs" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)


nit: it could have a more solid match.

Suggested change

- KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayCluster|TestGcs" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)

- KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "＾TestRayCluster|＾TestGcs" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)

solid review, thank you!

fscnick · 2025-08-10T15:43:14Z

.buildkite/test-e2e.yml

+    - set -o pipefail
+    - mkdir -p "$(pwd)/tmp" && export KUBERAY_TEST_OUTPUT_DIR=$(pwd)/tmp
+    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)


nit: ditto

Suggested change

- KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)

- KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "^TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)

solid review, thank you!

machichima

LGTM

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: fscnick <[email protected]>

Signed-off-by: Future-Outlier <[email protected]>

rueian · 2025-08-11T05:26:13Z

.buildkite/test-e2e.yml

+    - set -o pipefail
+    - mkdir -p "$(pwd)/tmp" && export KUBERAY_TEST_OUTPUT_DIR=$(pwd)/tmp
+    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -run "^TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)


Filtering both ^TestRayCluster|^TestGcs and ^TestRayJob could be risky, as it may silently ignore some tests that do not match the filters in the future. Or do we have a mechanism to force developers to use these prefixes? Otherwise, we'd better either split the test packages or use filters like ^TestRayJob and ^(?!TestRayJob) to cover all tests.

I prefer to use filters like ^TestRayJob and ^(?!TestRayJob) to include or exclude relevant tests in this PR.
Once this PR is merged, I can open an issue to split the test packages. Does this look good to you?

Go (Golang) use Google's RE2 regular expression engine, and it does not support negative lookahead or lookbehind in its regular expressions.
google/re2#156

So I use this go test -timeout 30m -v ./test/e2e -skip "^TestRayJob" in label Test RayCluster and GCS E2E (nightly operator) instead in this PR.

BTW, thank you for the review!

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: Rueian <[email protected]>

kevin85421 · 2025-08-11T18:04:21Z

.buildkite/test-e2e.yml

    - echo "KUBERAY_TEST_OUTPUT_DIR=$$KUBERAY_TEST_OUTPUT_DIR"
-    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)
-    - echo "--- END:e2e (nightly operator) tests finished"
+    - KUBERAY_TEST_TIMEOUT_SHORT=1m KUBERAY_TEST_TIMEOUT_MEDIUM=5m KUBERAY_TEST_TIMEOUT_LONG=10m go test -timeout 30m -v ./test/e2e -skip "^TestRayJob" 2>&1 | awk -f ../.buildkite/format.awk | tee $$KUBERAY_TEST_OUTPUT_DIR/gotest.log || (kubectl logs --tail -1 -l app.kubernetes.io/name=kuberay | tee $$KUBERAY_TEST_OUTPUT_DIR/kuberay-operator.log && cd $$KUBERAY_TEST_OUTPUT_DIR && find . -name "*.log" | tar -cf /artifact-mount/e2e-log.tar -T - && exit 1)


go test -timeout 30m -v ./test/e2e -skip "^TestRayJob"

This is pretty hacky. Can you create a new directory e2erayjob instead?

no problem, just updated, thank you!

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: kaihsun <[email protected]>

kenchung285

LGTM

kevin85421

Please avoid defining the same functions multiple times in the codebase. We can add some util functions which are used in multiple Golang packages to https://github.com/ray-project/kuberay/tree/master/ray-operator/test/support.

Note that I didn't list all of them.

kevin85421 · 2025-08-12T17:03:50Z

ray-operator/test/e2e/support.go


-func newRayClusterSpec(options ...option[rayv1ac.RayClusterSpecApplyConfiguration]) *rayv1ac.RayClusterSpecApplyConfiguration {
-	return rayClusterSpecWith(rayClusterSpec(), options...)
+func newRayClusterSpec() *rayv1ac.RayClusterSpecApplyConfiguration {


This function is used in both e2e/ and e2erayjob/. Could you avoid defining it multiple times?

kevin85421 · 2025-08-12T17:07:56Z

ray-operator/test/e2erayjob/support.go

+					}))))
+}
+
+func workerPodTemplateApplyConfiguration() *corev1ac.PodTemplateSpecApplyConfiguration {


This function is used in both e2e/ and e2erayjob/. Could you avoid defining it multiple times?

kevin85421 · 2025-08-12T17:09:05Z

ray-operator/test/e2erayjob/support.go

+	return apply(template, options...)
+}
+
+func headPodTemplateApplyConfiguration() *corev1ac.PodTemplateSpecApplyConfiguration {


This function is used in both e2e/ and e2erayjob/. Could you avoid defining it multiple times?

Signed-off-by: Future-Outlier <[email protected]>

…rt readfile Signed-off-by: Future-Outlier <[email protected]>

kevin85421

Overall LGTM

kevin85421 · 2025-08-13T20:23:35Z

ray-operator/test/e2erayservice/support.go

 //go:embed *.py
 var _files embed.FS

-func ReadFile(t Test, fileName string) []byte {


Please replace it with the upper-level ReadFile.

kevin85421 · 2025-08-13T20:24:32Z

ray-operator/test/e2eautoscaler/support.go

 //go:embed *.py
 var _files embed.FS

-func ReadFile(t Test, fileName string) []byte {


Please replace it with the upper-level ReadFile.

The initial issue has been fixed, but there are still some duplicated functions. I’ve opened a follow-up issue for additional cleanup: #3947

Thank you for the advice!

Signed-off-by: Future-Outlier <[email protected]>

…b runners (ray-project#3932) Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: machichima <[email protected]> Co-authored-by: fscnick <[email protected]> Co-authored-by: Rueian <[email protected]> Co-authored-by: kaihsun <[email protected]>

…e2erayservice/support.go by reusing test/support/support.go implementations to improve maintainability and reduce redundancy. Related to ray-project#3932 Signed-off-by: HSIU-CHI LIU (Tomlord) <[email protected]>

…e2erayservice/support.go by reusing test/support/support.go implementations to improve maintainability and reduce redundancy. Related to #3932 (#4038) Signed-off-by: HSIU-CHI LIU (Tomlord) <[email protected]> Signed-off-by: Hsiu-Chi Liu (Tomlord) <[email protected]>

Future-Outlier added 2 commits August 10, 2025 12:11

[Test] Split E2E nightly operator tests into RayCluster/GCS and RayJo…

9130246

…b runners Signed-off-by: Future-Outlier <[email protected]>

update

e20a0ab

Signed-off-by: Future-Outlier <[email protected]>

machichima reviewed Aug 10, 2025

View reviewed changes

owenowenisme reviewed Aug 10, 2025

View reviewed changes

update Nary's advice

ab0c36a

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: machichima <[email protected]>

win5923 approved these changes Aug 10, 2025

View reviewed changes

400Ping approved these changes Aug 10, 2025

View reviewed changes

fscnick approved these changes Aug 10, 2025

View reviewed changes

machichima approved these changes Aug 10, 2025

View reviewed changes

Future-Outlier and others added 2 commits August 11, 2025 08:32

update nick's advice

f795518

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: fscnick <[email protected]>

Trigger CI

7ab0743

Signed-off-by: Future-Outlier <[email protected]>

rueian reviewed Aug 11, 2025

View reviewed changes

Update RueiAn's advice

611ab23

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: Rueian <[email protected]>

kevin85421 requested changes Aug 11, 2025

View reviewed changes

Future-Outlier and others added 2 commits August 12, 2025 09:23

Merge branch 'master' into kuberay-flaky-test-e2e-nightly-operator-fix

9445225

update kai-hsun's advice

333aea9

Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: kaihsun <[email protected]>

kenchung285 approved these changes Aug 12, 2025

View reviewed changes

kevin85421 reviewed Aug 12, 2025

View reviewed changes

Future-Outlier added 2 commits August 13, 2025 20:26

Update Kai-Hsun's advice

cdede74

Signed-off-by: Future-Outlier <[email protected]>

turn ReadFile to readFile in e2eautoscaler and e2erayservice to suppo…

9440824

…rt readfile Signed-off-by: Future-Outlier <[email protected]>

Future-Outlier requested a review from kevin85421 August 13, 2025 13:37

kevin85421 reviewed Aug 13, 2025

View reviewed changes

Future-Outlier mentioned this pull request Aug 14, 2025

Remove duplicated functions in e2eautoscaler/support.go and e2erayservice/support.go to reuse test/support/support.go #3947

Closed

Future-Outlier added 7 commits August 14, 2025 11:48

remove ReadFile in e2eautoscaler and e2erayservice

797ef00

Signed-off-by: Future-Outlier <[email protected]>

Trigger CI

e413bd9

Signed-off-by: Future-Outlier <[email protected]>

TEST

456c4ee

Signed-off-by: Future-Outlier <[email protected]>

update

4858f86

Signed-off-by: Future-Outlier <[email protected]>

update

4214a7b

Signed-off-by: Future-Outlier <[email protected]>

update

4b40147

Signed-off-by: Future-Outlier <[email protected]>

update

58755b7

Signed-off-by: Future-Outlier <[email protected]>

Future-Outlier added 4 commits August 14, 2025 13:07

update

13459a2

Signed-off-by: Future-Outlier <[email protected]>

Merge branch 'master' into kuberay-flaky-test-e2e-nightly-operator-fix

9b5b3e1

Merge branch 'master' into kuberay-flaky-test-e2e-nightly-operator-fix

67f8cce

Merge branch 'fix-ci' into kuberay-flaky-test-e2e-nightly-operator-fix

16854dc

kevin85421 approved these changes Aug 14, 2025

View reviewed changes

kevin85421 merged commit f2f9c90 into ray-project:master Aug 14, 2025
26 checks passed

Tomlord1122 mentioned this pull request Sep 3, 2025

[Refactor] Consolidate duplicate test utilities for maintainability #4038

Merged

4 tasks

		@@ -1,4 +1,4 @@
		- label: 'Test E2E (nightly operator)'
		- label: 'Test E2E (nightly operator) - RayCluster & GCS'

	- label: 'Test E2E (nightly operator) - RayCluster & GCS'
	- label: 'Test RayCluster and GCS E2E (nightly operator)'

	- echo "--- END:e2e (nightly operator) RayCluster & GCS tests finished"
	- echo "--- END:RayCluster and GCS E2E (nightly operator) tests finished"

	- label: 'Test E2E (nightly operator) - RayJob'
	- label: 'Test RayJob E2E (nightly operator)'

	- echo "--- END:e2e (nightly operator) RayJob tests finished"
	- echo "--- END:RayJob E2E (nightly operator) tests finished"

[Test] Split E2E nightly operator tests into RayCluster/GCS and RayJob runners #3932

[Test] Split E2E nightly operator tests into RayCluster/GCS and RayJob runners #3932

Uh oh!

Conversation

Future-Outlier commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kenchung285 Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

owenowenisme Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

win5923 left a comment

Choose a reason for hiding this comment

Uh oh!

400Ping left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

machichima left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kenchung285 left a comment

Choose a reason for hiding this comment

Uh oh!

kevin85421 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Future-Outlier commented Aug 10, 2025 •

edited

Loading

kenchung285 Aug 11, 2025 •

edited

Loading

owenowenisme Aug 10, 2025 •

edited

Loading

Future-Outlier Aug 11, 2025 •

edited

Loading

kevin85421 left a comment •

edited

Loading