Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI test windows://python/ray/tests:test_actor_retry2 is flaky #47415

Closed
can-anyscale opened this issue Aug 29, 2024 · 269 comments
Closed

CI test windows://python/ray/tests:test_actor_retry2 is flaky #47415

can-anyscale opened this issue Aug 29, 2024 · 269 comments
Labels
bug Something that is supposed to be working; but isn't ci-test core Issues that should be addressed in Ray Core P2 Important issue, but not time-critical ray-test-bot Issues managed by OSS test policy stability windows

Comments

@can-anyscale
Copy link
Collaborator

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/6066#01919fcc-b1d6-4a8f-abfc-14ff5445379e
- https://buildkite.com/ray-project/postmerge/builds/6062#01919fa8-109d-407a-841a-ad20e0c1481d

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy
@can-anyscale can-anyscale added bug Something that is supposed to be working; but isn't ci-test core Issues that should be addressed in Ray Core flaky-tracker Issue created via Flaky Test Tracker https://flaky-tests.ray.io/ ray-test-bot Issues managed by OSS test policy stability triage Needs triage (eg: priority, bug/not-bug, and owning component) weekly-release-blocker Issues that will be blocking Ray weekly releases labels Aug 29, 2024
@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6072#0191a0d0-0054-4801-88c5-c18c0536c774

@can-anyscale can-anyscale reopened this Sep 3, 2024
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/6113#0191b91f-d5ad-476e-81b5-9a8015f5bb38
- https://buildkite.com/ray-project/postmerge/builds/6104#0191b45b-34cb-42af-a021-d243dfdb8206
- https://buildkite.com/ray-project/postmerge/builds/6103#0191b3b2-3e7c-4ec7-8d97-3fead106138d

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6120#0191bcae-f79d-407d-af3d-6722423ad1aa

@can-anyscale can-anyscale reopened this Sep 5, 2024
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/6131#0191bea8-989e-44ac-b9e9-e36c22216ec4
- https://buildkite.com/ray-project/postmerge/builds/6125#0191bed6-0355-4a33-a5b8-ab2398e57055
- https://buildkite.com/ray-project/postmerge/builds/6127#0191be4e-7f7c-438e-b41a-38b7ee6f507a

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale can-anyscale removed the weekly-release-blocker Issues that will be blocking Ray weekly releases label Sep 5, 2024
@can-anyscale
Copy link
Collaborator Author

not a new regression

@mattip
Copy link
Contributor

mattip commented Sep 5, 2024

Splitting the tests in two proved it is not a overall timeout, rather one or more of the tests is actually flaky. The tests are the AsyncTroubleMaker ones. Without the CI logs it is difficult to figure out what is going on. Is there a way to give me access?

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6139#0191c180-b0af-4b47-99d5-1b6635939245

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6141#0191c2f1-07e6-40dd-bf10-0714616f3436

@can-anyscale can-anyscale reopened this Sep 5, 2024
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/6142#0191c3d1-3d88-4868-b5ef-346d87bed6a6
- https://buildkite.com/ray-project/postmerge/builds/6145#0191c36f-5764-443e-95e9-4ffb6bbb36f0
- https://buildkite.com/ray-project/postmerge/builds/6144#0191c362-6618-49c5-b8a5-cafef1f5e77c

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6147#0191c43f-cc93-4c12-913a-48ca31b9e7b3

@can-anyscale can-anyscale reopened this Sep 6, 2024
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/6151#0191c488-ebd5-4307-bd75-602e3d28ded4
- https://buildkite.com/ray-project/postmerge/builds/6142#0191c3d1-3d88-4868-b5ef-346d87bed6a6
- https://buildkite.com/ray-project/postmerge/builds/6145#0191c36f-5764-443e-95e9-4ffb6bbb36f0
- https://buildkite.com/ray-project/postmerge/builds/6144#0191c362-6618-49c5-b8a5-cafef1f5e77c

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/6162#0191c5bb-54bf-43f8-96e0-e9be7db5513d

@can-anyscale can-anyscale reopened this Feb 18, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8384#019517ac-e0fc-4561-9b6f-bcdc7f75fb9d
- https://buildkite.com/ray-project/postmerge/builds/8379#01951681-1f0a-448f-8530-146811bf35e4
- https://buildkite.com/ray-project/postmerge/builds/8375#01951556-4966-4f07-8b7b-8d2bd148b231

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8389#01951991-89df-4e60-8f14-8df0d5436865

@can-anyscale can-anyscale reopened this Feb 19, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8399#01951b03-edca-449b-9b5c-93090e7daee9
- https://buildkite.com/ray-project/postmerge/builds/8392#01951b0c-9b55-44fb-b106-40353a92f18c
- https://buildkite.com/ray-project/postmerge/builds/8391#01951afb-ed75-49d6-91b6-2525a4deb1a5

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

1 similar comment
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8399#01951b03-edca-449b-9b5c-93090e7daee9
- https://buildkite.com/ray-project/postmerge/builds/8392#01951b0c-9b55-44fb-b106-40353a92f18c
- https://buildkite.com/ray-project/postmerge/builds/8391#01951afb-ed75-49d6-91b6-2525a4deb1a5

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8406#01951bc2-0b08-4089-9c97-70846cbd4686

@can-anyscale can-anyscale reopened this Feb 19, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8413#01951cfd-988c-4fab-8223-8beb19f00d91
- https://buildkite.com/ray-project/postmerge/builds/8412#01951cd4-44d2-42aa-94d6-c663dd2c4735
- https://buildkite.com/ray-project/postmerge/builds/8399#01951b03-edca-449b-9b5c-93090e7daee9

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8421#01951f03-d066-4527-b387-5bd36d591f63

@can-anyscale can-anyscale reopened this Feb 20, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8432#01952032-2a97-41d6-847c-5b2f1d3740bc
- https://buildkite.com/ray-project/postmerge/builds/8431#0195202e-c21a-4d28-bf84-4b3080867682
- https://buildkite.com/ray-project/postmerge/builds/8429#01951ff7-8347-4a14-b6ac-fbb17c6df00e

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8433#01952032-e36b-4741-b0f1-c0873def8971

@can-anyscale can-anyscale reopened this Feb 21, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8461#0195263e-f033-4551-b5e6-b8a8cd2768fa
- https://buildkite.com/ray-project/postmerge/builds/8460#0195260c-5ea8-4fb9-a2f6-e8ed6dc1949d
- https://buildkite.com/ray-project/postmerge/builds/8455#0195260f-6438-41b8-a9dc-91daf97b8922

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8469#01952749-5e7c-454a-84d8-bb38c1b09548

@can-anyscale can-anyscale reopened this Feb 21, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8467#0195271c-9c59-426a-ae3f-607cc664ddeb
- https://buildkite.com/ray-project/postmerge/builds/8460#019526c8-9af0-4a21-b6a4-61de66060f7f
- https://buildkite.com/ray-project/postmerge/builds/8461#0195263e-f033-4551-b5e6-b8a8cd2768fa
- https://buildkite.com/ray-project/postmerge/builds/8460#0195260c-5ea8-4fb9-a2f6-e8ed6dc1949d
- https://buildkite.com/ray-project/postmerge/builds/8455#0195260f-6438-41b8-a9dc-91daf97b8922

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8480#019529f3-0fe6-4275-afbb-de41cf851bf6

@can-anyscale can-anyscale reopened this Feb 24, 2025
@can-anyscale
Copy link
Collaborator Author

CI test windows://python/ray/tests:test_actor_retry2 is flaky. Recent failures:
- https://buildkite.com/ray-project/postmerge/builds/8526#019539c7-0aba-4851-8399-72daf1e46143
- https://buildkite.com/ray-project/postmerge/builds/8523#01953967-307e-4faf-9633-bd0d27fb0729
- https://buildkite.com/ray-project/postmerge/builds/8514#01953949-b48d-47fe-920b-6864b6e85231

DataCaseName-windows://python/ray/tests:test_actor_retry2-END
Managed by OSS Test Policy

@can-anyscale
Copy link
Collaborator Author

Test passed on latest run: https://buildkite.com/ray-project/postmerge/builds/8533#01953a4b-ab50-4fcc-be96-507783dcb37f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something that is supposed to be working; but isn't ci-test core Issues that should be addressed in Ray Core P2 Important issue, but not time-critical ray-test-bot Issues managed by OSS test policy stability windows
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mattip @jjyao @can-anyscale @jcotant1