[fix][0.6.6] Raylet connection closed & RayOutOfMemoryError #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR contains changes that help with memory issues ray-project#4942
This PR contains changes that help with memory issues ray-project#4945
This PR contains changes that help with memory issues ray-project#4964
This PR contains changes that help with memory issues ray-project#4586
Specifically, we are setting a travis like ADO pipeline following what is already present in the .travis.yml file in the root of the repo.
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
4 times, most recently
from
2a8c36c to
e78e2be
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
4 times, most recently
from
c87ad51 to
13a5173
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
from
13a5173 to
0ddcfa0
Compare
This PR contains changes that help with memory issues ray-project#4555
This PR contains changes that help with memory issues ray-project#4533
This PR contains changes that help with memory issues ray-project#4465
This PR contains changes that help with memory issues ray-project#4701
This PR contains changes that help with memory issues ray-project@d13e889
This PR contains changes that help with memory issues ray-project@0abcd07
This PR contains changes that help with memory issues ray-project@28d73f6
This PR contains changes that help with memory issues ray-project#4959
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
from
2116625 to
e7dbea2
Compare
This PR contains changes that help with memory issues ray-project#4615
This PR contains changes that help with memory issues ray-project#4711
This PR contains changes that help with memory issues ray-project#6073
This PR contains changes that help with memory issues ray-project#4862
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
10 times, most recently
from
754e2f9 to
3d074e2
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
2 times, most recently
from
310eed0 to
c1c7f5d
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
from
c1c7f5d to
33ca62b
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
from
c04ff62 to
5cd4651
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
2 times, most recently
from
2bb206b to
ab05c34
Compare
Edilmo
force-pushed
the
ms/fix-0.6.6-con-close
branch
from
ab05c34 to
2c264c7
Compare
Edilmo
changed the title
Edilmo
pushed a commit
that referenced
this pull request
Jan 5, 2021
Edilmo
pushed a commit
that referenced
this pull request
May 6, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have been experimenting "Raylet connection closed" and "RayOutOfMemoryError" errors with Ray 0.6.6. This PR is taking advantage of the CI introduce in the #2 PR and try to alleviate those problems by cherry picking changes made after 0.6.6 that, in one way or another, deal with the errors.
After some investigation we found several Issues/PRs trying to address errors with this message as main symptom.
Our real root cause has not been identified yet. But in order to test our dev-process around Ray/RLlib, we are taking this as opportunity to experience the process of cherrypicking changes from upstream and applying that to our running Ray version.
Links: