[rllib] Replace ray.get() with ray_get_and_free() to optimize memory usage #4586

ericl · 2019-04-09T08:25:53Z

What do these changes do?

When ray.get() is called in RLlib, the result should be freed immediately. This prevents object store memory from growing unnecessarily.

Related issue number

(many previous issues)

Linter

I've run scripts/format.sh to lint the changes in this PR.

ericl · 2019-04-09T08:28:18Z

@robertnishihara @guoyuhong do you see any performance issues with calling ray.internal.free up to a couple hundred times a second here?

AmplabJenkins · 2019-04-09T08:29:26Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/399/
Test PASSed.

AmplabJenkins · 2019-04-09T08:36:51Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/400/
Test PASSed.

AmplabJenkins · 2019-04-09T09:08:38Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13655/
Test FAILed.

AmplabJenkins · 2019-04-09T09:14:52Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13656/
Test FAILed.

guoyuhong · 2019-04-09T09:37:29Z

@ericl Maybe you need to do some tests. ray.internal.free will spread the message to all raylet nodes. In our internal code, we do not call ray.internal.free for each object. We will collect the object ids and send them by batch.

ericl · 2019-04-10T00:55:33Z

Hm yeah, that might be a high overhead if we broadcast to all nodes for each object created. I could also batch the objects and periodically flush them.

AmplabJenkins · 2019-04-10T02:04:26Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/403/
Test PASSed.

AmplabJenkins · 2019-04-10T02:11:26Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/404/
Test PASSed.

AmplabJenkins · 2019-04-10T02:24:47Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13671/
Test FAILed.

AmplabJenkins · 2019-04-10T02:25:00Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/405/
Test PASSed.

AmplabJenkins · 2019-04-10T02:30:13Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13672/
Test FAILed.

AmplabJenkins · 2019-04-10T02:31:50Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/406/
Test PASSed.

AmplabJenkins · 2019-04-10T02:50:55Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13677/
Test FAILed.

AmplabJenkins · 2019-04-10T04:04:41Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/408/
Test PASSed.

AmplabJenkins · 2019-04-10T04:28:54Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13680/
Test FAILed.

AmplabJenkins · 2019-04-10T04:35:37Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13675/
Test FAILed.

ericl · 2019-04-11T04:50:15Z

jenkins retest this please

AmplabJenkins · 2019-04-11T04:54:18Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/425/
Test PASSed.

AmplabJenkins · 2019-04-11T06:58:09Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13719/
Test FAILed.

AmplabJenkins · 2019-04-12T01:31:10Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/445/
Test PASSed.

AmplabJenkins · 2019-04-12T04:26:00Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13752/
Test FAILed.

ericl · 2019-04-12T18:33:31Z

cc @joneswong

ericl · 2019-04-13T02:16:13Z

python/ray/rllib/utils/memory.py

+
+    now = time.time()
+    if (len(_to_free) > MAX_FREE_QUEUE_SIZE
+            or now - _last_free_time > FREE_DELAY_S):


batch the free calls to avoid too much overheads

AmplabJenkins · 2019-04-13T04:48:32Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/467/
Test PASSed.

AmplabJenkins · 2019-04-13T04:55:03Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/468/
Test PASSed.

AmplabJenkins · 2019-04-13T07:20:50Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/13782/
Test FAILed.

ericl · 2019-04-17T23:46:28Z

@richardliaw ?

richardliaw · 2019-04-18T00:19:02Z

python/ray/internal/internal_api.py

    """
    worker = ray.worker.get_global_worker()

+    if ray.worker._mode() == ray.worker.LOCAL_MODE:


oh this is nice

This PR contains changes that help with memory issues ray-project#4586

ericl added 2 commits April 9, 2019 01:16

wip

f77c861

comment

9b74f05

ars

b6e485b

fix local mode

050b179

ericl added 2 commits April 9, 2019 19:06

fix internal

e2284b2

add max queue size

a848618

free

092443b

Merge remote-tracking branch 'upstream/master' into save-memory

411cd15

py2

8b3dfd3

Update memory.py

85a0597

ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Apr 12, 2019

ericl commented Apr 13, 2019

View reviewed changes

ericl added 3 commits April 12, 2019 21:47

Merge remote-tracking branch 'upstream/master' into save-memory

fa839c7

typo

d968d25

Merge branch 'save-memory' of github.com:ericl/ray into save-memory

5b13bef

comment

6f1f5cb

richardliaw reviewed Apr 18, 2019

View reviewed changes

richardliaw approved these changes Apr 18, 2019

View reviewed changes

ericl merged commit 6848dfd into ray-project:master Apr 18, 2019

ericl mentioned this pull request May 28, 2019

[rllib] Distributed learning constantly crashing due to memory #4877

Closed

Edilmo added a commit to BonsaiAI/ray that referenced this pull request Feb 7, 2020

Cherry picking changes from PR 4586

0be81b8

This PR contains changes that help with memory issues ray-project#4586

Edilmo mentioned this pull request Feb 8, 2020

[fix][0.6.6] Raylet connection closed & RayOutOfMemoryError BonsaiAI/ray#3

Merged

[rllib] Replace ray.get() with ray_get_and_free() to optimize memory usage #4586

[rllib] Replace ray.get() with ray_get_and_free() to optimize memory usage #4586

Uh oh!

Conversation

ericl commented Apr 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What do these changes do?

Related issue number

Linter

Uh oh!

ericl commented Apr 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AmplabJenkins commented Apr 9, 2019

Uh oh!

AmplabJenkins commented Apr 9, 2019

Uh oh!

AmplabJenkins commented Apr 9, 2019

Uh oh!

AmplabJenkins commented Apr 9, 2019

Uh oh!

guoyuhong commented Apr 9, 2019

Uh oh!

ericl commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

AmplabJenkins commented Apr 10, 2019

Uh oh!

ericl commented Apr 11, 2019

Uh oh!

AmplabJenkins commented Apr 11, 2019

Uh oh!

AmplabJenkins commented Apr 11, 2019

Uh oh!

AmplabJenkins commented Apr 12, 2019

Uh oh!

AmplabJenkins commented Apr 12, 2019

Uh oh!

ericl commented Apr 12, 2019

Uh oh!

ericl Apr 13, 2019

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented Apr 13, 2019

Uh oh!

AmplabJenkins commented Apr 13, 2019

Uh oh!

AmplabJenkins commented Apr 13, 2019

Uh oh!

ericl commented Apr 17, 2019

Uh oh!

richardliaw Apr 18, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ericl commented Apr 9, 2019 •

edited

Loading

ericl commented Apr 9, 2019 •

edited

Loading