Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Docs do-over (new API stack): Add scaling guide rst page. #49528

Merged
merged 28 commits into from
Jan 4, 2025
Merged

[RLlib] Docs do-over (new API stack): Add scaling guide rst page. #49528

merged 28 commits into from
Jan 4, 2025

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Jan 1, 2025 •
edited
Loading

Docs do-over (new API stack):

  • Add new scaling guide rst page.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@sven1977 sven1977 requested review from maxpumperla, simonsays1980 and a team as code owners January 1, 2025 15:45
@sven1977 sven1977 changed the title [RLlib] Docs do-over (new API stack): Add scaling guide rst page. [WIP; RLlib] Docs do-over (new API stack): Add scaling guide rst page. Jan 1, 2025
@sven1977 sven1977 added rllib RLlib related issues rllib-docs-or-examples Issues related to RLlib documentation or rllib/examples rllib-newstack labels Jan 1, 2025
@sven1977 sven1977 changed the title [WIP; RLlib] Docs do-over (new API stack): Add scaling guide rst page. [RLlib] Docs do-over (new API stack): Add scaling guide rst page. Jan 1, 2025
@sven1977
merge
8a26f20 
Signed-off-by: sven1977 <[email protected]>
@sven1977
fix
5c45c55 
Signed-off-by: sven1977 <[email protected]>
angelinalg
angelinalg approved these changes Jan 3, 2025
View reviewed changes
doc/source/rllib/rllib-training.rst Outdated
Configuring RLlib Algorithms
----------------------------

You can configure RLlib algorithms in a modular fashion by working with so-called
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can configure RLlib algorithms in a modular fashion by working with so-called
You can configure RLlib algorithms in a modular fashion by working with

doc/source/rllib/rllib-training.rst Outdated

You can configure RLlib algorithms in a modular fashion by working with so-called
`AlgorithmConfig` objects.
In essence, you first create a `config = AlgorithmConfig()` object and then call methods
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In essence, you first create a `config = AlgorithmConfig()` object and then call methods
First create a `config = AlgorithmConfig()` object and then call methods

doc/source/rllib/rllib-training.rst Outdated
In essence, you first create a `config = AlgorithmConfig()` object and then call methods
on it to set the desired configuration options.
Each RLlib algorithm has its own config class that inherits from `AlgorithmConfig`.
For instance, to create a `PPO` algorithm, you start with a `PPOConfig` object, to work
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For instance, to create a `PPO` algorithm, you start with a `PPOConfig` object, to work
For instance, to create a `PPO` algorithm, start with a `PPOConfig` object, to work

doc/source/rllib/rllib-training.rst Outdated
on it to set the desired configuration options.
Each RLlib algorithm has its own config class that inherits from `AlgorithmConfig`.
For instance, to create a `PPO` algorithm, you start with a `PPOConfig` object, to work
with a `DQN` algorithm, you start with a `DQNConfig` object, etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with a `DQN` algorithm, you start with a `DQNConfig` object, etc.
with a `DQN` algorithm, start with a `DQNConfig` object, etc.

doc/source/rllib/rllib-training.rst Outdated

.. note::

Each algorithm has its specific settings, but most configuration options are shared.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Each algorithm has its specific settings, but most configuration options are shared.
Each algorithm has its specific settings, but most configuration options are shared.

doc/source/rllib/scaling-guide.rst Outdated
single local :py:class:`~ray.rllib.core.learner.learner.Learner` instance (``num_learners=0``),
depends on whether you have a GPU available or not.
If exactly one GPU is available, you should run these two algorithms with ``num_learners=0, num_gpus_per_learner=1``,
if no GPU is available, you should set ``num_learners=1, num_gpus_per_learner=0``. For > 1 GPUs available, you should
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if no GPU is available, you should set ``num_learners=1, num_gpus_per_learner=0``. For > 1 GPUs available, you should
if no GPU is available, set ``num_learners=1, num_gpus_per_learner=0``. For > 1 GPUs available, you should

doc/source/rllib/scaling-guide.rst Outdated Show resolved Hide resolved
doc/source/rllib/scaling-guide.rst Outdated Show resolved Hide resolved
doc/source/rllib/scaling-guide.rst Outdated Show resolved Hide resolved
doc/source/rllib/scaling-guide.rst Outdated Show resolved Hide resolved
sven1977 and others added 4 commits January 4, 2025 06:38
@sven1977 sven1977 enabled auto-merge (squash) January 4, 2025 05:52
@github-actions github-actions bot added the go add ONLY when ready to merge, run all tests label Jan 4, 2025
@github-actions github-actions bot disabled auto-merge January 4, 2025 06:39
@sven1977 sven1977 enabled auto-merge (squash) January 4, 2025 06:39
@sven1977
fix
add15c5 
Signed-off-by: sven1977 <[email protected]>
@github-actions github-actions bot disabled auto-merge January 4, 2025 10:51
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into docs…
9f808da 
…_redo_scaling_guide

Signed-off-by: sven1977 <[email protected]>

# Conflicts:
#	doc/source/rllib/user-guides.rst
@sven1977 sven1977 enabled auto-merge (squash) January 4, 2025 11:09
@sven1977
wip
c22c792 
Signed-off-by: sven1977 <[email protected]>
@github-actions github-actions bot disabled auto-merge January 4, 2025 11:17
@sven1977 sven1977 merged commit e2f93ad into ray-project:master Jan 4, 2025
5 checks passed
@sven1977 sven1977 deleted the docs_redo_scaling_guide branch January 4, 2025 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@angelinalg angelinalg angelinalg approved these changes

@maxpumperla maxpumperla Awaiting requested review from maxpumperla maxpumperla is a code owner

@simonsays1980 simonsays1980 Awaiting requested review from simonsays1980 simonsays1980 is a code owner

Assignees

@simonsays1980 simonsays1980

@angelinalg angelinalg

Labels
go add ONLY when ready to merge, run all tests rllib RLlib related issues rllib-docs-or-examples Issues related to RLlib documentation or rllib/examples rllib-newstack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sven1977 @angelinalg @simonsays1980