[train] Update vicuna release test example to use V2 #57767
Conversation
Signed-off-by: Justin Yu <[email protected]>
Signed-off-by: Justin Yu <[email protected]>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This PR updates the cluster configuration for the Vicuna example to use 16 worker GPUs for training, which is a valid change. However, this introduces an inconsistency with the documentation in the accompanying Jupyter Notebook, which has not been updated. I've left a specific comment on the configuration file change. Please update the notebook documentation to match the new cluster setup to avoid user confusion.
| instance_type: m5.4xlarge | ||
|
|
||
| worker_node_types: | ||
| - name: worker_node | ||
| instance_type: g5.4xlarge | ||
| min_workers: 15 | ||
| max_workers: 15 | ||
| min_workers: 16 | ||
| max_workers: 16 |
There was a problem hiding this comment.
These changes correctly adjust the cluster to use 16 worker GPUs for training. However, this makes the documentation in the corresponding notebook (vicuna_13b_lightning_deepspeed_finetune.ipynb) outdated and misleading.
The notebook's 'Cluster Setting' section needs to be updated to reflect:
- The head node is now
m5.4xlarge(a CPU instance). - There are now 16 worker nodes.
- The tip about using a GPU head node for inference is no longer accurate and should be revised, as inference will now run on a worker node.
Please update the notebook to ensure the example remains consistent and clear for users.
| "name": "stderr", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "2025-10-15 15:50:45,333\tINFO worker.py:1833 -- Connecting to existing Ray cluster at address: 10.0.171.127:6379...\n", |
There was a problem hiding this comment.
We should delete these output cells here and elsewhere so they don't show up in the docs.
Signed-off-by: Justin Yu <[email protected]>
ray-gardener
bot
added
docs
Signed-off-by: Justin Yu <[email protected]>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Justin Yu <[email protected]>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Justin Yu <[email protected]>
Signed-off-by: Justin Yu <[email protected]>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Justin Yu <[email protected]>
Signed-off-by: Justin Yu <[email protected]>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <[email protected]>
Cherrypicking in e7a79ba Signed-off-by: Justin Yu <[email protected]> Co-authored-by: Justin Yu <[email protected]>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <[email protected]> Signed-off-by: xgui <[email protected]>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <[email protected]>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <[email protected]> Signed-off-by: Aydin Abiar <[email protected]>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <[email protected]> Signed-off-by: Future-Outlier <[email protected]>
2 participants
Description
Updates the vicuna lightning deepspeed example to run w/ Train V2.
Related issues
Types of change
Checklist
Does this PR introduce breaking changes?
Testing:
Code Quality:
git commit -s)Documentation:
doc/source/(if applicable)Additional context