Integrates Hostmesh v1 and RDMA #378
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meta-cla
bot
added
the
CLA Signed
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #378 +/- ##
=======================================
Coverage ? 64.44%
=======================================
Files ? 79
Lines ? 7735
Branches ? 0
=======================================
Hits ? 4985
Misses ? 2750
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
allenwang28
changed the title
joecummings
approved these changes
Oct 13, 2025
| dataset: | ||
| procs: 1 | ||
| with_gpus: false | ||
| mesh_name: dataset |
There was a problem hiding this comment.
These have to be manually specified?
There was a problem hiding this comment.
there's probably a way to do this in a not silly way but alas
vidhyav
reviewed
Oct 13, 2025
Comment on lines
+23
to
+24
| from monarch._src.actor.v1.host_mesh import this_proc | ||
| from monarch._src.actor.v1.proc_mesh import get_or_spawn_controller |
There was a problem hiding this comment.
We should remove this in lieu of the public APIs which have moved to v1 by default.
photomz
pushed a commit
to photomz/forge
that referenced
this pull request
Oct 25, 2025
* start adding env constants * adds env_constants * get_value on env var * makes changes in the rest of the codebase * final fix * a few more cleanups * merge fixes * test fix * initial commit for hostmesh v1 * more updates for testing * changes for running * extent * replace monarch build * rename wheel * fix * fix docs * grpo main still runs * fix docs build * rename ci wheel * fix the wheel path * wheel name * documentation * fix remote_setup for MAST
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
Extentnor HostMesh stuff)MONARCH_HOSTMESH_V1environment variable, which defaults to False.use_dcpis now controlled by whether or not RDMA is being used.To run:
This won't work until changes in TorchStore lands. Until then, the normal path works (i.e. defaults are
TORCHSTORE_RDMA_ENABLED=0andMONARCH_HOSTMESH_V1=0)Proof of 32B running with this setup: