Skip to content

Support Multiple Model initialization#1018

Closed
gwarmstrong wants to merge 33 commits intomainfrom
georgea/multiturn-conversation
Closed

Support Multiple Model initialization#1018
gwarmstrong wants to merge 33 commits intomainfrom
georgea/multiturn-conversation

Conversation

@gwarmstrong
Copy link
Collaborator

This PR supports having multiple models initialized in a generate command, like so:


from nemo_skills.pipeline.cli import wrap_arguments, generate

generate(
    ctx=wrap_arguments(""),
    cluster="ord",
    partition="interactive",
    model=["/hf_models/Qwen3-32B", "/hf_models/Qwen3-8B"],
    server_gpus=8,  # Broadcast to both models
    server_nodes=[2,1], # 2 nodes for 32B, 1 node for 8B
    server_type="vllm",
    generation_module="multiturn-conversation/multiturn_conversation.py",
    input_file="/nemo_run/code/multiturn-conversation/sample_prompts.jsonl",
    output_dir=f"/workspace/multiturn-conversation/minimal_example/output",
)

This generation module is not available in the branch (and likely won't due to the specificity of its implementation) but reach out to me if you want the example.

Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
@gwarmstrong gwarmstrong requested a review from Kipok November 4, 2025 21:45
Signed-off-by: George Armstrong <georgea@nvidia.com>
Signed-off-by: George Armstrong <georgea@nvidia.com>
Copy link
Collaborator

@Kipok Kipok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks! Feel free to merge as long as both gpu and slurm tests are passing

@gwarmstrong gwarmstrong self-assigned this Nov 10, 2025
@gwarmstrong
Copy link
Collaborator Author

Closed in favor of #1052

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants