Skip to content

Conversation

@lucaslie
Copy link
Collaborator

@lucaslie lucaslie commented Oct 2, 2025

will be transitioned to main repo after NVIDIA#8126 gets merged

PR switches to using kwargs instead of positional args everywhere in our transform system. Why? With mixed nn.Modules and GraphModules coming up, handling positional args reliably is becoming an increasing burden.

Fridah-nv and others added 14 commits October 1, 2025 13:13
…n models (NVIDIA#8070)

Signed-off-by: Chang Liu (Enterprise Products) <[email protected]>
Signed-off-by: Chang Liu <[email protected]>
Signed-off-by: Dongfeng Yu <[email protected]>
Signed-off-by: dongfengy <[email protected]>
Co-authored-by: Jin Li <[email protected]>
…DIA#8114)

Signed-off-by: Shunkang <[email protected]>
Signed-off-by: Patrice Castonguay <[email protected]>
Co-authored-by: Shunkang <[email protected]>
@lucaslie lucaslie self-assigned this Oct 2, 2025
@lucaslie lucaslie requested a review from Copilot October 2, 2025 22:13
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR transitions the AutoDeploy transform system from using positional arguments to keyword arguments throughout the pipeline. The change addresses challenges with handling mixed nn.Modules and GraphModules by standardizing on kwargs-based interfaces, making the system more robust and maintainable.

Key changes:

  • Replaced *cm.args with **cm.named_args across all model invocations
  • Removed strict forward method implementations and related configuration
  • Updated export functions to use kwargs instead of positional args

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tensorrt_llm/_torch/auto_deploy/export/export.py Updated torch_export_to_gm to make args optional and handle empty args tuple
tensorrt_llm/_torch/auto_deploy/transform/library/*.py Replaced args-based model calls with kwargs-based calls in transforms
tensorrt_llm/_torch/auto_deploy/models/*.py Removed strict forward method implementations and related infrastructure
tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py Updated model invocation to use kwargs interface
tests/unittest/_torch/auto_deploy/unit/singlegpu/**/*.py Updated test cases to use kwargs-based model calls
tensorrt_llm/_torch/auto_deploy/config/*.yaml Removed kwargs-related configuration options

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

gm: GraphModule,
name: str,
add_kwargs: bool = False,
add_kwargs: bool = True,
Copy link

Copilot AI Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the default value of add_kwargs from False to True is a breaking change that could affect existing callers who rely on the previous default behavior. Consider using a deprecation period or explicit migration strategy.

Suggested change
add_kwargs: bool = True,
add_kwargs: bool = False,

Copilot uses AI. Check for mistakes.
DynamicShape = Dict[int, Dim] # indicating the dynamic shape in tensor dimension
DynamicShapeCallback = Callable[[], DynamicShape]

Constant = Union[int, float, str, None]
Copy link

Copilot AI Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Constant type alias is moved from line 767 to line 27, but this creates a duplicate definition since the original at line 767 is not removed in the diff. This could lead to confusion about which definition to use.

Copilot uses AI. Check for mistakes.
@Fridah-nv
Copy link

Could you elaborate on what using kwargs everywhere means? Does it mainly indicate: the graph inputs are purely kwargs-based because of torch_export_to_gm(gm, args=(), kwargs=cm.named_args)? And the ops we inserted during graph transformations should not be affected.

I feel this is going to affect some pattern matcher behavior because we use torch_export_to_gm as the export function.

@lucaslie
Copy link
Collaborator Author

lucaslie commented Oct 3, 2025

see NVIDIA#8137

@lucaslie lucaslie closed this Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.