Cherry pick PR #7003 (#7441)

* Pass tp config via hydra Signed-off-by: Jan Baczek <[email protected]> * Remove self.ub_cfgs field - it isn't used anywhere else Signed-off-by: Jan Baczek <[email protected]> * Allow tp_overlap tree substitution in hydra config Signed-off-by: Jan Baczek <[email protected]> * Add warning in case of usage of the default tp config Signed-off-by: Jan Baczek <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <[email protected]> * Change warning message Signed-off-by: Jan Baczek <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <[email protected]> * Add compute capability resolver Signed-off-by: Jan Baczek <[email protected]> * Bugfix Signed-off-by: Jan Baczek <[email protected]> * Fix cherry pick Signed-off-by: Jan Baczek <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jan Baczek <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
NVIDIA · Oct 24, 2023 · b78b84f · b78b84f
1 parent d5ef8cd
commit b78b84f
Showing 1 changed file with 0 additions and 1 deletion.
diff --git a/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py b/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py
@@ -555,7 +555,6 @@ def initialize_ub_func(self):
             self.cfg.get('encoder_seq_length') * self.cfg.get('micro_batch_size'),
             self.cfg.get('hidden_size'),
         ]
-
         te_module.base.initialize_ub(
             shape=input_shape,
             tp_size=self.cfg.get('tensor_model_parallel_size'),