Skip to content

Change type to tuple in replace_wo_policy isinstance check #2387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lekurile merged 7 commits into from
Oct 7, 2022
Merged

Change type to tuple in replace_wo_policy isinstance check #2387

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
323db85
Change type to tuple in replace_wo_policy isinstance check
lekurile Oct 5, 2022
3669708
Merge branch 'master' into lekurile/fix_isinstance_type_check
awan-10 Oct 5, 2022
843825d
Add str type check as well for single value tuples that become strings
lekurile Oct 6, 2022
d8fa4de
Merge branch 'master' into lekurile/fix_isinstance_type_check
awan-10 Oct 6, 2022
73d41a0
pre-commit
lekurile Oct 7, 2022
367d1b2
Merge branch 'lekurile/fix_isinstance_type_check' of https://github.c…
lekurile Oct 7, 2022
809fd05
Merge branch 'master' into lekurile/fix_isinstance_type_check
lekurile Oct 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion deepspeed/module_inject/replace_module.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,9 @@ def _replace(child, name, conv_linear_layer):
weight_shape = child.weight.ds_shape
else:
weight_shape = child.weight.shape
if isinstance(all_reduce_linears, dict) and name in all_reduce_linears:
if (isinstance(all_reduce_linears,
tuple) or isinstance(all_reduce_linears,
str)) and name in all_reduce_linears:
new_weight = torch.empty((
weight_shape[1] if conv_linear_layer else weight_shape[0],
(weight_shape[0] if conv_linear_layer else weight_shape[1]) //
Expand Down