Skip to content

Conversation

alexander-soare
Copy link
Contributor

@alexander-soare alexander-soare commented Sep 21, 2021

Currently in draft for visibility.

Adds freeze/unfreeze to timm.utils. These let you provide a list of modules or module names for which to freeze parameters.

  • Resolve the ugliness of what to do if user provides a batch norm layer directly. Right now this won't always work because we need to do convert it in place within the scope of the parent module. But we can't do this if we aren't explicitly given the parent module.
  • Implement unfreezing of batch norm layers.
    • @rwightman any idea on this one? I would consider doing in place replacement of all FrozenBatchNorm2d layers to BatchNorm2d (so the exact reverse of FrozenBatchNorm2d.convert_frozen_batchnorm).

@alexander-soare alexander-soare marked this pull request as draft September 21, 2021 11:53
@alexander-soare
Copy link
Contributor Author

Will hopefully pick this up again next week!

@alexander-soare alexander-soare force-pushed the freeze-functionality branch 4 times, most recently from cf19462 to 003226d Compare September 21, 2021 12:19
@alexander-soare alexander-soare marked this pull request as ready for review October 2, 2021 14:56
@alexander-soare
Copy link
Contributor Author

@rwightman ready to review

The only thing kind of bothering me about this one is that there is a BatchNorm1d in levit and the freeze only handles BatchNorm2d. Wasn't familiar enough with levit to know if that case needs to be handled. In any case, the docstrings are pretty clear that only BatchNorm2d is handled.

@rwightman rwightman merged commit 5ca72dc into huggingface:master Oct 7, 2021
@rwightman
Copy link
Collaborator

@alexander-soare thanks! I tested witha few models today, looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants