'tf_keras' has no attribute 'activations'

[DeeraDigiSpoc]

error:

Failed to import transformers.trainer :
Failed to import transformers.integrations.integration_utils
Failed to import transformers.modeling_tf_utils 
module 'tf_keras' has no attribute 'activations'

System Info

tranformers_version == 4.41.0
python_version == 3.11
tensorflow_vesrion == 2.16.2

same issue with ,

tranformers_version == 4.40.0
python_version == 3.11
tensorflow_vesrion == 2.15.0

I knew this was from a version issue with Transformers. Still, I tried downgrading and upgrading. Has anyone else faced similar issues? Kindly share your solutions/thoughts.

@ArthurZucker @muellerz @SunMarc @Rocketknight1

Reproduction

1. using the version above
2. from trl import SFTConfig, SFTTrainer

Expected behavior

load model training.

[Rocketknight1]

@DeeraDigiSpoc Can you try pip install --upgrade tf-keras?

[DeeraDigiSpoc]

@DeeraDigiSpoc Can you try pip install --upgrade tf-keras?

Requirement already satisfied: tf-keras in ./anaconda3/envs/deera/lib/python3.10/site-packages (2.16.0)

[Rocketknight1]

@DeeraDigiSpoc I can't figure that one out! I've made an env with tf-keras == 2.16.0 here, and tf_keras.activations is a valid module.

[DeeraDigiSpoc]

I installed

keras-->3.4.1
tensorflow-->2.16..0rc0

@Rocketknight1 - This is entire AttributeError.

Reproduction

from trl import SFTConfig, SFTTrainer

AttributeError                            Traceback (most recent call last)
File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1535, in _LazyModule._get_module(self, module_name)
   1534 try:
-> 1535     return importlib.import_module("." + module_name, self.__name__)
   1536 except Exception as e:

File ~/anaconda3/envs/deera/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/modeling_tf_utils.py:38
     37 from . import DataCollatorWithPadding, DefaultDataCollator
---> 38 from .activations_tf import get_tf_activation
     39 from .configuration_utils import PretrainedConfig

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/activations_tf.py:120
    118     return keras.activations.gelu(x, approximate=True)
--> 120 gelu = keras.activations.gelu
    121 gelu_new = approximate_gelu_wrap

**AttributeError: module 'tf_keras' has no attribute 'activations'**

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1535, in _LazyModule._get_module(self, module_name)
   1534 try:
-> 1535     return importlib.import_module("." + module_name, self.__name__)
   1536 except Exception as e:

File ~/anaconda3/envs/deera/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/integrations/integration_utils.py:34
     32 import packaging.version
---> 34 from .. import PreTrainedModel, TFPreTrainedModel
     35 from .. import __version__ as version

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1525, in _LazyModule.__getattr__(self, name)
   1524 elif name in self._class_to_module.keys():
-> 1525     module = self._get_module(self._class_to_module[name])
   1526     value = getattr(module, name)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1537, in _LazyModule._get_module(self, module_name)
   1536 except Exception as e:
-> 1537     raise RuntimeError(
   1538         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1539         f" traceback):\n{e}"
   1540     ) from e

**RuntimeError: Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tf_keras' has no attribute 'activations'**

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1535, in _LazyModule._get_module(self, module_name)
   1534 try:
-> 1535     return importlib.import_module("." + module_name, self.__name__)
   1536 except Exception as e:

File ~/anaconda3/envs/deera/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/trainer.py:41
     39 # Integrations must be imported before ML frameworks:
     40 # isort: off
---> 41 from .integrations import (
     42     get_reporting_integration_callbacks,
     43     hp_params,
     44 )
     46 # isort: on

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1525, in _LazyModule.__getattr__(self, name)
   1524 elif name in self._class_to_module.keys():
-> 1525     module = self._get_module(self._class_to_module[name])
   1526     value = getattr(module, name)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1537, in _LazyModule._get_module(self, module_name)
   1536 except Exception as e:
-> 1537     raise RuntimeError(
   1538         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1539         f" traceback):\n{e}"
   1540     ) from e

RuntimeError: Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tf_keras' has no attribute 'activations'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/anaconda3/envs/deera/lib/python3.10/site-packages/trl/import_utils.py:180, in _LazyModule._get_module(self, module_name)
    179 try:
--> 180     return importlib.import_module("." + module_name, self.__name__)
    181 except Exception as e:

File ~/anaconda3/envs/deera/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:28
     27 from huggingface_hub.utils._deprecation import _deprecate_arguments
---> 28 from transformers import (
     29     AutoModelForCausalLM,
     30     AutoTokenizer,
     31     DataCollator,
     32     DataCollatorForLanguageModeling,
     33     PreTrainedModel,
     34     PreTrainedTokenizerBase,
     35     Trainer,
     36 )
     37 from transformers.modeling_utils import unwrap_model

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1525, in _LazyModule.__getattr__(self, name)
   1524 elif name in self._class_to_module.keys():
-> 1525     module = self._get_module(self._class_to_module[name])
   1526     value = getattr(module, name)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/transformers/utils/import_utils.py:1537, in _LazyModule._get_module(self, module_name)
   1536 except Exception as e:
-> 1537     raise RuntimeError(
   1538         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1539         f" traceback):\n{e}"
   1540     ) from e

RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tf_keras' has no attribute 'activations'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[19], line 1
----> 1 from trl import SFTConfig, SFTTrainer

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/trl/import_utils.py:171, in _LazyModule.__getattr__(self, name)
    169 elif name in self._class_to_module.keys():
    170     module = self._get_module(self._class_to_module[name])
--> 171     value = getattr(module, name)
    172 else:
    173     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/trl/import_utils.py:170, in _LazyModule.__getattr__(self, name)
    168     value = self._get_module(name)
    169 elif name in self._class_to_module.keys():
--> 170     module = self._get_module(self._class_to_module[name])
    171     value = getattr(module, name)
    172 else:

File ~/anaconda3/envs/deera/lib/python3.10/site-packages/trl/import_utils.py:182, in _LazyModule._get_module(self, module_name)
    180     return importlib.import_module("." + module_name, self.__name__)
    181 except Exception as e:
--> 182     raise RuntimeError(
    183         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
    184         f" traceback):\n{e}"
    185     ) from e

RuntimeError: Failed to import trl.trainer.sft_trainer because of the following error (look up to see its traceback):
Failed to import transformers.trainer because of the following error (look up to see its traceback):
Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_tf_utils because of the following error (look up to see its traceback):
module 'tf_keras' has no attribute 'activations'

[Rocketknight1]

Hi @DeeraDigiSpoc, I cannot reproduce the issue here with Tensorflow version 2.16.1, tf_keras version 2.16.0 and the latest versions of transformers and trl installed with pip install --upgrade transformers trl.

As such, I'm not sure what to suggest! Please try pip install --upgrade transformers trl yourself, and if that doesn't resolve it, I'm not sure what we can do, because the issue seems to be some kind of environment misconfiguration