AttributeError: 'NoneType' object has no attribute 'device'

[jovany-wang]

🐛 Describe the bug

trlx_gptj_text_summarization.py raised error:

  File "trlx_gptj_text_summarization.py", line 179, in <module>
    trainer = trlx.train(
  File "/root/workspace/qwang/trlx/trlx/trlx.py", line 103, in train
    trainer.make_experience(config.method.num_rollouts)
  File "/root/workspace/qwang/trlx/trlx/trainer/accelerate_ppo_trainer.py", line 408, in make_experience
    ref_logits = self.model.forward_hydra(
  File "/root/workspace/qwang/trlx/trlx/models/modeling_ppo.py", line 387, in forward_hydra
    hydra_outputs = self.frozen_head(input_hidden_state, output_shape, forward_kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, kwargs)
  File "/root/workspace/qwang/trlx/trlx/models/modeling_ppo.py", line 515, in forward
    outputs = block(
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(args, **kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 308, in forward
    attn_outputs = self.attn(
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(args, **kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 219, in forward
    embed_positions = self._get_embed_positions(position_ids)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 188, in _get_embed_positions
    if embed_positions.device != position_ids.device:
AttributeError: 'NoneType' object has no attribute 'device'

1 machine 4*16GV100, and all configs are by default.

Which trlX version are you using?

current master

Additional system and package information

python3.8
transformers==4.28.1

[PhungVanDuy]

🐛 Describe the bug

trlx_gptj_text_summarization.py raised error:

  File "trlx_gptj_text_summarization.py", line 179, in <module>
    trainer = trlx.train(
  File "/root/workspace/qwang/trlx/trlx/trlx.py", line 103, in train
    trainer.make_experience(config.method.num_rollouts)
  File "/root/workspace/qwang/trlx/trlx/trainer/accelerate_ppo_trainer.py", line 408, in make_experience
    ref_logits = self.model.forward_hydra(
  File "/root/workspace/qwang/trlx/trlx/models/modeling_ppo.py", line 387, in forward_hydra
    hydra_outputs = self.frozen_head(input_hidden_state, output_shape, forward_kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, kwargs)
  File "/root/workspace/qwang/trlx/trlx/models/modeling_ppo.py", line 515, in forward
    outputs = block(
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(args, **kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 308, in forward
    attn_outputs = self.attn(
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(args, **kwargs)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 219, in forward
    embed_positions = self._get_embed_positions(position_ids)
  File "/root/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/gptj/modeling_gptj.py", line 188, in _get_embed_positions
    if embed_positions.device != position_ids.device:
AttributeError: 'NoneType' object has no attribute 'device'

1 machine 4*16GV100

Which trlX version are you using?

current master

Additional system and package information

python3.8

What is your transformers version?

[jovany-wang]

@PhungVanDuy transformers==4.28.1

[jovany-wang]

Let me downgrade transformers==4.27.1 and re-trying it...

[PhungVanDuy]

Let me downgrade transformers==4.27.1 and re-trying it...

please try with transformers==4.26.0. I just want to make sure it works with the version the last time I trained with this script.

[jovany-wang]

OK. Please hold on. I'm trying transformers==4.27.1, which is pined in trlx/requirements.txt

[jovany-wang]

It's training now with transformers==4.27.1

[jovany-wang]

Let me pin it.