Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FineTune Model #1

Open
mdv3101 opened this issue Apr 13, 2023 · 0 comments
Open

FineTune Model #1

mdv3101 opened this issue Apr 13, 2023 · 0 comments

Comments

@mdv3101
Copy link

mdv3101 commented Apr 13, 2023

I am trying to fine-tune the model on a different face dataset. However I am getting the following error.

Traceback (most recent call last):
  File "run_train.py", line 8, in <module>
    train(conf, gpus=gpus, nodes=nodes)
  File "/home/Diffusion-Video-Autoencoders/experiment.py", line 769, in train
    trainer.fit(model)
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 552, in fit
    self._run(model)
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 914, in _run
    self.checkpoint_connector.restore_training_state()
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 164, in restore_training_state
    self.restore_loops()
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 217, in restore_loops
    self.trainer.fit_loop.load_state_dict(state_dict["fit_loop"])
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 191, in load_state_dict
    v.load_state_dict(state_dict.copy(), prefix + k + ".", restart_progress)
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 191, in load_state_dict
    v.load_state_dict(state_dict.copy(), prefix + k + ".", restart_progress)
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 188, in load_state_dict
    self._load_from_state_dict(state_dict.copy(), prefix, restart_progress)
  File "/home/env_tf/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 196, in _load_from_state_dict
    v.load_state_dict(state_dict[prefix + k])
KeyError: 'epoch_loop.batch_loop.optim_progress'

Is there an issue with the checkpoint file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant