Distributed Trainer: 2 little fixes

[sshleifer]

1. fix DDP access to model.config. We could also set self.config = model.config earlier in __init__
2. switch torch.Tensor -> torch.tensor. The latter "infers the dtype automatically"
   After which the command in Seq2SeqTrainer Distributed: AttributeError and the RuntimeError #7460 works.

CC @patil-suraj , @TevenLeScao

[sgugger]

Can we see when the config is accessed (in your error message)? model.config should be accessed as sparsely as possible in Trainer to work with any kind of model and I'll probably remove the requirement entirely soon.

[sgugger]

We shouldn't assume model has a config without proper test, having Trainer work with models that are not PreTrainedModels is a feature that has been asked. If there is an access to config that makes the code fail, we should fix that place.

[sshleifer]

It's already assumed that model.config exists. The base trainer.py accesses model.config 23 times, including in the statement below this one

https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L682

[sshleifer]

Seq2SeqTrainer uses model.config 8 times. Mostly pad_token_id to avoid counting padding in the loss func.

[sgugger]

It should add an assert the model is a PreTrainedModel at init just to be clean, then for your specific problem, it should use the function self._actual_model() to grab the config to avoid your error (e.g., self.model.config -> self._actual_model().config).

Trainer is on its way to fully handle models without config, see #7464.

[sshleifer]

OK. I reduced scope of this PR to just the Tensor -> tensor.

[sgugger]

That works for me :-)