Fix unsupported setting of self._n_gpu in training_args on XPU devices

[Liangliang-Ma]

In current training_args, self._n_gpu is set to device_count on XPU device, which will cause crash on XPU devices.
In Trainer, if self.args.n_gpu greater than one, it will utilize torch.nn.DataParallel to wrap the model. But Ipex(intel_extension_for_pytorch) don't support DataParallel, while it suggests using DDP instead.
So to make huggingface Trainer work on intel devices, this fix should be applied.

[ArthurZucker]

Hey! This seems to have been introduce in #25714, and thus I am not convinced that the fix is as simple as that!
Would you mind also sharing a reproducer of the issue? Might be related to specific hard / soft versions

[Liangliang-Ma]

Hey! This seems to have been introduce in #25714, and thus I am not convinced that the fix is as simple as that! Would you mind also sharing a reproducer of the issue? Might be related to specific hard / soft versions

Hi, @ArthurZucker! I have discussed with @abhilash1910 about the issue before this PR and we thought it can be fixed like this. This could be reproduced in a multi intel GPUs env, for the distributedType should be set to MULTI_XPU in accelerator. I think most example scripts using Trainer and more than one gpu should be reproducer.

[abhilash1910]

Thanks @Liangliang-Ma for addressing this.
@ArthurZucker yes the current limitation of pure dp on ipex only allows us to use n_gpu=1 .
Could you help re-trigger the CI test ? Thanks

[ArthurZucker]

Alright thanks all for checking, triggering the CI

[Liangliang-Ma]

@ArthurZucker Seems CI got block by Internet issue. Could you please check with that? Thanks

[ArthurZucker]

Seems like I can't would you mind merging with main to trigger it?!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[ArthurZucker]

Thanks both 🤗