Use Xet high performance mode for Transformers v5

[hmellor]

Enable high performance mode for Xet when Transformers v5 is installed, falling back to hf_transfer for Transformers v4. This change optimizes performance based on the installed version of the Transformers library.

[gemini-code-assist]

Code Review

This pull request updates the logic for enabling high-performance download modes in huggingface_hub to support both Transformers v4 and v5. The change correctly detects the huggingface_hub version and enables either Xet high-performance mode (for v5) or hf_transfer (for v4). However, the implementation for enabling Xet mode unconditionally sets HF_XET_HIGH_PERFORMANCE to True, which overrides any user-defined environment variable setting. I've suggested a change to respect the user's environment configuration, making it consistent with the existing logic for hf_transfer.

[DarkLight1337]

Though I'm not sure whether we should force users to enable this mode

[hmellor]

In v4 we would force hf_transfer if it was installed, but didn't want to force the extra dependency.

For v5 I think this is safe because all it does is boost the number of CPU cores used in order to saturate network/disk bandwidth while downloading models. (docs). Since it doesn't require an extra dependency and can still be disabled by users who really don't want it, it should be ok.