Revert to older transformers version

[loadams]

Recent changes in logging level in HF transformers broke DeepSpeed where we relied on certain logging prints that they changed the defaults on:. There is currently a PR out that provides a workaround to keep backwards compatibility.

[stas00]

apologies for this happening, as transformers can't afford to run deepspeed tests other than nightly runs - so such breakages get discovered usually on the next day.

HF Transformers live CI only runs non-gpu tests.

I'm just waiting for an approval to merge the fix.

But yes please always revert to some 2-day older version to quickly unblock your CI while the fix is in works.

[loadams]

apologies for this happening, as transformers can't afford to run deepspeed tests other than nightly runs - so such breakages get discovered usually on the next day.

HF Transformers live CI only runs non-gpu tests.

I'm just waiting for an approval to merge the fix.

But yes please always revert to some 2-day older version to quickly unblock your CI.

No problem at all, glad you had a PR out on that side when I looked. We can probably afford to wait for the changes - thoughts @mrwyattii?

[stas00]

ok, got approval, just waiting for our CI to finish and will merge then.

[stas00]

ok, merged. please let me know if you run into any issues.

I run tests locally, reproduced the problem you got on your CI and fixed it.

[stas00]

oh, I guess since I merged my branch and it automatically got deleted, your test with my branch failed:

Run if [[ -d ./torch-extensions ]]; then rm -rf ./torch-extensions; fi
Cloning into 'transformers'...
error: pathspec 'ds-fix-tests' did not match any file(s) known to git.
Error: Process completed with exit code 1.

normal main branch should be fine now.

[stas00]

best is to just restart CI on some existing DS PR and see if all works now.

[loadams]

Testing here

[loadams]

Looks like its passing now, thanks @stas00!