Ig/diffuser 0.34.0

[imangohari1]

What does this PR do?

This PR upgrades the diffuser versions to the latest (as of date) upstream: https://github.com/huggingface/diffusers/releases

Tests

The following tests have been conducted to make sure the upgrade is functional:

• g2: CI#184
• g3: CI#49
• g2: example/stable-diffusion/README.md
• g2: example/stable-diffusion/training/README.md

Notes:

• All tests passes.
• details: internal ticket HS-6662

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[imangohari1]

@astachowiczhabana @libinta @dsocek. FYI on this

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[astachowiczhabana]

Thank you @imangohari1
Since there's almost no changes. I think we can merge it. @regisss can you please approve?
On synapse 1.22 fast test + fast tests diffusers passed

[regisss]

LGTM

[yafshar]

@regisss @imangohari1 this upgrade will break the current trl which requires peft==0.12.0. The new diffuser requires peft>=0.15.0
We should revert this change until upgrade the peft requirements

[imangohari1]

@regisss @imangohari1 this upgrade will break the current trl which requires peft==0.12.0. The new diffuser requires peft>=0.15.0 We should revert this change until upgrade the peft requirements

@yafshar thanks for checking on this but I don't believe that's the case:

• the OH v1.18.0 installs the peft 0.16.0 (below).

• trl should install the peft 0.12.0: https://github.com/huggingface/optimum-habana/blob/main/examples/trl/requirements.txt#L2

please make sure your setup is correct. I don't believe a revert is needed.
thanks.

[yafshar]

@imangohari1 you can try this example after main install

python -m pytest tests/test_examples.py::MultiCardSFTChatExampleTester::test_sft_Qwen2-7B_multi_card

this is the error

Traceback (most recent call last):
  File "/root/optimum-habana/examples/trl/sft.py", line 19, in <module>
    from optimum.habana.trl import GaudiSFTConfig, GaudiSFTTrainer
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/__init__.py", line 1, in <module>
    from .models.modeling_base import adapt_PreTrainedModelWrapper_to_gaudi
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/models/__init__.py", line 21, in <module>
    from .modeling_sd_base import (
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/models/modeling_sd_base.py", line 19, in <module>
    from diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion import rescale_noise_cfg
  File "/usr/local/lib/python3.12/dist-packages/diffusers/__init__.py", line 5, in <module>
    from .utils import (
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/__init__.py", line 21, in <module>
    from .constants import (
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/constants.py", line 59, in <module>
    dep_version_check("peft")
  File "/usr/local/lib/python3.12/dist-packages/diffusers/dependency_versions_check.py", line 34, in dep_version_check
    require_version(deps[pkg], hint)
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: peft>=0.15.0 is required for a normal functioning of this module, but found peft==0.12.0.

[imangohari1]

@imangohari1 you can try this example after main install

python -m pytest tests/test_examples.py::MultiCardSFTChatExampleTester::test_sft_Qwen2-7B_multi_card

this is the error

Traceback (most recent call last):
  File "/root/optimum-habana/examples/trl/sft.py", line 19, in <module>
    from optimum.habana.trl import GaudiSFTConfig, GaudiSFTTrainer
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/__init__.py", line 1, in <module>
    from .models.modeling_base import adapt_PreTrainedModelWrapper_to_gaudi
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/models/__init__.py", line 21, in <module>
    from .modeling_sd_base import (
  File "/usr/local/lib/python3.12/dist-packages/optimum/habana/trl/models/modeling_sd_base.py", line 19, in <module>
    from diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion import rescale_noise_cfg
  File "/usr/local/lib/python3.12/dist-packages/diffusers/__init__.py", line 5, in <module>
    from .utils import (
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/__init__.py", line 21, in <module>
    from .constants import (
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/constants.py", line 59, in <module>
    dep_version_check("peft")
  File "/usr/local/lib/python3.12/dist-packages/diffusers/dependency_versions_check.py", line 34, in dep_version_check
    require_version(deps[pkg], hint)
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/usr/local/lib/python3.12/dist-packages/diffusers/utils/versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: peft>=0.15.0 is required for a normal functioning of this module, but found peft==0.12.0.

You likely are missing this: https://github.com/huggingface/optimum-habana/blob/main/Makefile#L167-L170
can you please do this and retry? tnx

python -m pip install trl==0.9.6
python -m pip install peft==0.12.0

[yafshar]

the same issue. The trl version and peft are correct, but the new diffuser requires different peft version which causes this issue. There is an import inside trl

[imangohari1]

@yafshar Thanks. I understand the problem now better.

@regisss @astachowiczhabana
so the issue that Yaser is poining out comes from the min req change in peft in diffuser upstream:

• diffuser 0.33.1 had a min req for peft=0.6.0: https://github.com/huggingface/diffusers/blob/v0.33.1/setup.py#L119
• which is updated to at least 0.15.0 in 0.34.0: https://github.com/huggingface/diffusers/blob/v0.34.0/setup.py#L119

this leads to a dependency issue for trl:

• our trl version 0.9.X is from 2024, and currently we are using peft=0.12.0 for it: https://github.com/huggingface/optimum-habana/blob/main/examples/trl/requirements.txt
• trl uses a base module from diffuser https://github.com/huggingface/trl/blob/main/trl/models/modeling_sd_base.py#L26 ... so with v0.34.0 min peft is not met.

here are 3 solutions that I can think of:

• to revert this PR (least ideal but we can re-merge this later after trl update here: https://github.com/huggingface/optimum-habana/pull/2130/files)
• to pin the diffusers to 0.33.1 for trl till it's update is completed.

diff --git a/examples/trl/requirements.txt b/examples/trl/requirements.txt
index 3a9be362..0f26d035 100644
--- a/examples/trl/requirements.txt
+++ b/examples/trl/requirements.txt
@@ -4,3 +4,4 @@ datasets == 2.19.2
 tyro
 evaluate
 scikit-learn == 1.5.2
+diffusers == 0.33.1

• I have confirmed this works with RUN_SLOW=True python -m pytest tests/test_examples.py::MultiCardSFTChatExampleTester::test_sft_Qwen2-7B_multi_card -s -v but it shows a pip depencdeny error

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
optimum-habana 1.19.0.dev0 requires diffusers<0.34.1,>=0.34.0, but you have diffusers 0.33.1 which is incompatible.

• to update the trl peft version to 0.15.0 since https://github.com/huggingface/trl/blob/main/setup.cfg suggest anything that 0.8.0 is fine.

diff --git a/examples/trl/requirements.txt b/examples/trl/requirements.txt
index 3a9be362..502a2d99 100644
--- a/examples/trl/requirements.txt
+++ b/examples/trl/requirements.txt
@@ -1,5 +1,5 @@
 trl == 0.9.6
-peft == 0.12.0
+peft == 0.15.0
 datasets == 2.19.2
 tyro

• I have confirmed this works with RUN_SLOW=True python -m pytest tests/test_examples.py::MultiCardSFTChatExampleTester::test_sft_Qwen2-7B_multi_card -s -v and perf is unchanged, but likely needs more testing.

Let me know which one you think is the best and I open a PR for it.
Thank you.

[regisss]

Well since this PR has very small changes, I think it's okay to revert it and merge it again once TRL is updated. WDYT?

[imangohari1]

• https://github.com/huggingface/trl/blob/main/setup.cfg suggest anything that 0.8.0 is fine

Hi regiss,
I understand your viewpoint but I did test the CI and examples of trl with updated peft and they are all working well.
#2160

I think it is more useful to update that version versus downgrading the diffusers IMHO. WDYT?

[regisss]

I'm okay with that as well