Update readme files for explicit lazy mode by jasi306 · Pull Request #1921 · huggingface/optimum-habana

jasi306 · 2025-04-10T14:53:38Z

What does this PR do?

Fixes # (issue)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

HuggingFaceDocBuilderDev · 2025-04-10T14:57:34Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

libinta

can you also check if the fast/slow test needs to be changed to add this env?

emascarenhas · 2025-04-16T17:20:34Z

@jasi306 , Please resolve conflicts.

Also did you already check if hpu graphs are turned on then the lazy mode flag is turned on for any tests in the readme.

emascarenhas · 2025-04-23T16:06:03Z

@jasi306 , Please resolve conflicts.

emascarenhas · 2025-05-07T16:26:00Z

@jasi306 , Please resolve conflicts.

imangohari1 · 2025-05-12T19:31:33Z

@jasi306 CC: @libinta @dsocek
this PR does NOT address the changes needed for CI. FYI.
example:

 RUN_SLOW=true python -m pytest --device gaudi2 -s -v tests/test_diffusers.py -k test_attention_slicing_forward_pass

leads to

============================================================================================================================================ short test summary info =============================================================================================================================================
FAILED tests/test_diffusers.py::StableDiffusionInpaintPipelineTests::test_attention_slicing_forward_pass - RuntimeError: HPUGraph class is available in lazy mode only.
FAILED tests/test_diffusers.py::StableDiffusionXLInpaintPipelineTests::test_attention_slicing_forward_pass - RuntimeError: HPUGraph class is available in lazy mode only.
================================================================================================================================= 2 failed, 147 deselected, 4 warnings in 6.18s ==================================================================================================================================
Exception ignored in: <function HPUGraph.__del__ at 0x7f3fda3512d0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/habana_frameworks/torch/hpu/graphs.py", line 139, in __del__
    self.reset()
  File "/usr/local/lib/python3.10/dist-packages/habana_frameworks/torch/hpu/graphs.py", line 136, in reset
    _hpu_C.destroy(self.hpu_graph)
AttributeError: 'HPUGraph' object has no attribute 'hpu_graph'

karol-brejna-i · 2025-05-14T11:44:23Z

@jasi306 , Please resolve conflicts.

I believe the conflicts are resolved now.

karol-brejna-i · 2025-05-14T11:48:31Z

@jasi306 CC: @libinta @dsocek this PR does NOT address the changes needed for CI. FYI. example:

 RUN_SLOW=true python -m pytest --device gaudi2 -s -v tests/test_diffusers.py -k test_attention_slicing_forward_pass

leads to

============================================================================================================================================ short test summary info =============================================================================================================================================
FAILED tests/test_diffusers.py::StableDiffusionInpaintPipelineTests::test_attention_slicing_forward_pass - RuntimeError: HPUGraph class is available in lazy mode only.
FAILED tests/test_diffusers.py::StableDiffusionXLInpaintPipelineTests::test_attention_slicing_forward_pass - RuntimeError: HPUGraph class is available in lazy mode only.

@jasi306 is OOO at this moment. I'll try to follow up.
I am not sure I understand your commen/request. Could you please explain (possibly reach out to me)?

In recent versions of the Gaudi software stack, lazy mode has been disabled by default. This commit explicitly sets the PT_HPU_LAZY_MODE=1 flag to enable it back. - add PT_HPU_LAZY_MODE=1 flag - add missing bash/python specification to markdown blocks - remove redundant PT_HPU_LAZY_MODE=0 flag Author: Jan Kaminski <jkaminski@habana.ai> Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

karol-brejna-i · 2025-05-29T09:48:39Z

@pbielak rebased the PR, resolved conflicts and improved readability (language declaration in code blocks). Similar change for slow tests is already merged.

@regisss , please take a look.

pbielak · 2025-06-03T11:06:33Z

@regisss kind reminder :)

regisss

LGTM

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

* Merge v1.18-release * Hot fix regional compilation (huggingface#2005) Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> * Enable mixtral 8x7b accuracy evaluation (huggingface#1986) Co-authored-by: Rafal <rbogdanowicz@habana.ai> * Update readme files for explicit lazy mode (huggingface#1921) Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com> * [llama-vision] Remove token_idx_cpu parameter (huggingface#2018) Integer parameter token_idx_cpu passed to mllama's forward() method caused an issue with hpu graph cache which led to performance drop. Signed-off-by: Urszula <urszula.golowicz@intel.com> * Update README examples (huggingface#2020) * Fix examples in README audio-classification: - add space between "False" and backslash image-to-text: - add "datasets" to requirements.txt pytorch-image-models: - add "datasets" to requirements.txt sentence-transformers-training/nli: - add command to properly discover HABANA_VISIBLE_MODULES sentence-transformers-training/sts: - add command to properly discover HABANA_VISIBLE_MODULES speech-recognition: - add `--trust_remote_code` for seq2seq examples stable-diffusion/training: - add missing OpenCV requirement for ControlNet Training Co-authored-by: Karol Brejna <karol.brejna@intel.com> * Review fixes: remove grabbing all modules --------- Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: karol-brejna-i <karolbrejna@apache.org> * Pin latest optimum to force mutual updates (huggingface#2016) pin latest optimum to force mutual updates * Fix FP8 support and address related issues (huggingface#2010) - Resolve bugs related to FP8 (floating point 8-bit) computation - Improve stability and correctness of FP8 operations - Add/fix tests to validate FP8 functionality - Update relevant documentation and comments Co-authored-by: IlyasMoutawwakil --------- Signed-off-by: Urszula <urszula.golowicz@intel.com> Co-authored-by: Adam Stachowicz <astachowicz@habana.ai> Co-authored-by: Ilyas Moutawwakil <57442720+IlyasMoutawwakil@users.noreply.github.com> Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> Co-authored-by: Rafal Bogdanowicz <rafal.bogdanowicz@intel.com> Co-authored-by: Rafal <rbogdanowicz@habana.ai> Co-authored-by: Jan Kamiński <jkaminski@habana.ai> Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com> Co-authored-by: Urszula Golowicz <urszula.golowicz@intel.com> Co-authored-by: Piotr Bielak <pbielak@users.noreply.github.com> Co-authored-by: karol-brejna-i <karolbrejna@apache.org>

jasi306 requested a review from regisss as a code owner April 10, 2025 14:53

This was referenced Apr 10, 2025

[SW-218526] Updated Readme files for explicite lazy mode part2 (#177) #1889

Closed

[SW-218526] Updated Readme files for explicite lazy mode (#174) #1890

Closed

karol-brejna-i added the synapse 1.21 label Apr 11, 2025

libinta reviewed Apr 16, 2025

View reviewed changes

libinta added the run-test Run CI for PRs from external contributors label Apr 22, 2025

karol-brejna-i force-pushed the dev/jasi306/update-test-files-for-explicit-lazy branch from e5cafb6 to 858d09e Compare May 14, 2025 11:40

imangohari1 mentioned this pull request May 16, 2025

fea(): Diffuser upgrade to 0.33.1 #1981

Merged

3 tasks

pbielak force-pushed the dev/jasi306/update-test-files-for-explicit-lazy branch from 858d09e to 39e9089 Compare May 29, 2025 08:08

pbielak force-pushed the dev/jasi306/update-test-files-for-explicit-lazy branch from 39e9089 to cad3eb0 Compare May 29, 2025 09:29

regisss approved these changes Jun 3, 2025

View reviewed changes

regisss merged commit 3bbede7 into huggingface:main Jun 3, 2025
4 checks passed

regisss pushed a commit that referenced this pull request Jun 3, 2025

Update readme files for explicit lazy mode (#1921)

2188aaa

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

pbielak deleted the dev/jasi306/update-test-files-for-explicit-lazy branch June 4, 2025 07:52

astachowiczhabana pushed a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jun 5, 2025

Update readme files for explicit lazy mode (huggingface#1921)

37a3d66

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

astachowiczhabana pushed a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jun 10, 2025

Update readme files for explicit lazy mode (huggingface#1921)

027cfc9

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

astachowiczhabana pushed a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jul 3, 2025

Update readme files for explicit lazy mode (huggingface#1921)

15d66f0

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

astachowiczhabana added a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jul 8, 2025

Update readme files for explicit lazy mode (huggingface#1921)

38f94c1

astachowiczhabana added a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jul 8, 2025

Update readme files for explicit lazy mode (huggingface#1921)

ac38d87

astachowiczhabana added a commit that referenced this pull request Jul 8, 2025

Update readme files for explicit lazy mode (#1921)

a27bc13

astachowiczhabana added a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jul 10, 2025

Update readme files for explicit lazy mode (huggingface#1921)

67baf4a

astachowiczhabana added a commit to HabanaAI/optimum-habana-fork that referenced this pull request Jul 11, 2025

Update readme files for explicit lazy mode (huggingface#1921)

0955a80

astachowiczhabana added a commit that referenced this pull request Aug 22, 2025

Update readme files for explicit lazy mode (#1921)

e3ca86f

astachowiczhabana pushed a commit that referenced this pull request Sep 10, 2025

Update readme files for explicit lazy mode (#1921)

a52091c

Co-authored-by: Karol Brejna <karol.brejna@intel.com> Co-authored-by: Piotr Bielak <piotr.bielak@intel.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update readme files for explicit lazy mode#1921

Update readme files for explicit lazy mode#1921
regisss merged 1 commit into
huggingface:mainfrom
HabanaAI:dev/jasi306/update-test-files-for-explicit-lazy

jasi306 commented Apr 10, 2025 •

edited

Loading

Uh oh!

HuggingFaceDocBuilderDev commented Apr 10, 2025

Uh oh!

libinta left a comment

Uh oh!

emascarenhas commented Apr 16, 2025

Uh oh!

emascarenhas commented Apr 23, 2025

Uh oh!

emascarenhas commented May 7, 2025

Uh oh!

imangohari1 commented May 12, 2025

Uh oh!

karol-brejna-i commented May 14, 2025

Uh oh!

karol-brejna-i commented May 14, 2025

Uh oh!

karol-brejna-i commented May 29, 2025

Uh oh!

pbielak commented Jun 3, 2025

Uh oh!

regisss left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

jasi306 commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Uh oh!

HuggingFaceDocBuilderDev commented Apr 10, 2025

Uh oh!

libinta left a comment

Choose a reason for hiding this comment

Uh oh!

emascarenhas commented Apr 16, 2025

Uh oh!

emascarenhas commented Apr 23, 2025

Uh oh!

emascarenhas commented May 7, 2025

Uh oh!

imangohari1 commented May 12, 2025

Uh oh!

karol-brejna-i commented May 14, 2025

Uh oh!

karol-brejna-i commented May 14, 2025

Uh oh!

karol-brejna-i commented May 29, 2025

Uh oh!

pbielak commented Jun 3, 2025

Uh oh!

regisss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jasi306 commented Apr 10, 2025 •

edited

Loading