Skip to content

Add optimization level for tuning strategy #179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
yiliu30 wants to merge 53 commits into from
Closed

Add optimization level for tuning strategy #179

yiliu30 wants to merge 53 commits into from

Conversation

@yiliu30
Copy link
Contributor

@yiliu30 yiliu30 commented Nov 28, 2022 •
edited
Loading

Type of Change

  • Feature
  • API changed or not: Add optimization_level to select the tuning strategy

Description

detail description
JIRA ticket: ILITV-2358

Expected Behavior & Potential Risk

  • Add conservative traverse logic
    - Start with fp32 model, convert op(s) into lower-precision(bf16/fp16/int8).
    - Convert ops into lower precision by op-type wise.
    - Convert op into lower precision one by one based on the result of the previous step.
  • Add optimization level
    - 0: using conservative strategy
    - 1: using basic strategy by default
  • UTs
  • Doc, will be submitted by another PR

How has this PR been tested?

Dependency Change?

None

@yiliu30 yiliu30 changed the title Ly/opt level add conservative tuning strategy Nov 28, 2022
@yiliu30 yiliu30 changed the title add conservative tuning strategy [Strategy] Add optimization level for tuning strategy Dec 1, 2022
@yiliu30 yiliu30 marked this pull request as ready for review December 5, 2022 03:30
@yiliu30 yiliu30 added this to the v2.0 milestone Dec 6, 2022
ftian1
ftian1 reviewed Dec 6, 2022
View reviewed changes
neural_compressor/strategy/conservative.py
# self.best_tune_result = self.baseline
# Initialize the best qmodel as fp32 model
self.best_qmodel = self._fp32_model
# self.best_qmodel = self._fp32_model
Copy link
Contributor

@ftian1 ftian1 Dec 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this line gets commented off?

Copy link
Contributor Author

@yiliu30 yiliu30 Dec 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will initialize the self.best_qmodel with the first tuning result whose accuracy meets the requirements. code
If we initialize the self.best_qmodel with the fp32 model, we may return the fp32 model as the tuning result which may confuse the user, right?

chensuyue and others added 23 commits December 7, 2022 09:25
@chensuyue @WenjiaoYue @yiliu30
support pydocstyle check (#151)
1a1c80a 
Signed-off-by: chensuyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@lvliang-intel @yiliu30
Fix Time_series_LSTM tuning issue (#163)
03cceb6 
* Fix Time_series_LSTM tuning issue

Signed-off-by: Lv, Liang1 <[email protected]>

* fix pylint issue

Signed-off-by: Lv, Liang1 <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@lvliang-intel @yiliu30
Enable dilated contraction fusion for ITEX (#146)
54af942 
Signed-off-by: Lv, Liang1 <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@ChendaLi-Intel @yiliu30
Enable QDQ-adapted _QuantizedFusedBatchNorm+LeakyRelu fusion (#160)
1ae2f36 
Signed-off-by: Li, Chenda <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@chensuyue @yiliu30
fix pylint issue (#169)
7bcc400 
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@changwangss @yiliu30
fix ipex quantization (#168)
2d0e453 
Signed-off-by: Wang, Chang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@XuehaoSun @yiliu30
Fix Model-Test pipeline status (#167)
f736360 
Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@PenghuiCheng @yiliu30
Improved new API (#142)
791b0e2 
Signed-off-by: yiliu30 <[email protected]>
@WenjiaoYue @yiliu30
IO docs refine (#135)
83b56a6 
Signed-off-by: yiliu30 <[email protected]>
@ChendaLi-Intel @yiliu30
Enable _MklFusedBatchMatMulV2 BF16 conversion (#147)
d7c363c 
Signed-off-by: Li, Chenda <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@WenjiaoYue @yiliu30
modify IO style (#170)
a33adc2 
Signed-off-by: Yue, Wenjiao <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
feat(strategy): add optimization level 0
77c38ca 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
feat(st): update compare logic
0b69174 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
feat(st): update best model based on perf and acc
8020881 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
feat(st): add optimization_level
aedf02c 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
fixed some bugs
70a724d 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
update the op queue
ce4422e 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add optype-wise conversion
61f07f7 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add one by one conversion
40d907c 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
WA for save q model
84e421e 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
fix some bugs
c8207f9 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
remove some comments
9ee9203 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
update the model with best performance
f9cf333 
Signed-off-by: yiliu30 <[email protected]>
lvliang-intel and others added 18 commits December 7, 2022 09:30
@lvliang-intel @yiliu30
Disable multi instance for ITEX GPU benchmark (#204)
0ffdc5e 
Signed-off-by: Lv, Liang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@intel-zhangyi @yiliu30
Revert "remove op-wise cfgs for testing. (#1521)" (#202)
32e7615 
Signed-off-by: intel-zhangyi <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@changwangss @yiliu30
add examples for GPTJ (#162)
cc45329 
Signed-off-by: Wang, Chang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@kaikaiyao @yiliu30
Neural Coder mod launcher arg: "strategy" to "approach" (#201)
4a87b5b 
Signed-off-by: yiliu30 <[email protected]>
@chensuyue @yiliu30
update publication_list.md (#212)
bf0a33f 
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@XinyuYe-Intel @yiliu30
Added distributed training support for distillation of CNN-2. (#208)
518cd15 
Signed-off-by: Xinyu Ye <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@XinyuYe-Intel @yiliu30
Added distributed training support for distillation of MobileNetV2. (#…
3c482f8 
…166)

Signed-off-by: Xinyu Ye <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@changwangss @yiliu30
fix load issue (#194)
d7213db 
Signed-off-by: changwa1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@lvliang-intel @yiliu30
Fix NTM-One-Shot failed with KeyError (#210)
e283b27 
Signed-off-by: Lv, Liang1 <[email protected]>

Signed-off-by: Lv, Liang1 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@lvliang-intel @sys-lpot-val @yiliu30
Fix TextRNN and centernet_hg104 tuning issue (#171)
d1d375f 
* Fix centernet_hg104 tuning issue

Signed-off-by: sys-lpot-val <[email protected]>

* Fix TextRNN tuning issue

Signed-off-by: sys-lpot-val <[email protected]>

Signed-off-by: sys-lpot-val <[email protected]>
Signed-off-by: Lv, Liang1 <[email protected]>
Co-authored-by: sys-lpot-val <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add strategy kwargs for new api
2e6e32c 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add uts
8b41f0c 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
remove the line for debug
ddc188c 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
fixed some uts
d6c8fc6 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add optimization_level in BaseQuantizationConfig
2e189d8 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add optimization_level to conf and pythonic_conf
e1cfe60 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
add UTs based on new API
aa37294 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30
remove the UTs based on old API(yaml)
ab0ce74 
Signed-off-by: yiliu30 <[email protected]>
@yiliu30 yiliu30 changed the title [Strategy] Add optimization level for tuning strategy Add optimization level for tuning strategy Dec 7, 2022
@yiliu30 yiliu30 removed this from the v2.0 milestone Dec 7, 2022
@yiliu30
Copy link
Contributor Author

yiliu30 commented Dec 7, 2022

The branch was polluted, the feature will be added by PR 227, close it first.

@yiliu30 yiliu30 closed this Dec 7, 2022
@yiliu30 yiliu30 deleted the ly/opt_level branch December 13, 2022 10:10
VincyZhang pushed a commit that referenced this pull request Feb 12, 2023
@PenghuiCheng
Fixed load best model error in transformers trainer (#179)
9dc808d
yiliu30 pushed a commit that referenced this pull request Apr 5, 2025
[SW-222220] Quantising Llama3.2. 11B/90B fails with GC error (#179)
3200ca9 
* refine PatchVLLMKVCache

* move cache out of args

* revert option2

* add get_cache

* Revert "add get_cache"

This reverts commit a89d9d23810ce594743504fea4bc5cd49e8d4192.

* Revert "revert option2"

This reverts commit d2b124c1d30717baf482eb887ba5ab3cb09ac51d.

* add comments

* update comment

* Dummy commit for triggering CI

* Dummy commit for triggering CI
xin3he pushed a commit that referenced this pull request Apr 22, 2025
@xinhe3
[SW-222220] Quantising Llama3.2. 11B/90B fails with GC error (#179)
a421ae8 
* refine PatchVLLMKVCache

* move cache out of args

* revert option2

* add get_cache

* Revert "add get_cache"

This reverts commit a89d9d23810ce594743504fea4bc5cd49e8d4192.

* Revert "revert option2"

This reverts commit d2b124c1d30717baf482eb887ba5ab3cb09ac51d.

* add comments

* update comment

* Dummy commit for triggering CI

* Dummy commit for triggering CI
XuehaoSun pushed a commit that referenced this pull request May 13, 2025
@XuehaoSun
[SW-222220] Quantising Llama3.2. 11B/90B fails with GC error (#179)
11294db 
* refine PatchVLLMKVCache

* move cache out of args

* revert option2

* add get_cache

* Revert "add get_cache"

This reverts commit a89d9d23810ce594743504fea4bc5cd49e8d4192.

* Revert "revert option2"

This reverts commit d2b124c1d30717baf482eb887ba5ab3cb09ac51d.

* add comments

* update comment

* Dummy commit for triggering CI

* Dummy commit for triggering CI
This was referenced Dec 9, 2025
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ftian1 ftian1 ftian1 left review comments

Assignees

No one assigned

Labels

new feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.