Fix Inductor bench BC change #638
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/638
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit ae3e78d with merge base 34b24f7 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
|
||
| try: | ||
| from torch._inductor.utils import do_bench | ||
| except: | ||
| from torch._inductor.runtime.runtime_utils import do_bench | ||
| except ImportError: | ||
| try: | ||
| from torch._inductor.runtime.runtime_utils import do_bench | ||
| except ImportError: | ||
| from torch._inductor.runtime.benchmarking import benchmarker | ||
| do_bench = benchmarker.benchmark |
There was a problem hiding this comment.
Do we need these imports?
There was a problem hiding this comment.
AO needs to work over multiple pytorch versions and this api has been broken multiple times
We can explore getting rid of the usage completely @HDCharles
There was a problem hiding this comment.
Can we not inline the imports in the IF/ELSE blocks later on when they get used (I see some already are?)?
| from torch._inductor.runtime.runtime_utils import do_bench_gpu | ||
| res = do_bench_gpu(lambda: graph.replay(), warmup=warmup, rep=rep, return_mode="median") | ||
| elif TORCH_VERSION_AFTER_2_5 and torch.cuda.is_available(): |
There was a problem hiding this comment.
Why torch.cuda.is_available()?
| from torch._inductor.runtime.runtime_utils import do_bench_gpu | ||
| res = do_bench_gpu(lambda: graph.replay(), warmup=warmup, rep=rep, return_mode="median") | ||
| elif TORCH_VERSION_AFTER_2_5 and torch.cuda.is_available(): |
There was a problem hiding this comment.
Should this be TORCH_VERSION_AFTER_2_4?
| if TORCH_VERSION_AFTER_2_3 and not TORCH_VERSION_AFTER_2_5: | ||
| from torch._inductor.runtime.runtime_utils import do_bench_gpu | ||
| res = do_bench_gpu(lambda: graph.replay(), warmup=warmup, rep=rep, return_mode="median") | ||
| elif TORCH_VERSION_AFTER_2_5 and torch.cuda.is_available(): | ||
| from torch._inductor.runtime.benchmarking import benchmarker | ||
| do_bench_gpu = benchmarker.benchmark_gpu | ||
| res = do_bench_gpu(lambda: graph.replay(), warmup=warmup, rep=rep, return_mode="median") |
There was a problem hiding this comment.
The newest (i.e. from torch._inductor.runtime.benchmarking import benchmarker) block needs to go first, as from my understanding TORCH_VERSION_AFTER_X_Y=True always in fbcode.
|
@msaroufim left some comments, I'm not sure if this resolves the BC changes (sorry if I'm just confused here on what is happening) |
Summary: Sync torchao installs across ExecuTorch install_requirements. torchao is used in: - llama2 - flamingo (via torchtune) - phi3 lora (via torchtune) Currently: - ExecuTorch llama2 installs torchao==0.0.1 - Torchtune installs torchao==0.3.0 - To export flamingo, which contains recent changes in torch and torchtune, we require a recent version, due to torch x torchao bc issue fixed by pytorch/ao#638 This PR adds a torchao commit hash, so that we can use a recent and consistent version of torchao in ExecuTorch. Note that torchao does not have prebuilt macos nightlies, which is why we use a commit hash instead of nightly. Test Plan: Imported from GitHub, without a `Test Plan:` line. ``` ./install_requirements.sh --pybind xnnpack bash examples/models/flamingo/install_requirements.sh python examples/models/flamingo/export_preprocess.py ``` confirm torchao==0.5.0+git0916b5b installed confirm preprocess.pte generated Reviewed By: larryliu0820 Differential Revision: D61925882 Pulled By: lucylq
Fixes #637