Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Skip Tests for GPUs Not Supporting
bf16
(#159)
## Summary Closes #87 Skipped tests for `bfloat16` on GPUs with compute capability below Ampere architecture (`sm_80`). <!--- This is a required section; please describe the main purpose of this proposed code change. ---> <!--- ## Details This is an optional section; is there anything specific that reviewers should be aware of? ---> ## Testing Done <!--- This is a required section; please describe how this change was tested. ---> <!-- Replace BLANK with your device type. For example, A100-80G-PCIe Complete the following tasks before sending your PR, and replace `[ ]` with `[x]` to indicate you have done them. --> - Hardware Type: NVIDIA **T4** (should skip most cases) - [X] run `make test` to ensure correctness - [X] run `make checkstyle` to ensure code style - [X] run `make test-convergence` to ensure convergence ``` ⚡ main ~/Liger-Kernel make all python -m pytest --disable-warnings test/ --ignore=test/convergence HF_DATASETS_OFFLINE=1 python -m pytest --disable-warnings test/convergence flake8 .; flake8_status=$?; \ isort .; isort_status=$?; \ black .; black_status=$?; \ if [ $flake8_status -ne 0 ] || [ $isort_status -ne 0 ] || [ $black_status -ne 0 ]; then \ exit 1; \ fi =================================================================== test session starts ==================================================================== platform linux -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0 rootdir: /teamspace/studios/this_studio/Liger-Kernel plugins: anyio-4.4.0 collecting ... =================================================================== test session starts ==================================================================== platform linux -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0 rootdir: /teamspace/studios/this_studio/Liger-Kernel plugins: anyio-4.4.0 collecting ... Skipped 1 files All done! ✨ 🍰 ✨ 58 files left unchanged. collected 163 items test/transformers/test_auto_model.py . [ 0%] test/transformers/test_cross_entropy.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 36%] collected 28 items test/convergence/test_mini_models.py .....s.....s.... [ 43%] test/transformers/test_geglu.py .s....ssss [ 48%] test/transformers/test_monkey_patch.py ..... [ 51%] test/transformers/test_rms_norm.py ........ssssssss...............ssssssss........ [ 80%] test/transformers/test_rope.py ......ssssss [ 88%] test/transformers/test_swiglu.py ....ssss.s....ssss [ 98%] test/transformers/test_trainer_integration.py . [ 98%] test/triton/test_triton_monkey_patch.py .. [100%] ======================================================== 71 passed, 92 skipped in 136.69s (0:02:16) ======================================================== .s.s.s [ 50%] test/convergence/test_mini_models_no_logits.py .s.s.s.s.s.s.s [100%] ======================================================== 14 passed, 14 skipped in 353.27s (0:05:53) ======================================================== ``` - Hardware Type: NVIDIA **L4** (should skip few cases) - [X] run `make test` to ensure correctness - [X] run `make checkstyle` to ensure code style - [X] run `make test-convergence` to ensure convergence ``` ⚡ main ~/Liger-Kernel make all python -m pytest --disable-warnings test/ --ignore=test/convergence HF_DATASETS_OFFLINE=1 python -m pytest --disable-warnings test/convergence flake8 .; flake8_status=$?; \ isort .; isort_status=$?; \ black .; black_status=$?; \ if [ $flake8_status -ne 0 ] || [ $isort_status -ne 0 ] || [ $black_status -ne 0 ]; then \ exit 1; \ fi =================================================================== test session starts ==================================================================== platform linux -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0 rootdir: /teamspace/studios/this_studio/Liger-Kernel plugins: anyio-4.4.0 collecting ... =================================================================== test session starts ==================================================================== platform linux -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0 rootdir: /teamspace/studios/this_studio/Liger-Kernel plugins: anyio-4.4.0 collecting ... Skipped 1 files All done! ✨ 🍰 ✨ 58 files left unchanged. collected 163 items test/transformers/test_auto_model.py . [ 0%] collected 28 items test/convergence/test_mini_models.py ........................................................ss [ 36%] test/transformers/test_fused_linear_cross_entropy.py ............... [ 43%] test/transformers/test_geglu.py ......... [ 48%] test/transformers/test_monkey_patch.py ..... [ 51%] test/transformers/test_rms_norm.py ................................................. [ 80%] test/transformers/test_rope.py ............ [ 88%] test/transformers/test_swiglu.py .................. [ 98%] test/transformers/test_trainer_integration.py . [ 98%] test/triton/test_triton_monkey_patch.py .. [100%] ======================================================== 161 passed, 2 skipped in 90.45s (0:01:30) ========================================================= ....... [ 50%] test/convergence/test_mini_models_no_logits.py .............. [100%] ============================================================== 28 passed in 290.65s (0:04:50) ============================================================== ``` ## Additional Context FYR, here’s a list of NVIDIA architecture names, and which compute capabilities they have: <img width="1268" alt="Screenshot 2024-08-29 at 6 04 56 PM" src="https://github.com/user-attachments/assets/6675ae9e-9137-4adb-8af7-ee1226733353"> --------- Signed-off-by: Austin Liu <[email protected]> Co-authored-by: Shao Tang <[email protected]>
- Loading branch information