Add Qwen3 2507 Instruct models - #2125
Merged
Merged
Lit OSS [bot] / tests.yaml / Lit Job (compiler)
succeeded
Sep 8, 2025 in 25m 28s
Job finished
Job ci-run_Lightning-AI-litgpt-cb52953-event-812aca70-tests.yaml_/Lit_Job(compiler) finished as Completed with exit code 0.
Details
nvidia:
Mon Sep 8 19:09:53 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.172.08 Driver Version: 570.172.08 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L4 Off | 00000000:00:04.0 Off | 0 |
| N/A 40C P8 11W / 72W | 0MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA L4 Off | 00000000:00:05.0 Off | 0 |
| N/A 40C P8 12W / 72W | 0MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
Python 3.10.18
pip 25.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
Package Version
------------------------ ------------------
absl-py 2.3.1
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
apex 0.1
asv 0.6.4
asv_runner 0.2.1
asvdb 0.4.2
async-timeout 5.0.1
attrs 25.3.0
bitsandbytes 0.46.1
blinker 1.7.0
boto3 1.40.6
botocore 1.40.6
build 1.3.0
certifi 2025.8.3
charset-normalizer 3.4.3
coverage 7.9.2
cryptography 41.0.7
dbus-python 1.3.2
distlib 0.4.0
distro 1.9.0
docstring_parser 0.17.0
einops 0.8.1
exceptiongroup 1.3.0
execnet 2.1.1
expecttest 0.3.0
fdm 0.5.0
filelock 3.13.1
frozenlist 1.7.0
fsspec 2024.6.1
graphviz 0.21
hf-xet 1.1.7
httplib2 0.20.4
huggingface-hub 0.34.4
hypothesis 6.136.9
idna 3.10
importlib_metadata 8.7.0
importlib_resources 6.5.2
iniconfig 2.1.0
jax 0.6.2
jaxlib 0.6.2
Jinja2 3.1.4
jmespath 1.0.1
json5 0.12.0
jsonargparse 4.32.1
launchpadlib 1.11.0
lazr.restfulclient 0.14.6
lazr.uri 1.0.6
lightning 2.4.0
lightning-utilities 0.15.2
litgpt 0.5.0
MarkupSafe 2.1.5
ml_dtypes 0.5.3
mpmath 1.3.0
multidict 6.6.3
networkx 3.3
numpy 1.26.4
nvfuser_cu126_torch27 0.2.28.dev20250801
nvidia-cublas-cu12 12.6.4.1
nvidia-cuda-cupti-cu12 12.6.80
nvidia-cuda-nvrtc-cu12 12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12 9.5.1.17
nvidia-cudnn-frontend 1.10.0
nvidia-cufft-cu12 11.3.0.4
nvidia-cufile-cu12 1.11.1.6
nvidia-curand-cu12 10.3.7.77
nvidia-cusolver-cu12 11.7.1.2
nvidia-cusparse-cu12 12.5.4.2
nvidia-cusparselt-cu12 0.6.3
nvidia-nccl-cu12 2.26.2
nvidia-nvjitlink-cu12 12.6.85
nvidia-nvtx-cu12 12.6.77
oauthlib 3.2.2
opt_einsum 3.4.0
packaging 24.2
pandas 2.3.1
pip 25.2
platformdirs 4.3.8
pluggy 1.6.0
propcache 0.3.2
py-cpuinfo 9.0.0
PyGObject 3.48.2
PyJWT 2.7.0
Pympler 1.1
pyparsing 3.1.1
pyproject_hooks 1.2.0
pytest 8.3.5
pytest-benchmark 5.1.0
pytest-cov 6.2.1
pytest-random-order 1.2.0
pytest-timeout 2.4.0
pytest-timestamper 0.0.10
pytest-xdist 3.7.0
python-apt 2.7.7+ubuntu5
python-dateutil 2.9.0.post0
pytorch-lightning 2.5.2
pytz 2025.2
PyYAML 6.0.2
regex 2025.7.34
requests 2.32.4
s3transfer 0.13.1
safetensors 0.6.2
scipy 1.15.3
setuptools 80.9.0
six 1.16.0
sortedcontainers 2.4.0
sympy 1.13.3
tabulate 0.9.0
tokenizers 0.21.4
tomli 2.2.1
torch 2.7.1+cu126
torchmetrics 1.8.1
tqdm 4.67.1
transformers 4.52.4
triton 3.3.1
typeshed_client 2.8.2
typing_extensions 4.12.2
tzdata 2025.2
urllib3 2.5.0
virtualenv 20.33.1
wadllib 1.3.6
wheel 0.45.1
xlsxwriter 3.2.5
yarl 1.20.1
zipp 3.23.0
+ pip install -q '.[extra,test]' torch==2.7.1 cffi -U
DEPRECATION: Building 'pytest-dependency' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible
↪ replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of
↪ 'pytest-dependency'. Discussion can be found at https://github.com/pypa/pip/issues/6334
DEPRECATION: Building 'rouge-score' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is
↪ to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'rouge-score'.
↪ Discussion can be found at https://github.com/pypa/pip/issues/6334
DEPRECATION: Building 'sqlitedict' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is
↪ to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'sqlitedict'.
↪ Discussion can be found at https://github.com/pypa/pip/issues/6334
DEPRECATION: Building 'word2number' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is
↪ to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'word2number'.
↪ Discussion can be found at https://github.com/pypa/pip/issues/6334
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a
↪ virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
+ '[' compiler == compiler ']'
+ pip uninstall -y torchvision torchaudio
Found existing installation: torchvision 0.22.1
Uninstalling torchvision-0.22.1:
Successfully uninstalled torchvision-0.22.1
WARNING: Skipping torchaudio as it is not installed.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a
↪ virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
+ pip install -q '.[compiler,extra,test]' torch==2.7.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a
↪ virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
+ python -c 'from thunder.executors import nvfuser_available ; assert nvfuser_available(), '\''nvFuser is missing!'\'''
/usr/local/lib/python3.10/dist-packages/nvfuser_direct/__init__.py:9: UserWarning: Be careful! You've imported nvfuser_direct when the nvfuser module is already imported.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/thunder/executors/transformer_engine_v2ex.py:25: UserWarning: transformer_engine module not found!
warnings.warn("transformer_engine module not found!")
+ python -c 'from thunder.executors.triton_utils import triton_version ; assert triton_version() is not None, '\''triton is missing!'\'''
/usr/local/lib/python3.10/dist-packages/nvfuser_direct/__init__.py:9: UserWarning: Be careful! You've imported nvfuser_direct when the nvfuser module is already imported.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/thunder/executors/transformer_engine_v2ex.py:25: UserWarning: transformer_engine module not found!
warnings.warn("transformer_engine module not found!")
+ pip list
Package Version
------------------------ ------------------
absl-py 2.3.1
accelerate 1.10.1
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
annotated-types 0.7.0
anyio 4.10.0
apex 0.1
asv 0.6.4
asv_runner 0.2.1
asvdb 0.4.2
async-timeout 5.0.1
attrs 25.3.0
bitsandbytes 0.46.1
blinker 1.7.0
boto3 1.40.6
botocore 1.40.6
build 1.3.0
certifi 2025.8.3
cffi 1.17.1
chardet 5.2.0
charset-normalizer 3.4.3
click 8.2.1
colorama 0.4.6
coverage 7.9.2
cryptography 41.0.7
DataProperty 1.1.0
datasets 3.6.0
dbus-python 1.3.2
dill 0.3.8
distlib 0.4.0
distro 1.9.0
docstring_parser 0.17.0
einops 0.8.1
evaluate 0.4.5
exceptiongroup 1.3.0
execnet 2.1.1
expecttest 0.3.0
fastapi 0.116.1
fdm 0.5.0
filelock 3.13.1
frozenlist 1.7.0
fsspec 2024.6.1
graphviz 0.21
grpcio 1.74.0
h11 0.16.0
hf_transfer 0.1.9
hf-xet 1.1.7
httplib2 0.20.4
httptools 0.6.4
huggingface-hub 0.34.4
hypothesis 6.136.9
idna 3.10
importlib_metadata 8.7.0
importlib_resources 6.5.2
iniconfig 2.1.0
jax 0.6.2
jaxlib 0.6.2
Jinja2 3.1.4
jmespath 1.0.1
joblib 1.5.2
json5 0.12.0
jsonargparse 4.41.0
jsonlines 4.0.0
launchpadlib 1.11.0
lazr.restfulclient 0.14.6
lazr.uri 1.0.6
lightning 2.5.5
lightning-thunder 0.2.5.dev20250907
lightning-utilities 0.15.2
litdata 0.2.51
litgpt 0.5.10
litserve 0.2.15
lm_eval 0.4.9
looseversion 1.3.0
lxml 6.0.1
Markdown 3.9
MarkupSafe 2.1.5
mbstrdecoder 1.1.4
ml_dtypes 0.5.3
more-itertools 10.8.0
mpmath 1.3.0
multidict 6.6.3
multiprocess 0.70.16
networkx 3.3
nltk 3.9.1
numexpr 2.11.0
numpy 1.26.4
nvfuser_cu126_torch27 0.2.28.dev20250801
nvidia-cublas-cu12 12.6.4.1
nvidia-cuda-cupti-cu12 12.6.80
nvidia-cuda-nvrtc-cu12 12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12 9.5.1.17
nvidia-cudnn-frontend 1.10.0
nvidia-cufft-cu12 11.3.0.4
nvidia-cufile-cu12 1.11.1.6
nvidia-curand-cu12 10.3.7.77
nvidia-cusolver-cu12 11.7.1.2
nvidia-cusparse-cu12 12.5.4.2
nvidia-cusparselt-cu12 0.6.3
nvidia-nccl-cu12 2.26.2
nvidia-nvjitlink-cu12 12.6.85
nvidia-nvtx-cu12 12.6.77
oauthlib 3.2.2
obstore 0.8.1
opt_einsum 3.4.0
optree 0.17.0
packaging 24.2
pandas 2.3.1
pathvalidate 3.3.1
peft 0.17.1
pillow 11.3.0
pip 25.2
platformdirs 4.3.8
pluggy 1.6.0
portalocker 3.2.0
propcache 0.3.2
protobuf 6.32.0
psutil 7.0.0
py-cpuinfo 9.0.0
pyarrow 21.0.0
pybind11 3.0.1
pycparser 2.22
pydantic 2.11.7
pydantic_core 2.33.2
PyGObject 3.48.2
PyJWT 2.7.0
Pympler 1.1
pyparsing 3.1.1
pyproject_hooks 1.2.0
pytablewriter 1.2.1
pytest 8.3.5
pytest-benchmark 5.1.0
pytest-cov 6.2.1
pytest-dependency 0.6.0
pytest-random-order 1.2.0
pytest-rerunfailures 16.0.1
pytest-timeout 2.4.0
pytest-timestamper 0.0.10
pytest-xdist 3.7.0
python-apt 2.7.7+ubuntu5
python-dateutil 2.9.0.post0
python-dotenv 1.1.1
pytorch-lightning 2.5.2
pytz 2025.2
PyYAML 6.0.2
pyzmq 27.0.2
regex 2025.7.34
requests 2.32.4
rouge_score 0.1.2
s3transfer 0.13.1
sacrebleu 2.5.1
safetensors 0.6.2
scikit-learn 1.7.1
scipy 1.15.3
sentencepiece 0.2.1
setuptools 80.9.0
six 1.16.0
sniffio 1.3.1
sortedcontainers 2.4.0
sqlitedict 2.1.0
starlette 0.47.3
sympy 1.13.3
tabledata 1.3.4
tabulate 0.9.0
tcolorpy 0.1.7
tensorboard 2.20.0
tensorboard-data-server 0.7.2
threadpoolctl 3.6.0
tifffile 2025.5.10
tokenizers 0.21.4
tomli 2.2.1
torch 2.7.1+cu126
torchmetrics 1.8.1
torchvision 0.22.1
tqdm 4.67.1
tqdm-multiprocess 0.0.11
transformers 4.52.4
triton 3.3.1
typepy 1.3.4
typeshed_client 2.8.2
typing_extensions 4.12.2
typing-inspection 0.4.1
tzdata 2025.2
urllib3 2.5.0
uvicorn 0.35.0
uvloop 0.21.0
virtualenv 20.33.1
wadllib 1.3.6
watchfiles 1.1.0
websockets 15.0.1
Werkzeug 3.1.3
wheel 0.45.1
word2number 1.1
xlsxwriter 3.2.5
xxhash 3.5.0
yarl 1.20.1
zipp 3.23.0
zstandard 0.24.0
+ python -c 'import torch ; gpus = torch.cuda.device_count() ; assert gpus >= 2, f'\''GPU: {gpus}'\'''
+ python -c 'from torch import __version__ as ver ; assert str(ver).split('\''+'\'')[0] == '\''2.7.1'\'', f'\''PyTorch: installed {ver} but expected 2.7.1'\'''
+ pytest -v --durations=100
============================= test session starts ==============================
platform linux -- Python 3.10.18, pytest-8.3.5, pluggy-1.6.0 -- /usr/bin/python3.10
cachedir: .pytest_cache
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
hypothesis profile 'default'
rootdir: /workspace
configfile: pyproject.toml
plugins: anyio-4.10.0, dependency-0.6.0, rerunfailures-16.0.1, timestamper-0.0.10, xdist-3.7.0, random-order-1.2.0, cov-6.2.1, benchmark-5.1.0, hypothesis-6.136.9, timeout-2.4.0
collecting ...
The number of tests has been filtered from 2129 to 587 after the filters ['RUN_ONLY_CUDA_TESTS'].
2 tests are marked as unconditional skips.
In total, 585 tests will run.
collected 2129 items
[2025-09-08 19:10:53] tests/convert/test_lit_checkpoint.py::test_against_original_gemma[device1-dtype1-gemma-2b] XFAIL [ 0%]
[2025-09-08 19:10:56] tests/convert/test_lit_checkpoint.py::test_against_original_gemma[device1-dtype1-gemma-7b] XFAIL [ 0%]
[2025-09-08 19:10:57] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_2[device1-dtype1-gemma-2-2b] XFAIL [ 0%]
[2025-09-08 19:10:59] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_2[device1-dtype1-gemma-2-9b] XFAIL [ 0%]
[2025-09-08 19:11:01] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_2[device1-dtype1-gemma-2-27b] XFAIL [ 0%]
[2025-09-08 19:11:02] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_3[device1-dtype1-gemma-3-1b-it] XFAIL [ 1%]
[2025-09-08 19:11:04] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_3[device1-dtype1-gemma-3-4b-it] XFAIL [ 1%]
[2025-09-08 19:11:06] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_3[device1-dtype1-gemma-3-12b-it] XFAIL [ 1%]
[2025-09-08 19:11:07] tests/convert/test_lit_checkpoint.py::test_against_original_gemma_3[device1-dtype1-gemma-3-27b-it] XFAIL [ 1%]
[2025-09-08 19:11:09] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-1.5B] XFAIL [ 1%]
[2025-09-08 19:11:10] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-Coder-1.5B] XFAIL [ 1%]
[2025-09-08 19:11:11] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-Math-1.5B] XFAIL [ 2%]
[2025-09-08 19:11:12] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-QwQ-32B-Preview] XFAIL [ 2%]
[2025-09-08 19:11:13] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-QwQ-32B] XFAIL [ 2%]
[2025-09-08 19:11:14] tests/convert/test_lit_checkpoint.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-7B-Instruct-1M] XFAIL [ 2%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_no_backward_sync_thunder[ddp] SKIPPED [ 2%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_no_backward_sync_thunder[fsdp] SKIPPED [ 2%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_jit_ddp_before_setup[False] SKIPPED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_jit_ddp_before_setup[True] SKIPPED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_strategy_ddp_setup_already_traced PASSED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_materialize_meta_tensors SKIPPED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_save_load_full_checkpoint SKIPPED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_load_full_checkpoint_only_model SKIPPED [ 3%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_save_load_sharded_checkpoint SKIPPED [ 4%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_jit_fsdp_before_setup[False] SKIPPED [ 4%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_jit_fsdp_before_setup[True] SKIPPED [ 4%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_distributed.py::test_strategy_fsdp_setup_already_traced PASSED [ 4%]
[2025-09-08 19:11:15] tests/ext_thunder/test_thunder_pretrain.py::test_pretrain_thunder PASSED [ 4%]
[2025-09-08 19:12:00] tests/ext_thunder/test_unsloth_executor.py::test_unsloth_cross_entropy[none] PASSED [ 4%]
[2025-09-08 19:12:00] tests/ext_thunder/test_unsloth_executor.py::test_unsloth_cross_entropy[mean] PASSED [ 5%]
[2025-09-08 19:12:01] tests/ext_thunder/test_unsloth_executor.py::test_unsloth_swiglu PASSED [ 5%]
[2025-09-08 19:12:02] tests/ext_thunder/test_unsloth_executor.py::test_unsloth_gpt PASSED [ 5%]
[2025-09-08 19:12:12] tests/generate/test_sequentially.py::test_model_1device_cuda PASSED [ 5%]
[2025-09-08 19:12:12] tests/generate/test_sequentially.py::test_model_forward_hooks PASSED [ 5%]
[2025-09-08 19:12:12] tests/generate/test_sequentially.py::test_base_with_sequentially PASSED [ 5%]
[2025-09-08 19:12:21] tests/generate/test_tp.py::test_tp PASSED [ 6%]
[2025-09-08 19:12:37] tests/test_adapter.py::test_adapter_bitsandbytes PASSED [ 6%]
[2025-09-08 19:12:37] tests/test_adapter.py::test_against_original_gemma_2[device1-dtype1-gemma-2-9b] XFAIL [ 6%]
[2025-09-08 19:12:39] tests/test_adapter.py::test_against_original_gemma_2[device1-dtype1-gemma-2-27b] XFAIL [ 6%]
[2025-09-08 19:12:40] tests/test_adapter.py::test_against_original_gemma_3[device1-dtype1-gemma-3-1b-it] XFAIL [ 6%]
[2025-09-08 19:12:42] tests/test_adapter.py::test_against_original_gemma_3[device1-dtype1-gemma-3-4b-it] XFAIL [ 7%]
[2025-09-08 19:12:44] tests/test_adapter.py::test_against_original_gemma_3[device1-dtype1-gemma-3-12b-it] XFAIL [ 7%]
[2025-09-08 19:12:45] tests/test_adapter.py::test_against_original_gemma_3[device1-dtype1-gemma-3-27b-it] XFAIL [ 7%]
[2025-09-08 19:12:47] tests/test_adapter_v2.py::test_adapter_v2_bitsandbytes PASSED [ 7%]
[2025-09-08 19:12:47] tests/test_api.py::test_more_than_1_device_for_sequential_gpu PASSED [ 7%]
[2025-09-08 19:12:49] tests/test_api.py::test_more_than_1_device_for_tensor_parallel_gpu SKIPPED [ 7%]
[2025-09-08 19:12:49] tests/test_api.py::test_sequential_tp_incompatibility_with_random_weights[sequential] XPASS [ 8%]
[2025-09-08 19:12:50] tests/test_api.py::test_sequential_tp_incompatibility_with_random_weights[tensor_parallel] XPASS [ 8%]
[2025-09-08 19:12:50] tests/test_api.py::test_quantization_is_applied PASSED [ 8%]
[2025-09-08 19:12:50] tests/test_api.py::test_fixed_kv_cache PASSED [ 8%]
[2025-09-08 19:12:50] tests/test_batch.py::test_simple_batch PASSED [ 8%]
[2025-09-08 19:12:50] tests/test_batch.py::test_batch_generate PASSED [ 8%]
[2025-09-08 19:12:54] tests/test_batch.py::test_batch_generate_equivalence PASSED [ 9%]
[2025-09-08 19:12:59] tests/test_ci.py::test_gpu_ci_installs_bitsandbytes PASSED [ 9%]
[2025-09-08 19:12:59] tests/test_distributed.py::test_no_backward_sync[ddp] SKIPPED [ 9%]
[2025-09-08 19:12:59] tests/test_distributed.py::test_no_backward_sync[fsdp] SKIPPED [ 9%]
[2025-09-08 19:12:59] tests/test_generate_speculatively.py::test_main PASSED [ 9%]
[2025-09-08 19:12:59] tests/test_lora.py::test_lora_merge_with_bitsandbytes PASSED [ 9%]
[2025-09-08 19:12:59] tests/test_lora.py::test_lora_bitsandbytes PASSED [ 10%]
[2025-09-08 19:12:59] tests/test_lora.py::test_lora_model_fsdp_init SKIPPED [ 10%]
[2025-09-08 19:12:59] tests/test_lora.py::test_parallelize_fn SKIPPED [ 10%]
[2025-09-08 19:12:59] tests/test_lora.py::test_load_from_full_model_state_dict SKIPPED [ 10%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-16-1-0.25] XPASS [ 10%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-16-1-1] XPASS [ 10%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-16-3-0.25] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-16-3-1] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-32-1-0.25] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-32-1-1] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-32-3-0.25] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-False-32-3-1] XPASS [ 11%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-16-1-0.25] XPASS [ 12%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-16-1-1] XPASS [ 12%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-16-3-0.25] XPASS [ 12%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-16-3-1] XPASS [ 12%]
[2025-09-08 19:12:59] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-32-1-0.25] XPASS [ 12%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-32-1-1] XPASS [ 12%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-32-3-0.25] XPASS [ 13%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_gpt_neox_model[device1-dtype1-True-32-3-1] XPASS [ 13%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_hf_falcon[device1-dtype1-kwargs0] XFAIL [ 13%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_hf_falcon[device1-dtype1-kwargs1] XFAIL [ 13%]
[2025-09-08 19:13:00] tests/test_model.py::test_against_original_open_llama_3b[device1-dtype1] XFAIL [ 13%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs0] XFAIL [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs1] XPASS [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs2] XFAIL [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs3] XFAIL [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs4] XPASS [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs5] XFAIL [ 14%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs6] XFAIL [ 15%]
[2025-09-08 19:13:01] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs7] XFAIL [ 15%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs8] XFAIL [ 15%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs9] XFAIL [ 15%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs10] XFAIL [ 15%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs11] XFAIL [ 15%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_llama_2_and_3[device1-dtype1-ours_kwargs12] XFAIL [ 16%]
[2025-09-08 19:13:02] tests/test_model.py::test_against_hf_phi[device1-dtype1-phi-1_5] XFAIL [ 16%]
[2025-09-08 19:13:03] tests/test_model.py::test_against_hf_phi[device1-dtype1-phi-2] XFAIL [ 16%]
[2025-09-08 19:13:03] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-3-mini-4k-instruct] XFAIL [ 16%]
[2025-09-08 19:13:05] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-3-mini-128k-instruct] XFAIL [ 16%]
[2025-09-08 19:13:06] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-3.5-mini-instruct] XFAIL [ 16%]
[2025-09-08 19:13:07] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-phi-4] XFAIL [ 17%]
[2025-09-08 19:13:10] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-4-mini-instruct] XFAIL [ 17%]
[2025-09-08 19:13:11] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-4-reasoning] XFAIL [ 17%]
[2025-09-08 19:13:13] tests/test_model.py::test_against_hf_phi_3[device1-dtype1-Phi-4-mini-reasoning] XFAIL [ 17%]
[2025-09-08 19:13:15] tests/test_model.py::test_against_mistral_hf_models[Mistral-7B-Instruct-v0.1-device1-dtype1] XFAIL [ 17%]
[2025-09-08 19:13:15] tests/test_model.py::test_against_mistral_hf_models[Mistral-7B-v0.1-device1-dtype1] XFAIL [ 17%]
[2025-09-08 19:13:15] tests/test_model.py::test_against_mathstral_hf_models[device1-dtype1] XFAIL [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_olmo[device1-dtype1-OLMo-1B-hf] XPASS [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_olmo[device1-dtype1-OLMo-7B-hf] XPASS [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_olmo2[device1-dtype1-OLMo-2-1124-7B] XPASS [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_olmo2[device1-dtype1-OLMo-2-1124-13B] XPASS [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_original_stablelm_zephyr_3b[device1-dtype1] XPASS [ 18%]
[2025-09-08 19:13:16] tests/test_model.py::test_against_original_gemma[device1-dtype1-gemma-2b] XFAIL [ 19%]
[2025-09-08 19:13:18] tests/test_model.py::test_against_original_gemma[device1-dtype1-gemma-7b] XFAIL [ 19%]
[2025-09-08 19:13:20] tests/test_model.py::test_against_original_gemma_2[device1-dtype1-gemma-2-9b] XFAIL [ 19%]
[2025-09-08 19:13:21] tests/test_model.py::test_against_original_gemma_2[device1-dtype1-gemma-2-27b] XFAIL [ 19%]
[2025-09-08 19:13:23] tests/test_model.py::test_against_original_gemma_3[device1-dtype1-gemma-3-1b-it] XFAIL [ 19%]
[2025-09-08 19:13:25] tests/test_model.py::test_against_original_gemma_3[device1-dtype1-gemma-3-4b-it] XFAIL [ 20%]
[2025-09-08 19:13:26] tests/test_model.py::test_against_original_gemma_3[device1-dtype1-gemma-3-12b-it] XFAIL [ 20%]
[2025-09-08 19:13:28] tests/test_model.py::test_against_original_gemma_3[device1-dtype1-gemma-3-27b-it] XFAIL [ 20%]
[2025-09-08 19:13:30] tests/test_model.py::test_against_multimodal_gemma_3[device1-dtype1-gemma-3-4b-it] XFAIL [ 20%]
[2025-09-08 19:13:35] tests/test_model.py::test_against_multimodal_gemma_3[device1-dtype1-gemma-3-12b-it] XFAIL [ 20%]
[2025-09-08 19:13:40] tests/test_model.py::test_against_multimodal_gemma_3[device1-dtype1-gemma-3-27b-it] XFAIL [ 20%]
[2025-09-08 19:13:44] tests/test_model.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-1.5B] XFAIL [ 21%]
[2025-09-08 19:13:45] tests/test_model.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-Coder-1.5B] XFAIL [ 21%]
[2025-09-08 19:13:46] tests/test_model.py::test_against_original_qwen_2_5[device1-dtype1-Qwen2.5-Math-1.5B] XFAIL [ 21%]
[2025-09-08 19:13:48] tests/test_model.py::test_against_original_qwen_2_5[device1-dtype1-QwQ-32B-Preview] XFAIL [ 21%]
[2025-09-08 19:13:49] tests/test_model.py::test_against_original_qwen_2_5[device1-dtype1-QwQ-32B] XFAIL [ 21%]
[2025-09-08 19:13:50] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-0.6B] XFAIL [ 21%]
[2025-09-08 19:13:51] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-8B] XFAIL [ 22%]
[2025-09-08 19:13:52] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-4B-Base] XFAIL [ 22%]
[2025-09-08 19:13:53] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-14B-Base] XFAIL [ 22%]
[2025-09-08 19:13:53] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-32B] XFAIL [ 22%]
[2025-09-08 19:13:54] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-4B-Thinking-2507] XFAIL [ 22%]
[2025-09-08 19:13:55] tests/test_model.py::test_against_original_qwen_3[device1-dtype1-Qwen3-4B-Instruct-2507] XFAIL [ 22%]
[2025-09-08 19:13:56] tests/test_model.py::test_against_original_qwen_3_moe[device1-dtype1-Qwen3-30B-A3B] XFAIL [ 23%]
[2025-09-08 19:13:57] tests/test_model.py::test_against_original_qwen_3_moe[device1-dtype1-Qwen3-235B-A22B] XFAIL [ 23%]
[2025-09-08 19:13:58] tests/test_model.py::test_against_original_qwen_3_moe[device1-dtype1-Qwen3-235B-A22B-Thinking-2507] XFAIL [ 23%]
[2025-09-08 19:13:59] tests/test_model.py::test_against_original_qwen_3_moe[device1-dtype1-Qwen3-235B-A22B-Instruct-2507] XFAIL [ 23%]
[2025-09-08 19:14:00] tests/test_model.py::test_against_original_salamandra[device1-dtype1-salamandra-2b] XFAIL [ 23%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_original_salamandra[device1-dtype1-salamandra-7b] XFAIL [ 23%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_original_smollm2[device1-dtype1-SmolLM2-135M] XFAIL [ 24%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_original_smollm2[device1-dtype1-SmolLM2-360M] XFAIL [ 24%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_original_smollm2[device1-dtype1-SmolLM2-1.7B] XPASS [ 24%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_hf_falcon3[device1-dtype1-Falcon3-1B-Base] XFAIL [ 24%]
[2025-09-08 19:14:01] tests/test_model.py::test_against_hf_falcon3[device1-dtype1-Falcon3-7B-Base] XFAIL [ 24%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-base-alpha-3b] PASSED [ 24%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-base-alpha-7b] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-tuned-alpha-3b] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-tuned-alpha-7b] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-3b-4e1t] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablelm-zephyr-3b] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablecode-completion-alpha-3b] PASSED [ 25%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablecode-completion-alpha-3b-4k] PASSED [ 26%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stablecode-instruct-alpha-3b] PASSED [ 26%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[stable-code-3b] PASSED [ 26%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-14m] PASSED [ 26%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-31m] PASSED [ 26%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-70m] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-160m] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-410m] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-1b] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-1.4b] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-2.8b] PASSED [ 27%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-6.9b] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-12b] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-70m-deduped] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-160m-deduped] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-410m-deduped] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-1b-deduped] PASSED [ 28%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-1.4b-deduped] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-2.8b-deduped] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-6.9b-deduped] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[pythia-12b-deduped] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-7b] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-7b-instruct] PASSED [ 29%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-40b] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-40b-instruct] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-180B] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[falcon-180B-chat] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-1B-Base] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-1B-Instruct] PASSED [ 30%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-3B-Base] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-3B-Instruct] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-7B-Base] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-7B-Instruct] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-10B-Base] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Falcon3-10B-Instruct] PASSED [ 31%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[open_llama_3b] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[open_llama_7b] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[open_llama_13b] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-7b-hf] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-7b-chat-hf] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-13b-hf] PASSED [ 32%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-13b-chat-hf] PASSED [ 33%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-70b-hf] PASSED [ 33%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-2-70b-chat-hf] PASSED [ 33%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-3-8B] PASSED [ 33%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-3-8B-Instruct] PASSED [ 33%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-3.1-8B] PASSED [ 34%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-3.1-8B-Instruct] PASSED [ 34%]
[2025-09-08 19:14:02] tests/test_model.py::test_sdpa_choice[Llama-3-70B] PASSED [ 34%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3-70B-Instruct] PASSED [ 34%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.1-70B] PASSED [ 34%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.1-70B-Instruct] PASSED [ 34%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.1-405B] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.1-405B-Instruct] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.2-1B] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.2-1B-Instruct] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.2-3B] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.2-3B-Instruct] PASSED [ 35%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.3-70B-Instruct] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[Llama-3.1-Nemotron-70B-Instruct-HF] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-1B-hf] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-7B-hf] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-7B-Instruct-hf] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-7B] PASSED [ 36%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-7B-SFT] PASSED [ 37%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-7B-DPO] PASSED [ 37%]
[2025-09-08 19:14:03] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-7B-Instruct] PASSED [ 37%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-13B] PASSED [ 37%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-13B-SFT] PASSED [ 37%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-13B-DPO] PASSED [ 37%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[OLMo-2-1124-13B-Instruct] PASSED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2b] PASSED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-7b] PASSED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-2b] SKIPPED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-9b] SKIPPED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-27b] SKIPPED [ 38%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2b-it] PASSED [ 39%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-7b-it] PASSED [ 39%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-2b-it] SKIPPED [ 39%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-9b-it] SKIPPED [ 39%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-2-27b-it] SKIPPED [ 39%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-3-1b-it] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-3-4b-it] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-3-12b-it] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Gemma-3-27b-it] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeGemma-7b-it] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[FreeWilly2] PASSED [ 40%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-7b-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-13b-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-34b-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-70b-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-7b-Python-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-13b-Python-hf] PASSED [ 41%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-34b-Python-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-70b-Python-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-7b-Instruct-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-13b-Instruct-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-34b-Instruct-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[CodeLlama-70b-Instruct-hf] PASSED [ 42%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Platypus-30B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Platypus2-7B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Platypus2-13B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Platypus2-70B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Camel-Platypus2-13B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Camel-Platypus2-70B] PASSED [ 43%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Stable-Platypus2-13B] PASSED [ 44%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[Platypus2-70B-instruct] PASSED [ 44%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[LLaMA-2-7B-32K] PASSED [ 44%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[phi-1_5] PASSED [ 44%]
[2025-09-08 19:14:04] tests/test_model.py::test_sdpa_choice[phi-2] PASSED [ 44%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-3-mini-4k-instruct] PASSED [ 44%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-3-mini-128k-instruct] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-3.5-mini-instruct] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[phi-4] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-4-reasoning] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-4-reasoning-plus] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-4-mini-instruct] PASSED [ 45%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Phi-4-mini-reasoning] PASSED [ 46%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mathstral-7B-v0.1] PASSED [ 46%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-v0.1] PASSED [ 46%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-Instruct-v0.1] PASSED [ 46%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mixtral-8x7B-v0.1] PASSED [ 46%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mixtral-8x7B-Instruct-v0.1] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mixtral-8x22B-v0.1] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mixtral-8x22B-Instruct-v0.1] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-v0.2] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-Instruct-v0.2] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-v0.3] PASSED [ 47%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-7B-Instruct-v0.3] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-Large-Instruct-2407] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Mistral-Large-Instruct-2411] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[tiny-llama-1.1b] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[tiny-llama-1.1b-chat] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[micro-llama-300M] PASSED [ 48%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Llama-2-7b-chat-hf-function-calling-v2] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-0.5B] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-0.5B-Instruct] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-1.5B] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-1.5B-Instruct] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-3B] PASSED [ 49%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-3B-Instruct] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-7B] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-7B-Instruct] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-14B] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-14B-Instruct] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-32B] PASSED [ 50%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-32B-Instruct] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-72B] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-72B-Instruct] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-0.5B] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-0.5B-Instruct] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-1.5B] PASSED [ 51%]
[2025-09-08 19:14:05] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-1.5B-Instruct] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-3B] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-3B-Instruct] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-7B] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-7B-Instruct] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-14B] PASSED [ 52%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-14B-Instruct] PASSED [ 53%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-32B] PASSED [ 53%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Coder-32B-Instruct] PASSED [ 53%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-1.5B] PASSED [ 53%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-1.5B-Instruct] PASSED [ 53%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-7B] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-7B-Instruct] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-72B] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-Math-72B-Instruct] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-7B-Instruct-1M] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen2.5-14B-Instruct-1M] PASSED [ 54%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[QwQ-32B] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[QwQ-32B-Preview] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-0.6B] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-0.6B-Base] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-1.7B] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-1.7B-Base] PASSED [ 55%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-4B] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-4B-Base] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-8B] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-8B-Base] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-14B] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-14B-Base] PASSED [ 56%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-32B] PASSED [ 57%]
[2025-09-08 19:14:06] tests/test_model.py::test_sdpa_choice[Qwen3-30B-A3B] PASSED [ 57%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-30B-A3B-Base] PASSED [ 57%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-235B-A22B] PASSED [ 57%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-235B-A22B-Thinking-2507] PASSED [ 57%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-235B-A22B-Instruct-2507] PASSED [ 57%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-30B-A3B-Thinking-2507] PASSED [ 58%]
[2025-09-08 19:14:07] tests/test_model.py::test_sdpa_choice[Qwen3-30B-A3B-Instruct-2507] PASSED [ 58%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[Qwen3-4B-Thinking-2507] PASSED [ 58%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[Qwen3-4B-Instruct-2507] PASSED [ 58%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[salamandra-2b] PASSED [ 58%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[salamandra-2b-instruct] PASSED [ 58%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[salamandra-7b] PASSED [ 59%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[salamandra-7b-instruct] PASSED [ 59%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-135M] PASSED [ 59%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-135M-Instruct] PASSED [ 59%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-360M] PASSED [ 59%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-360M-Instruct] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-1.7B] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[SmolLM2-1.7B-Instruct] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[R1-Distill-Llama-8B] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice[R1-Distill-Llama-70B] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-base-alpha-3b] PASSED [ 60%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-base-alpha-7b] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-tuned-alpha-3b] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-tuned-alpha-7b] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-3b-4e1t] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablelm-zephyr-3b] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablecode-completion-alpha-3b] PASSED [ 61%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablecode-completion-alpha-3b-4k] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stablecode-instruct-alpha-3b] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[stable-code-3b] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-14m] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-31m] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-70m] PASSED [ 62%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-160m] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-410m] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-1b] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-1.4b] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-2.8b] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-6.9b] PASSED [ 63%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-12b] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-70m-deduped] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-160m-deduped] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-410m-deduped] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-1b-deduped] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-1.4b-deduped] PASSED [ 64%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-2.8b-deduped] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-6.9b-deduped] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[pythia-12b-deduped] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-7b] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-7b-instruct] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-40b] PASSED [ 65%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-40b-instruct] PASSED [ 66%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-180B] PASSED [ 66%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[falcon-180B-chat] PASSED [ 66%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-1B-Base] PASSED [ 66%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-1B-Instruct] PASSED [ 66%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-3B-Base] PASSED [ 67%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-3B-Instruct] PASSED [ 67%]
[2025-09-08 19:14:08] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-7B-Base] PASSED [ 67%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-7B-Instruct] PASSED [ 67%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-10B-Base] PASSED [ 67%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Falcon3-10B-Instruct] PASSED [ 67%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[open_llama_3b] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[open_llama_7b] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[open_llama_13b] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-7b-hf] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-7b-chat-hf] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-13b-hf] PASSED [ 68%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-13b-chat-hf] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-70b-hf] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-2-70b-chat-hf] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3-8B] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3-8B-Instruct] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-8B] PASSED [ 69%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-8B-Instruct] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3-70B] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3-70B-Instruct] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-70B] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-70B-Instruct] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-405B] PASSED [ 70%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-405B-Instruct] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.2-1B] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.2-1B-Instruct] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.2-3B] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.2-3B-Instruct] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.3-70B-Instruct] PASSED [ 71%]
[2025-09-08 19:14:09] tests/test_model.py::test_sdpa_choice_kv_cache[Llama-3.1-Nemotron-70B-Instruct-HF] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-1B-hf] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-7B-hf] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-7B-Instruct-hf] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-7B] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-7B-SFT] PASSED [ 72%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-7B-DPO] PASSED [ 73%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-7B-Instruct] PASSED [ 73%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-13B] PASSED [ 73%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-13B-SFT] PASSED [ 73%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-13B-DPO] PASSED [ 73%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[OLMo-2-1124-13B-Instruct] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2b] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-7b] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-2b] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-9b] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-27b] PASSED [ 74%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-7b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-2b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-9b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-2-27b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-3-1b-it] PASSED [ 75%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-3-4b-it] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-3-12b-it] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[Gemma-3-27b-it] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[CodeGemma-7b-it] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[FreeWilly2] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[CodeLlama-7b-hf] PASSED [ 76%]
[2025-09-08 19:14:10] tests/test_model.py::test_sdpa_choice_kv_cache[CodeLlama-13b-hf] PASSED [ 77%]
…(truncated)
Loading