Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions tests/compass/test_memory_compare.py
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,13 @@ def test_the_tied_head_is_one_embedding_of_0_290_gib_on_the_0_6b():
assert round(nbytes / (1 << 30), 3) == 0.290


def test_the_tied_head_scales_with_a_4_byte_element_size():
# A float32 build of the same geometry owes twice the bfloat16 correction.
nbytes = tied_lm_head_bytes(qwen_0_6b(), dtype_bytes=4)
assert nbytes == 151_936 * 1_024 * 4
assert nbytes == 622_329_856
Comment on lines +320 to +321

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shrink: (principle 3, non-blocking) L320-321 assert the same value twice, once as the product and once as its literal. One line carries both: assert nbytes == 151_936 * 1_024 * 4 == 622_329_856. The T2 red keeps its message under the chained form: pytest still prints assert 311164928 == ((151936 * 1024) * 4). net: -1 line.



def test_an_untied_model_owes_no_correction(qwen):
assert qwen.tie_word_embeddings is False
assert tied_lm_head_bytes(qwen, dtype_bytes=2) == 0
Expand Down