Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Add Large Tensor Test for linalg_syrk #18782

Merged
merged 5 commits into from
Jul 24, 2020
Merged

Add Large Tensor Test for linalg_syrk #18782

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6c1a904
add large tensor test for syrk, foward and backward
Jul 23, 2020
0cefe25
Merge branch 'v1.x' into add_syrk_large_tensor_test
Zha0q1 Jul 23, 2020
ae4318a
change to batch input
Jul 24, 2020
8d9b188
Merge branch 'v1.x' into add_syrk_large_tensor_test
Zha0q1 Jul 24, 2020
b49f999
move syrk test into test-linalg
Jul 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion tests/nightly/test_large_array.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
LARGE_X = 100000000
SMALL_X = 100
SMALL_Y = 50
LARGE_SQ_X = 80000
LARGE_SQ_X = 70000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Contributor

@ChaiBapchya ChaiBapchya Jul 24, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

70000*70000/2**32 is just over 2**32
80k is lot more

Copy link
Contributor Author

@Zha0q1 Zha0q1 Jul 24, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We figured 7000 is large enough to overflow int 32. This is a new constant just introduced so the few of us decided to tweak it to 70000

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

70k * 70k = 4.9 billion
int32 range < 4.3 billion
Therefore increasing input size will only increase test runtime.

LARGE_SIZE = LARGE_X * SMALL_Y
LARGE_TENSOR_SHAPE = 2**32
RNN_LARGE_TENSOR = 2**28
Expand Down Expand Up @@ -1190,9 +1190,26 @@ def check_potri():
# output should be an identity matrix
for i in range(LARGE_SQ_X):
assert out[i,i] == 1

def check_syrk_batch():
# test both forward and backward
# batch syrk will be applied to the last two dimensions
A = nd.zeros((2, LARGE_SQ_X, LARGE_SQ_X))
for i in range(LARGE_SQ_X):
A[0,i,i] = 1
A[1,i,i] = 0.1
A.attach_grad()
with mx.autograd.record():
out = nd.linalg.syrk(A, alpha=2, transpose=False)
assert out[0,0,0] == 2
assert_almost_equal(out[1,0,0], nd.array([0.02]), rtol=1e-3, atol=1e-5)
out.backward()
assert A.grad[0,0,0] == 4
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)

check_potrf()
check_potri()
check_syrk_batch()


def test_basic():
Expand Down