[tvegas1]

What?

Fix flacky test, apparently under load the write thread could end-up being running after the normally subsequent reader thread.

Also added optional visibility-based test exercising the memory acquire/release aspect of the read and write lock.

2026-08-20T09:37:28.4719473Z [       OK ] test_rwlock.memory_barriers (30905 ms)
2026-08-20T09:37:28.4719957Z [ RUN      ] test_rwlock.lock <> <>
2026-08-20T09:37:28.4756001Z /scrap/azure/agent-05/AZP_WORKSPACE/1/s/contrib/../test/gtest/ucs/test_type.cc:230: Failure
2026-08-20T09:37:28.4756701Z Value of: read_taken
2026-08-20T09:37:28.4757094Z   Actual: true
2026-08-20T09:37:28.4757452Z Expected: false
2026-08-20T09:37:28.4768151Z [  FAILED  ] test_rwlock.lock, where TypeParam =  and GetParam() =  (5 ms)
2026-08-20T09:37:28.4768938Z [----------] 2 tests from test_rwlock (30910 ms total)

[brminich]

pls fix a type mentioned by @iyastreb

[svc-ucx]

🤖 CI Triage Agent — UCX PR (AddressSanitizer BlueField on worker 1) · commit b33231ce

TL;DR: The AddressSanitizer/BlueField run failed only in dcx/test_ucp_am_nbx_seg_size.multi/1, where the multi-fragment eager AM send request completed with UCS_ERR_NO_MEMORY so the receiver never got the message and the test burned the full 180 s deadline in wait_receives(); this is unrelated to PR #11809 (which only bumps a sleep in test/gtest/ucs/test_rwlock.cc) — re-run the job, and fix the test to fail fast on send errors instead of spinning 180 s.

Full analysis

Summary: gtest failed with 1 FAILED TEST — dcx/test_ucp_am_nbx_seg_size.multi/1 (dc_x/proto_v1), causing make: *** [Makefile:4713: test] Error 1 and Bash exited with code '2'.

Root cause: At 13:12:34 the test printed seg size 1790 data size 3580 (random IB_SEG_SIZE=1790 → a 2-fragment eager AM over dc_x, proto v1, with an extra sender entity created by test_ucp_am_nbx_seg_size::init()). The next log line is 3 m 20 s later: ucp_test.cc:304 UCX ERROR request 0xffff4d1ca418 completed with error Out of memory, followed by test_ucp_am.cc:542: Expected equality ... m_recv_counter Which is: 0 / m_send_counter Which is: 1. So the send request completed with UCS_ERR_NO_MEMORY (allocation/registration failure on the AM send path — plausible under ASan's redzone overhead on a memory-constrained BlueField after 2647 s of tests), the receiver counter never advanced, and wait_receives() → wait_for_value() spun until the gtest deadline (ucs::test_timeout_in_sec = 180, test_helpers.cc:28), which explains the 200596 ms test duration and the silent 200 s gap. The gap is a deadline expiry after a failed request, not a genuine hang of the build, and not a wall-clock capacity problem.

Nothing in the PR touches this path: PR #11809 ("GTEST/UCS: Increase rwlock test sleep to avoid CI races") only changes test/gtest/ucs/test_rwlock.cc — visible in the log as test_rwlock.memory_barriers - 21871 ms in the TOP-20 list, which passed. All 8607 other tests passed.

Implicated commit: unknown — not caused by b33231c (PR #11809); the AM path was last touched by 2bafbb0 (Evgeny Leksikov, "UCP/AM/FT: introduce resend flag for first fragment retransmissions", #11452) and b6b8531 (tomerg-nvidia, "UCP/UCT/UCS: Add memory registration flags", #11553)

File: test/gtest/ucp/test_ucp_am.cc:542 (assertion), test/gtest/ucp/test_ucp_am.cc:1695-1713 (test_am_different_seg_sizes), test/gtest/ucp/ucp_test.cc:304 (error report)

Suggested fix:

1. Re-run the BlueField ASan job — this is an environment/flakiness failure unrelated to the rwlock test change; do not block PR GTEST/UCS: Increase rwlock test sleep to avoid CI races #11809 on it.
2. Make the test surface the real error immediately instead of hiding it behind a 180 s deadline: in test_am_send_recv() (test_ucp_am.cc:530-535), check UCS_PTR_IS_ERR(sptr)/ucp_request_check_status(sptr) and ASSERT/return before calling wait_receives(), or reorder to request_wait(sptr) first. That converts a 200 s timeout into an instant, diagnosable failure.
3. To root-cause the UCS_ERR_NO_MEMORY itself, re-run this single variant with UCX_LOG_LEVEL=diag (and UCX_PROTO_ENABLE=n, UCX_IB_SEG_SIZE=1790, UCX_TLS=dc_x) so the failing mpool/registration allocation is logged; also capture free memory on the BF worker, since ASan roughly doubles the footprint of the 3-entity seg-size test.

Related: #11809 (the PR under test; test-only rwlock sleep change, not the cause)