[tvegas1]

What?

Add an aarch64 st64b BlueFlame copy mode for mlx5, with runtime selection and generic-copy fallback.

Why?

st64b can issue a single 64-byte store for BlueFlame doorbell writes, which may improve MMIO write-combining behavior on LS64-capable ARM CPUs.

How?

Detect LS64 support with getauxval(AT_HWCAP3) & HWCAP3_LS64, add an mlx5 BF copy function using st64b, select it through the BF copy-mode config, and add a gtest variant for the ST64B MMIO path.

Testing

UCX_RC_MLX5_MMIO_MODE=bf_post UCX_DC_MLX5_MMIO_MODE=bf_post \
UCX_UD_MLX5_MMIO_MODE=bf_post UCX_GGA_MLX5_MMIO_MODE=bf_post \
UCX_RC_MLX5_BF_COPY_MODE=st64b UCX_DC_MLX5_BF_COPY_MODE=st64b \
UCX_UD_MLX5_BF_COPY_MODE=st64b UCX_GGA_MLX5_BF_COPY_MODE=st64b \
./test/gtest/gtest --gtest_filter='rc_mlx5/*:dc_mlx5/*:ud_mlx5/*:gga_mlx5/*'

./test/gtest/gtest --gtest_filter='*test_rc_mlx5_bf_copy*'

[tvegas1]

Initially tested on supporting platform with added gtest.

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

Residual coverage gap: the st64b path is only exercised when the build machine's assembler supports .arch_extension ls64 and the runtime CPU reports HWCAP3_LS64; otherwise both the new gtest and the new configure define are inert. Per REVIEW.md (configure/build options should have CI coverage for the changed path), it is worth confirming whether any aarch64 CI job actually compiles HAVE_AARCH64_ST64B_ASM=1 and runs test_rc_mlx5_bf_copy_st64b, otherwise this feature ships untested in CI.

Residual note: UCT_IB_MLX5_BF_COPY_MODE is exposed for every mlx5 TL through uct_ib_mlx5_iface_config_table, but gdaki hard-codes GENERIC, so st64b is silently ignored there while it errors out on other TLs. Harmless today (gdaki uses MMIO_MODE_DB), so not raised inline.

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

Residual coverage gap: the st64b path only runs when the build machine's assembler accepts .arch_extension ls64 and the runtime CPU reports HWCAP3_LS64. Worth confirming whether any AArch64 CI job actually compiles with HAVE_AARCH64_ST64B_ASM=1 and executes test_rc_mlx5_bf_copy_st64b (buildlib/tools/builds.sh / contrib/test_jenkins.sh), otherwise the new code path ships without CI coverage — only the generic variant would be exercised.

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

Can we also assert the selected copy mode here, next to the mmio_mode check? test_bf_copy() asserts reg->mode but never wq->bf_copy_mode. On LS64 hardware auto also resolves to ST64B, so if BF_COPY_MODE ever stops reaching the txwq (six call sites now thread it) the generic case would silently run ST64B and still pass. Something like an expected-mode argument to test_bf_copy() plus EXPECT_EQ(expected, ep->tx.wq.bf_copy_mode) under #if UCT_IB_MLX5_HAVE_ST64B.

[svc-nvidia-pr-review]

minor: assignment alignment — docs/CodeStyle.md: "the longest left-hand side gets one space before =, others align to it." The longest LHS here has four.

Suggested change

	self->tx.mmio_mode = config->mlx5_common.mmio_mode;
	self->tx.mmio_mode = config->mlx5_common.mmio_mode;
	self->tx.bf_copy_mode = config->mlx5_common.bf_copy_mode;
	self->tx.wq.super.type = UCT_IB_MLX5_OBJ_TYPE_LAST;

[svc-nvidia-pr-review]

Residual coverage gap (already reported on this PR, not repeated inline): the st64b path requires both an assembler that accepts .arch_extension ls64 and a runtime CPU reporting HWCAP3_LS64, and no AArch64 CI job appears to satisfy both.

[svc-ucx]

🤖 CI Triage Agent — UCX PR (Tests roce on worker 0) · commit ea9b8e1e

TL;DR: The roce gtest shard did not time out because it was slow — dc_mlx5/uct_amo_and_or_test.and32/1 <dc_mlx5/mlx5_1:1> hung with zero output for the full 900 s gtest watchdog window, blocked in the AMO sender thread waiting for send completions that never arrived (lost doorbell / missing TX CQE on the DCI), and the only delta in this build is the mlx5 BlueFlame doorbell / MMIO-mode rework of PR #11542 merged with master's #11989.

Full analysis

Summary: make -C test/gtest test aborted (make: *** [Makefile:4809: test] Aborted) after the gtest watchdog fired ("Connection timed out - abort testing", test_helpers.cc:57) on dc_mlx5/uct_amo_and_or_test.and32/1.

Root cause: Log timestamps show the test started at 16:25:08.32 and produced no line at all until the watchdog abort at 16:40:08.32 — one 900 s gap that is essentially the whole runtime, i.e. a hang, not slowness. The backtrace shows the main thread parked in pthread_join (test_amo.cc:199 ← uct_amo_test::run_workers test_amo.cc:135), so an AMO sender thread never finished its post loop: it was spinning in while (status == UCS_ERR_NO_RESOURCE) { m_entity.progress(); ... } (test/gtest/uct/test_amo.cc:176-180). TX BBs are only released by send CQEs, and no CQE (not even an error CQE) ever arrived — the classic signature of a WQE whose doorbell never reached the HCA. This is the first multi-BB (masked-atomic) WQE posted in the shard; the earlier single-BB dc_mlx5/uct_p2p_rma_test cases on mlx5_0 passed. The code that was just reworked is exactly the doorbell path: uct_ib_mlx5_bf_copy()/UCT_IB_MLX5_BF_COPY multi-BB copy macro and uct_ib_mlx5_txwq_ring_doorbell() (src/uct/ib/mlx5/ib_mlx5.inl:603-688), the new bf_copy_mode/mmio_mode plumbing through uct_ib_mlx5_qp_attr_t → uct_rc_mlx5_iface_fill_attr() (rc_mlx5_common.c:526-547) → uct_dc_mlx5_iface_create_dci() (dc_mlx5.c:450-554), and uct_ib_mlx5_get_mmio_mode() (ib_mlx5.c:774-800). Note the agent (swx-rain04) is x86_64 (test_arch.nt_buffer_transfer_nt_src_erms → "Built without AVX support"), so UCT_IB_MLX5_HAVE_ST64B == 0 and the ST64B assembly itself is compiled out — meaning the suspect is the MMIO-mode / bf_copy-mode configuration plumbing, not the ST64B store. A related latent bug is visible in that plumbing: uct_ib_mlx5_txwq_init_bf_copy() never assigns txwq->bf_copy_mode on the non-ST64B branch (ib_mlx5.c:824-836), leaving it at whatever the txwq memory held. A wrong/garbage mode or a BF_POST mode selected for a non-WC (NC) UAR makes the 64-byte BlueFlame write invisible to the device, which reproduces exactly this silent stall (cf. the earlier fix [REDACTED:Hex High Entropy String] "Fix DEVX for not WC-supported UAR (#11310)").

Implicated commit: ea9b8e1e (Thomas Vegas, merge of upstream/master into pr11542) bringing together the PR's doorbell rework (db95d46a, bf3e7cf1, b6d85113, 1855df30/938781d6) and master's 8cfe4b80 "UCT/IB/MLX5: Honor configured MMIO mode with a verbs SRQ (#11989)".

File: src/uct/ib/mlx5/ib_mlx5.c:802-837 (uct_ib_mlx5_txwq_init_bf_copy, bf_copy_mode left uninitialized) and src/uct/ib/mlx5/ib_mlx5.inl:603-688 (UCT_IB_MLX5_BF_COPY / uct_ib_mlx5_txwq_ring_doorbell); hang observed from test/gtest/uct/test_amo.cc:176-180.

Suggested fix:

1. In uct_ib_mlx5_txwq_init_bf_copy(), always initialize the mode first, unconditionally: txwq->bf_copy_mode = UCT_IB_MLX5_BF_COPY_MODE_GENERIC; before any #if UCT_IB_MLX5_HAVE_ST64B logic, so the field can never be garbage on any build/arch.
2. Verify the new mmio_mode/bf_copy_mode plumbing for DEVX DCIs: assert in uct_ib_mlx5_get_mmio_mode() / DCI creation that BF_POST* is never selected when the UAR was allocated as UCT_IB_MLX5_UAR_ALLOC_TYPE_NC (!(md->flags & UCT_IB_MLX5_MD_FLAG_UAR_USE_WC)), and log the chosen mode per DCI (the existing ucs_debug("tx wq ... mmio_mode %s") is only on the verbs path).
3. To confirm/bisect, re-run just this test on the same node: gtest --gtest_filter=dc_mlx5/uct_amo_and_or_test.* with (a) UCX_IB_MLX5_MMIO_MODE=db, (b) UCX_IB_MLX5_BF_COPY_MODE=generic, and (c) the same shard on plain master. If (a) passes and default hangs, the doorbell-mode selection is the culprit. Do not raise the 200 m/watchdog limits — the process was hung, not slow.

Related: PR #11542 (#11542, "UCT/IB/MLX5: Add AArch64 ST64B BF copy m

🛡️ This comment had 1 potential secret(s) redacted (Hex High Entropy String). See request_id 00b4b352-6183-4965-b2ba-1ee0909e1f87 in the triage console for the audit trail.

[brminich]

do you think we can define new mmio mode instead?

[tvegas1]

Merging them into one user-facing value doesn't really work, because the two autos answer different questions: MMIO auto depends on thread mode, need_lock and WC UAR support, while BF copy auto depends only on the LS64 HWCAP. It would also lose the mixed cases, e.g. MMIO_MODE=auto BF_COPY_MODE=generic to keep automatic doorbell selection but disable ST64B.

I did prototype merging them internally (following recent bot comment): resolve both at setup time into extra uct_ib_mlx5_mmio_mode_t values past _LAST (BF_POST_ST64B, BF_POST_MT_ST64B) that the config table cannot reach, so the configuration is unchanged. That drops txwq->bf_copy_mode, leaves a single dispatch in ring_doorbell(), and removes the per-WQE branch you flagged in ib_mlx5.inl.

But the code change is significant and perf is unchanged, so I'd prefer to keep the current split.