Skip to content

UCS/ARCH/LOONGARCH64: Add loongarch64 support - #12187

Open
loongson-zn wants to merge 1 commit into
openucx:masterfrom
loongson-zn:la-dev
Open

loongson-zn wants to merge 1 commit into
openucx:masterfrom
loongson-zn:la-dev

Conversation

@loongson-zn

@loongson-zn loongson-zn commented Oct 28, 2026 •

Copy link
Copy Markdown

What?

Add LoongArch64 architecture support to UCS (cpu/bitops/atomic/global_opts,
memory barriers, high-resolution clock) and UCM (BISTRO binary patcher)

Why?

Enables UCX to build and run on LoongArch64 (Loongson) platforms.

How?

Barrier hints match the Linux kernel arch/loongarch/include/asm/barrier.h;
the timer frequency comes from CPUCFG.4/5 as in the kernel's calc_const_freq().

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

@svc-ucx

svc-ucx commented Oct 28, 2026

Copy link
Copy Markdown

🤖 CI Triage Agent — UCX PR (Codestyle AUTHORS file update check) · commit 7f9040aa

TL;DR: The "AUTHORS file update check" failed because PR #12187 introduces a commit from a new contributor (Na Zhang zhangna@loongson.cn) who is not listed in the AUTHORS file; the fix is to run ./contrib/authors_update.sh and commit the updated AUTHORS file in the PR.

Full analysis

Summary: Azure Pipelines stage "AUTHORS file check" exited with code 1 because git diff --exit-code detected an uncommitted change to AUTHORS after running the authors-update script.

Root cause: Not a code or infrastructure defect — this is the codestyle policy gate working as designed. The step computes the PR commit range (20fe448e..7f9040aa), runs contrib/authors_update.sh, and then asserts the tree is clean. The script reported ++ Na Zhang <zhangna@loongson.cn> (the ++ prefix at contrib/authors_update.sh:38 means "new author appended"), and the subsequent git diff --exit-code printed the pending one-line insertion into AUTHORS after Min Fang <minf@nvidia.com> and returned non-zero, failing the Bash task. The current repo AUTHORS (lines 89–90) indeed goes straight from Min Fang to Nathan Bellalou, with no entry for Na Zhang, and neither the email zhangna@loongson.cn nor the name Na Zhang matches any existing line, so the script's dedup check at contrib/authors_update.sh:36 correctly treated it as a new contributor. There was no build error, timeout, or hang — total stage runtime was well under a second (03:48:06.3 → 03:48:06.9).

Implicated commit: [REDACTED:Hex High Entropy String] (PR head, author Na Zhang zhangna@loongson.cn) — merged as [REDACTED:Hex High Entropy String] onto base [REDACTED:Hex High Entropy String]. The check script itself is unchanged and not at fault.

File: AUTHORS:89 (insertion point; diff hunk header @@ -87,6 +87,7 @@), check logic at contrib/authors_update.sh:36-46

Suggested fix: In the PR branch la-dev, add the contributor to AUTHORS and amend/append a commit:

git fetch origin
./contrib/authors_update.sh origin/master..HEAD
git diff            # should show only the one AUTHORS line
git add AUTHORS && git commit -m "AUTHORS: add Na Zhang"
git push

Or add the line manually, preserving LC_COLLATE=C sort order, between Min Fang <minf@nvidia.com> and Nathan Bellalou <nbellalou@nvidia.com>:

Na Zhang <zhangna@loongson.cn>

Make sure the email used in the AUTHORS entry exactly matches the git log author email on the commits (zhangna@loongson.cn), otherwise the check will re-trigger. No retry or pipeline change is needed — the gate will pass once AUTHORS is committed.

Related: PR #12187 (the failing PR). No existing issue found for this check; it is the standard new-contributor AUTHORS gate, consistent with prior commits such as 42c438a and 90c8961 which each updated AUTHORS alongside the contributor's first change.

🛡️ This comment had 1 potential secret(s) redacted (Hex High Entropy String). See request_id 504d787a-cd6b-4cd7-b7eb-fcfd51b55787 in the triage console for the audit trail.

Comment thread src/ucm/bistro/bistro.c Resolved
Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
Comment thread src/ucs/arch/loongarch64/bitops.h Resolved
return UCS_ERR_UNSUPPORTED;
}

static inline uint64_t ucs_arch_read_hres_clock()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoongArch has a constant-rate counter (rdtime.d, with frequency derivable from cpucfg 0x4/0x5). why not use it instead of falling back to gettimeofday()? ucs_get_time() is on the progress/timer hot path, and configure.m4 already defines HAVE_HW_TIMER=1 for non-aarch64 hosts, so test_time.get_time will run and assert ucs_get_time() under 40ns.

Comment thread src/ucs/arch/cpu.c Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

adding a new architecture with no CI job means nothing here is build- or run-tested upstream. can you share which tests were run on LoongArch hardware (at least gtest ucm/malloc_hook for the new bistro patch, and ucx_info -s)?

Residual coverage gaps: No build or runtime CI for __loongarch_lp64; nothing in buildlib/ or contrib/test_jenkins.sh exercises this path.

Residual coverage gap: ld.d at func_ptr + 0x10 is an 8-byte load whose alignment depends on the patched function's alignment (functions are only guaranteed 4-byte aligned). Same design as rv64 and relies on hardware/kernel unaligned-access support, so I did not raise it, but it is untested here.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

* @param[in] _imm 16-bit immediate value
*/
#define UCM_BISTRO_LA_JIRL(_regd, _regj, _imm) \
(0x4c000000 | ((_imm) << 10) | ((_regj) << 5) | (_regd))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: offs16/offs26 are word-scaled (offs = imm << 2), but the doc comments describe them as plain immediates and the macros neither shift nor mask _imm. Harmless today (only 0 is passed) but a trap for the next user — please document it, otherwise a future non-zero value will silently corrupt the opcode.

Comment thread src/ucm/Makefile.am Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

src/ucs/arch/loongarch64/cpu.h

blocker: cfd (and cc_freq/cfm) can be 0 — it is taken straight from CPUCFG.5 and used as a divisor with no check. The kernel's calc_const_freq() checks CPUCFG2.LLFTP and !base_freq || !cfm || !cfd before dividing. Here a zero CC_DIV is an integer division by zero (SIGFPE) inside ucs_init_cpu_clocks_per_sec(), i.e. at library init; a zero result would also trip the ucs_assert(clocks_per_sec > 0.0) in ucs_get_cpu_clocks_per_sec().

Suggested fix:

    if ((cc_freq == 0) || (cfm == 0) || (cfd == 0)) {
        return ucs_arch_generic_get_clocks_per_sec();
    }

    return (double)(cc_freq * cfm / cfd);

src/ucs/arch/loongarch64/cpu.h

comment: ucs_memory_bus_cacheline_wc_flush() and ucs_memory_cpu_wc_fence() are the device/WC-facing barriers (used around the mlx5 BlueFlame doorbell in uct/ib/mlx5/ib_mlx5.inl:659,665), but they map to ucs_memory_cpu_fence() = dbar 0x10, which the file's own comment classifies as ordering-only / inter-CPU. The two sibling ucs_memory_bus_* macros correctly use the completion hints (0x0a, 0x05). Can we use the completion barrier (dbar 0, kernel iob()) here and for ucs_memory_cpu_wc_fence()?

Residual coverage gaps: there is still no __loongarch_lp64 CI job and the ld.d alignment assumption is untested. Additionally, ucs_arch_get_clocks_per_sec() and rdtime.d are only correct when the stable counter is present, and nothing in the diff validates that — ucx_info -s output from real hardware would cover both the frequency math and ucs_cpu_vendor_name()/ucs_cpu_model_name() indexing for the two new enum values.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Review complete — no issues found.

@loongson-zn

Copy link
Copy Markdown
Author

LoongArch64 port — test results

Tested on real LoongArch64 hardware.

Environment

Item Value
CPU Loongson-3A5000M (4 cores, 2000 MHz)
ISA loongarch64 (loongarch32r / loongarch32s / loongarch64)
Kernel Linux 7.1.8+deb14.1-loong64
Toolchain gcc/g++ 16.2.0 (Debian), binutils 2.47
OS Debian GNU/Linux forky/sid

Build

Configured with --enable-gtest (default CFLAGS include -Werror).

$ ./autogen.sh && ./configure --prefix=$PWD/install --enable-gtest && make -j4

libucs, libuct, libucp, libucm, ucx_info, and the gtest binary all
built cleanly (no warnings) on LoongArch64.

ucx_info sanity check

$ ./src/tools/info/ucx_info -v
# Library version: 1.24.0
# Git branch 'la-dev', revision <PR head>

$ ./src/tools/info/ucx_info -s
# Timer frequency: 100.000 MHz
# Timer accuracy: 99.999 %
# CPU vendor: Generic LoongArch64
# CPU model: LoongArch 64-bit
# Product name: L71S

This validates the rdtime.d + cpucfg (CC_FREQ/CC_MUL/CC_DIV) frequency math
and the two new enum values' name-table indexing.

gtest — timer (test_time.get_time)

The ucs_get_time() latency assertion (< 40 ns) passes, confirming the
Stable Counter path replaces the old gettimeofday() fallback.

$ ./test/gtest/gtest --gtest_filter='test_time.*'
[ RUN      ] test_time.time_calc
[       OK ] test_time.time_calc (1 ms)
[ RUN      ] test_time.get_time
[       OK ] test_time.get_time (1000 ms)
[ RUN      ] test_time.timerq
[       OK ] test_time.timerq (328 ms)
[  PASSED  ] 3 tests.

Standalone measurement of the rdtime.d read: ~9 ns (well below the 40 ns
budget).

gtest — UCM bistro (malloc_hook.bistro_patch)

The new LoongArch64 BISTRO trampoline (pcaddu12i / ld.d / jirl) is exercised by
malloc_hook.bistro_patch and passes.

$ ./test/gtest/gtest --gtest_filter='malloc_hook.bistro_patch:malloc_hook.asprintf:malloc_hook.test_event*'
[ RUN      ] malloc_hook.asprintf
[       OK ] malloc_hook.asprintf (1 ms)
[ RUN      ] malloc_hook.bistro_patch
[       OK ] malloc_hook.bistro_patch (51 ms)
[ RUN      ] malloc_hook.test_event
[       OK ] malloc_hook.test_event (0 ms)
[ RUN      ] malloc_hook.test_event_failed
[       OK ] malloc_hook.test_event_failed (102 ms)
[ RUN      ] malloc_hook.test_event_unmap
[       OK ] malloc_hook.test_event_unmap (51 ms)
[  PASSED  ] 5 tests.

Note: malloc_hook.single_thread / multi_threads are SKIPped because
valgrind is not installed on this host (skip_on_bistro_without_valgrind()).

@svc-ucx

svc-ucx commented Oct 28, 2026

Copy link
Copy Markdown

🤖 CI Triage Agent — UCX PR (Tests roce on worker 0) · commit bf592e40

TL;DR: The roce on worker 0 job failed because ./test/apps/test_hooks -n 38 dumped core on iteration 22/30 of the "UCM Bistro hook test" — a pre-existing race in the x86_64 bistro code-patching path (threads executing a half-patched mmap prologue), not anything introduced by this loongarch64 PR; re-run the job and, if it reproduces, raise UCX_BISTRO_GRACE_DURATION / harden the patch-lock step.

Full analysis

Summary: contrib/test_jenkins.sh: line 1101: 3059437 Segmentation fault timeout 10 ./test/apps/test_hooks -n 38 → timeout: the monitored command dumped core → script exit 1 ("Bash exited with code '1'").

Root cause: test_hooks deliberately races N threads calling mmap() against UCM bistro installing its hooks (it dlopens libucs.so while threads spin in mmap, see test/apps/test_hooks.c:83 and :189). On x86_64 the patch is applied non-atomically: ucm_bistro_apply_patch_atomic() writes a 2-byte self-jump "lock" (0xEB 0xFE), waits ucm_global_opts.bistro_grace_duration, then memcpy()s the remaining 3–10 patch bytes over the live function. The code itself documents that this is only probabilistically safe ("assuming that we will only truncate the current instruction which is assumed to be already fetched in case of race", src/ucm/bistro/bistro_x86_64.c:430-440). The thread count is randomized as (RANDOM % (2*NPROC)) + 1 (contrib/test_jenkins.sh:1102); this iteration drew 38 — the maximum, i.e. 2× oversubscription — which is the worst case for the race window, and a thread executed a truncated instruction stream → SIGSEGV. There is prior art for exactly this flakiness: c77bb48 "UCM/BISTRO: Increase grace time to reduce patch race (#11112)" and 676b579 "Add locking step while patching library functions".

Evidence it is not caused by the PR commit: [REDACTED:Hex High Entropy String] touches src/ucm/bistro/bistro.h, bistro.c, src/ucs/arch/cpu.h and cpu.c, but every change is arch-guarded (#elif defined(__loongarch_lp64) include branch at bistro.h:26, the || defined(__loongarch_lp64) guard at bistro.c:123) or inert new enum/array entries (UCS_CPU_MODEL_LOONGARCH64, UCS_CPU_VENDOR_GENERIC_LOONGARCH64, with matching designated initializers in cpu.c:84 / :178 / :211). No x86_64 patching code path is altered. The gtest shard and all other steps passed.

Secondary anomaly worth noting (not the failure): earlier in the same log, ./test/apps/test_ucp_config: error: '.../build-test/test/apps/.libs/test_ucp_config' does not exist — the libtool wrapper exists but the real binary does not, so the UCX_MODULES=^ib,rdmacm module-exclusion check silently became a no-op (the following grep found nothing and passed). That points at an incomplete/shared build tree on this worker (cf. 390a30e "AZP/TESTS: Build once per GPU test row and share the tree with the workers").

Implicated commit: none for the crash — pre-existing race; the mechanism dates to 676b579 (Thomas Vegas) and was last tuned in c77bb48 (shasson5). The PR head [REDACTED:Hex High Entropy String] (Na Zhang, LoongArch64) is not implicated.

File: src/ucm/bistro/bistro.c:70-103 (ucm_bistro_apply_patch_atomic, non-atomic memcpy behind a 2-byte lock) and src/ucm/bistro/bistro_x86_64.c:424-442; triggered from contrib/test_jenkins.sh:1101-1105.

Suggested fix:

  1. Re-run the roce on worker 0 job — this is a known flaky race, unrelated to the loongarch64 changes, and should not block PR UCS/ARCH/LOONGARCH64: Add loongarch64 support #12187.
  2. To reduce recurrence in CI, export a larger grace period for this test in test_ucm_hooks() (e.g. UCX_BISTRO_GRACE_DURATION → ucm_global_opts.bistro_grace_duration) and/or cap threads at NPROC instead of 2*NPROC in contrib/test_jenkins.sh:1102, so the test does not run in the maximally-oversubscribed worst case.
  3. Proper fix for the underlying race: make the x86_64 patch write fully atomic — build the complete patch and publish it with a single aligned 8-byte atomic store (the near-jump patch is only 5 bytes, so an 8-byte ucs_atomic_swap64 of the function prologue would remove the window entirely) instead of lock + memcpy + unlock.
  4. Separately, fix the broken test_ucp_config invocation: make test_ucm_hooks/test_no_ib fail loudly if the binary is missing (e.g. test -x ./test/apps/.libs/test_ucp_config || exit 1), since right now a missing binary turns the IB-exclusion assertion into a silent pass — likely a stale shared build tree from the "build once per row" change.

Related: PR #12187 (this PR, LoongArch64); #11112 "UCM/BISTRO: Increase grace time to reduce patch race"; #11896 / 390a30e "AZP/TESTS: Build once per GPU test row and share the tree with the workers" (for the missing test_ucp_config binary).

🛡️ This comment had 1 potential secret(s) redacted (Hex High Entropy String). See request_id d5dd1aac-0a4b-4860-aa27-175c2234db6f in the triage console for the audit trail.

@tvegas1

tvegas1 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

This is a continuation of #9899.

@tvegas1 tvegas1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ucm_reloc_get_pointer() in src/ucm/util/reloc.c treats .dynamic entries as base offsets on RISC-V (glibc does not relocate them when .dynamic is read-only) — is LoongArch64 also in that case? The tests above only cover bistro mode; could you run reloc mode too:

UCX_MEM_MMAP_HOOK_MODE=reloc ./test/gtest/gtest --gtest_filter='malloc_hook*'

Comment thread src/ucs/arch/generic/bitops.h Resolved
Comment thread src/ucs/arch/global_opts.h Resolved
Comment thread src/ucs/arch/rv64/global_opts.h Resolved
Comment thread src/ucs/arch/bitops.h Resolved
Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
Comment thread src/ucs/arch/loongarch64/cpu.h Resolved
@tvegas1

tvegas1 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

no packaging included

@tvegas1

tvegas1 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

@loongson-zn, did you or your organisation approve the CLA?

@tvegas1

tvegas1 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

@Alexey-Rivkin, how would suggest to add building loongarch64 in CI? Maybe we could use cross-compile packages like below:

apt-get install -y --no-install-recommends \
    gcc-loongarch64-linux-gnu g++-loongarch64-linux-gnu libc6-dev-loong64-cross \
    autoconf automake libtool make pkg-config

@Alexey-Rivkin

Copy link
Copy Markdown
Contributor

@Alexey-Rivkin, how would suggest to add building loongarch64 in CI? Maybe we could use cross-compile packages like below:

apt-get install -y --no-install-recommends \
    gcc-loongarch64-linux-gnu g++-loongarch64-linux-gnu libc6-dev-loong64-cross \
    autoconf automake libtool make pkg-config

Seems feasible. It needs testing, but first we need mgmt approval.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

Comment thread src/ucs/arch/generic/bitops.h Resolved
Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

Residual coverage gaps (already raised in earlier rounds, not re-posted): no __loongarch_lp64 build or run CI, the ld.d at func_ptr + 0x10 alignment assumption, and the ucs_fatal path when CPUCFG.2 LLFTP is absent (every other arch degrades to ucs_arch_generic_get_clocks_per_sec() instead of aborting at library init).

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

Comment thread src/ucs/Makefile.am Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

Residual coverage gap: ucs_arch_clear_cache() here is only reachable when HAVE___CLEAR_CACHE is 0; on a toolchain where __clear_cache exists, the dbar 0/ibar 0 pair in this patch is never executed and bistro relies on libgcc's implementation instead. Worth confirming which of the two paths the LoongArch test runs actually exercised (not raised as a blocker since the same ambiguity exists for rv64/aarch64 and the toolchain behavior could not be confirmed from the repo).

@tvegas1 tvegas1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please confirm if you signed CLA?

Comment thread src/ucs/arch/loongarch64/cpu.h Resolved
Comment thread src/ucs/arch/loongarch64/cpu.h Resolved
Comment thread src/ucs/Makefile.am Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

Comment thread src/ucm/bistro/bistro_loongarch64.c Resolved
@svc-nvidia-pr-review

Copy link
Copy Markdown

Residual coverage gap: there is no __loongarch_lp64 build or run job in buildlib/ or contrib/test_jenkins.sh.

Residual coverage gap: the ld.d at func_ptr + 0x10 assumes an 8-byte-aligned patch site, but functions are only guaranteed 4-byte aligned (same design as rv64).

Residual coverage gap: ucs_arch_get_clocks_per_sec() calls ucs_fatal() when CPUCFG.2[LLFTP] is absent or CPUCFG.4/5 are zero, aborting at library init; every other arch degrades to ucs_arch_generic_get_clocks_per_sec().

Residual coverage gap: ucs_arch_clear_cache() is dead code when the toolchain provides __clear_cache, so the new dbar 0/ibar 0 pair may never execute.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Review complete — no issues found.

@svc-nvidia-pr-review

Copy link
Copy Markdown

🤖 Starting review — findings will be posted here when done.


/*
* LoongArch `dbar` (data barrier) hint encoding, as defined by the LoongArch
* architecture manual and the Linux kernel (arch/loongarch/include/asm/barrier.h):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: this line is 83 columns; docs/CodeStyle.md caps at 80.

Suggested change
* architecture manual and the Linux kernel (arch/loongarch/include/asm/barrier.h):
* LoongArch `dbar` (data barrier) hint encoding, as defined by the LoongArch
* architecture manual and the Linux kernel
* (arch/loongarch/include/asm/barrier.h):

@svc-nvidia-pr-review

Copy link
Copy Markdown

Not re-raised (already covered in earlier rounds): there is no __loongarch_lp64 build or run job in buildlib/ or contrib/test_jenkins.sh.

Not re-raised (already covered in earlier rounds): ld.d at func_ptr + 0x10 assumes an 8-byte-aligned patch site while functions are only guaranteed 4-byte aligned (same design as rv64).

Not re-raised (already covered in earlier rounds): ucs_arch_get_clocks_per_sec() calls ucs_fatal() when CPUCFG.2[LLFTP] is absent or CPUCFG.4/5 are zero, aborting at library init, whereas every other arch degrades to ucs_arch_generic_get_clocks_per_sec(). Related observation for context: ppc64 keeps ucs_arch_get_clocks_per_sec() out of line (arch/ppc64/timebase.c), and arch/loongarch64/cpu.c already exists, so if the fatal path is reworked, moving the function there would also keep ucs_fatal out of a header that libucm pulls in via bistro_int.h.

Not re-raised (already covered in earlier rounds): ucs_arch_clear_cache() is dead code when the toolchain provides __clear_cache, so the new dbar 0/ibar 0 pair may never execute.

@loongson-zn

Copy link
Copy Markdown
Author

Could you please confirm if you signed CLA?

@tvegas1 The link http://www.openucx.org/license is inaccessible. Is this link related to CLA?

@tvegas1 tvegas1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loongson-zn the CLA forms should be available from openucx.org, which seems currently down.

note the latest push changed the copyright from Loongson to Na Zhang — is this contributed as an individual or on behalf of Loongson? that determines which CLA applies.

@tvegas1

tvegas1 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

could you pls share reloc mode test results — ucm_reloc_get_pointer() in src/ucm/util/reloc.c treats .dynamic entries as offsets on RISC-V, need to confirm LoongArch64 doesn't need the same:

UCX_MEM_MMAP_HOOK_MODE=reloc ./test/gtest/gtest --gtest_filter='malloc_hook*'

@loongson-zn

Copy link
Copy Markdown
Author

could you pls share reloc mode test results — ucm_reloc_get_pointer() in src/ucm/util/reloc.c treats .dynamic entries as offsets on RISC-V, need to confirm LoongArch64 doesn't need the same:

UCX_MEM_MMAP_HOOK_MODE=reloc ./test/gtest/gtest --gtest_filter='malloc_hook*'
zn@zn-pc:~/codex-zn/ucx-zn/ucx1104/ucx$ UCX_MEM_MMAP_HOOK_MODE=reloc ./test/gtest/gtest --gtest_filter='malloc_hook_cplusplus.mmap_ptrs'
[     INFO ] dc_x is not available
[     INFO ] ud_v is not available
[     INFO ] ud_x is not available
[     INFO ] rc_v is not available
[     INFO ] rc_x is not available
[     INFO ] srd is not available
[     INFO ] ugni is not available
[     INFO ] dc_x,cuda_copy,rocm_copy is not available
[     INFO ] ud_v,cuda_copy,rocm_copy is not available
[     INFO ] ud_x,cuda_copy,rocm_copy is not available
[     INFO ] rc_v,cuda_copy,rocm_copy is not available
[     INFO ] rc_x,cuda_copy,rocm_copy is not available
[     INFO ] srd,cuda_copy,rocm_copy is not available
[     INFO ] ugni,cuda_copy,rocm_copy is not available
[     INFO ] dc_mlx5 is not available
[     INFO ] ib,cuda_copy,rocm_copy is not available
[     INFO ] rc_x,cuda_copy is not available
[     INFO ] rc_v,cuda_copy is not available
[     INFO ] rc_x,cuda,rocm is not available
[     INFO ] rc_x,cuda_ipc,rocm_ipc,cuda_copy,rocm_copy is not available
[     INFO ] ib,cuda_copy is not available
[     INFO ] rc,dc is not available
[     INFO ] dc_x,rc_x,cuda_copy,rocm_copy is not available
[     INFO ] rc is not available
[     INFO ] ud is not available
[     INFO ] ud_mlx5 is not available
[     INFO ] ib is not available
[     INFO ] rc_x,rc_v,ud_x,ud_v is not available
[     INFO ] dc_x,ud_x,ud_v is not available
[     INFO ] rc_x,rc_v is not available
[     INFO ] dc is not available
[     INFO ] dc_x,ud,cuda_copy,rocm_copy is not available
[     INFO ] rc_mlx5,rc_verbs,cuda_copy,rocm_copy is not available
[     INFO ] Using random seed of 27257
Note: Google Test filter = malloc_hook_cplusplus.mmap_ptrs
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from malloc_hook_cplusplus, where TypeParam = 
[ RUN      ] malloc_hook_cplusplus.mmap_ptrs <> <>
[     INFO ] With 1 large blocks: allocated 100000 buffers of 262144 bytes in 1.47776 sec
^C
zn@zn-pc:~/codex-zn/ucx-zn/ucx1104/ucx$ UCX_MEM_MMAP_HOOK_MODE=reloc ./test/gtest/gtest --gtest_filter='malloc_hook*'
[     INFO ] dc_x is not available
[     INFO ] ud_v is not available
[     INFO ] ud_x is not available
[     INFO ] rc_v is not available
[     INFO ] rc_x is not available
[     INFO ] srd is not available
[     INFO ] ugni is not available
[     INFO ] dc_x,cuda_copy,rocm_copy is not available
[     INFO ] ud_v,cuda_copy,rocm_copy is not available
[     INFO ] ud_x,cuda_copy,rocm_copy is not available
[     INFO ] rc_v,cuda_copy,rocm_copy is not available
[     INFO ] rc_x,cuda_copy,rocm_copy is not available
[     INFO ] srd,cuda_copy,rocm_copy is not available
[     INFO ] ugni,cuda_copy,rocm_copy is not available
[     INFO ] dc_mlx5 is not available
[     INFO ] ib,cuda_copy,rocm_copy is not available
[     INFO ] rc_x,cuda_copy is not available
[     INFO ] rc_v,cuda_copy is not available
[     INFO ] rc_x,cuda,rocm is not available
[     INFO ] rc_x,cuda_ipc,rocm_ipc,cuda_copy,rocm_copy is not available
[     INFO ] ib,cuda_copy is not available
[     INFO ] rc,dc is not available
[     INFO ] dc_x,rc_x,cuda_copy,rocm_copy is not available
[     INFO ] rc is not available
[     INFO ] ud is not available
[     INFO ] ud_mlx5 is not available
[     INFO ] ib is not available
[     INFO ] rc_x,rc_v,ud_x,ud_v is not available
[     INFO ] dc_x,ud_x,ud_v is not available
[     INFO ] rc_x,rc_v is not available
[     INFO ] dc is not available
[     INFO ] dc_x,ud,cuda_copy,rocm_copy is not available
[     INFO ] rc_mlx5,rc_verbs,cuda_copy,rocm_copy is not available
[     INFO ] Using random seed of 27272
Note: Google Test filter = malloc_hook*
[==========] Running 24 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 1 test from malloc_hook_test_no_events, where TypeParam = 
[ RUN      ] malloc_hook_test_no_events.empty_event <> <>
[       OK ] malloc_hook_test_no_events.empty_event (0 ms)
[----------] 1 test from malloc_hook_test_no_events (0 ms total)

[----------] 9 tests from malloc_hook, where TypeParam = 
[ RUN      ] malloc_hook.single_thread <> <>
[     INFO ] single-thread: small mapped: 10027008, total mapped: 136069120, total unmapped: 130924544
[       OK ] malloc_hook.single_thread (21 ms)
[ RUN      ] malloc_hook.multi_threads <> <>
[     INFO ] thread 7/8: small mapped: 72843264, total mapped: 792494080, total unmapped: 718356480
[     INFO ] thread 5/8: small mapped: 67715072, total mapped: 835321856, total unmapped: 762904576
[     INFO ] thread 6/8: small mapped: 59539456, total mapped: 919388160, total unmapped: 847052800
[     INFO ] thread 1/8: small mapped: 59834368, total mapped: 920993792, total unmapped: 847822848
[     INFO ] thread 0/8: small mapped: 59424768, total mapped: 1004716032, total unmapped: 935903232
[     INFO ] thread 4/8: small mapped: 78856192, total mapped: 1044905984, total unmapped: 977125376
[     INFO ] thread 2/8: small mapped: 67584000, total mapped: 1088471040, total unmapped: 1022001152
[     INFO ] thread 3/8: small mapped: 68141056, total mapped: 1088749568, total unmapped: 1022132224
[       OK ] malloc_hook.multi_threads (233 ms)
[ RUN      ] malloc_hook.asprintf <> <>
[       OK ] malloc_hook.asprintf (1 ms)
[ RUN      ] malloc_hook.fork <> <>
[     SKIP ] ("broken")
[       OK ] malloc_hook.fork (4 ms)
[ RUN      ] malloc_hook.bistro_patch <> <>
[       OK ] malloc_hook.bistro_patch (50 ms)
[ RUN      ] malloc_hook.test_event <> <>
[       OK ] malloc_hook.test_event (1 ms)
[ RUN      ] malloc_hook.test_event_failed <> <>
[     SKIP ] (RUNNING_ON_VALGRIND || !skip_on_bistro())
[       OK ] malloc_hook.test_event_failed (0 ms)
[ RUN      ] malloc_hook.test_external_event <> <>
[     SKIP ] (RUNNING_ON_VALGRIND || !skip_on_bistro())
[       OK ] malloc_hook.test_external_event (0 ms)
[ RUN      ] malloc_hook.test_event_unmap <> <>
[     SKIP ] (RUNNING_ON_VALGRIND || !skip_on_bistro())
[       OK ] malloc_hook.test_event_unmap (0 ms)
[----------] 9 tests from malloc_hook (310 ms total)

[----------] 7 tests from malloc_hook_cplusplus, where TypeParam = 
[ RUN      ] malloc_hook_cplusplus.new_delete <> <>
[       OK ] malloc_hook_cplusplus.new_delete (8 ms)
[ RUN      ] malloc_hook_cplusplus.dynamic_mmap_enable <> <>
[       OK ] malloc_hook_cplusplus.dynamic_mmap_enable (3 ms)
[ RUN      ] malloc_hook_cplusplus.dynamic_mmap_disable <> <>
[       OK ] malloc_hook_cplusplus.dynamic_mmap_disable (3 ms)
[ RUN      ] malloc_hook_cplusplus.mallopt <> <>
[     SKIP ] (MALLOC_TRIM_THRESHOLD_ is not set)
[       OK ] malloc_hook_cplusplus.mallopt (1 ms)
[ RUN      ] malloc_hook_cplusplus.mmap_ptrs <> <>
[     INFO ] With 1 large blocks: allocated 100000 buffers of 262144 bytes in 1.45008 sec
[     INFO ] With 16109 large blocks: allocated 100000 buffers of 262144 bytes in 21.0214 sec
[     INFO ] With 16109 large blocks: allocated 100000 buffers of 262144 bytes in 21.0228 sec
[     INFO ] With 16109 large blocks: allocated 100000 buffers of 262144 bytes in 21.0549 sec
^[[B[     INFO ] With 16109 large blocks: allocated 100000 buffers of 262144 bytes in 21.0278 sec
[     INFO ] With 16109 large blocks: allocated 100000 buffers of 262144 bytes in 21.0496 sec
ucm/malloc_hook.cc:1040: Failure
Failed
Failed after 5 attempts
[  FAILED  ] malloc_hook_cplusplus.mmap_ptrs, where TypeParam =  and GetParam() =  (109908 ms)
[ RUN      ] malloc_hook_cplusplus.remap_override_single_thread <> <>
[       OK ] malloc_hook_cplusplus.remap_override_single_thread (0 ms)
[ RUN      ] malloc_hook_cplusplus.remap_override_multi_threads <> <>
[       OK ] malloc_hook_cplusplus.remap_override_multi_threads (1 ms)
[----------] 7 tests from malloc_hook_cplusplus (109924 ms total)

[----------] 7 tests from malloc_hook_dlopen, where TypeParam = 
[ RUN      ] malloc_hook_dlopen.indirect_dlopen <> <>
[       OK ] malloc_hook_dlopen.indirect_dlopen (5 ms)
[ RUN      ] malloc_hook_dlopen.indirect_ucm_dlopen <> <>
[       OK ] malloc_hook_dlopen.indirect_ucm_dlopen (1 ms)
[ RUN      ] malloc_hook_dlopen.rpath_dlopen <> <>
[       OK ] malloc_hook_dlopen.rpath_dlopen (3 ms)
[ RUN      ] malloc_hook_dlopen.rpath_ucm_dlopen <> <>
[       OK ] malloc_hook_dlopen.rpath_ucm_dlopen (2 ms)
[ RUN      ] malloc_hook_dlopen.ucm_dlopen_null_dlopen <> <>
[       OK ] malloc_hook_dlopen.ucm_dlopen_null_dlopen (0 ms)
[ RUN      ] malloc_hook_dlopen.ucm_dlopen_null_ucm_dlopen <> <>
[       OK ] malloc_hook_dlopen.ucm_dlopen_null_ucm_dlopen (5 ms)
[ RUN      ] malloc_hook_dlopen.dlopen_mt_with_memtype/mt_2 <> <>
[       OK ] malloc_hook_dlopen.dlopen_mt_with_memtype/mt_2 (186 ms)
[----------] 7 tests from malloc_hook_dlopen (202 ms total)

[----------] Global test environment tear-down
[==========] 24 tests from 4 test suites ran. (110437 ms total)
[  PASSED  ] 23 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] malloc_hook_cplusplus.mmap_ptrs, where TypeParam =  and GetParam() = 

 1 FAILED TEST

I also reproduced the same failure on x86, so this is not LoongArch-specific.

@loongson-zn

Copy link
Copy Markdown
Author

@loongson-zn the CLA forms should be available from openucx.org, which seems currently down.

note the latest push changed the copyright from Loongson to Na Zhang — is this contributed as an individual or on behalf of Loongson? that determines which CLA applies.

I’m contributing in my individual capacity, not on behalf of my employer. I haven’t signed the individual CLA yet because the ICLA PDF and the license page linked from the contributor guidelines are currently inaccessible from my side. Could you please let me know whether I should wait for the site to come back online, or if there is an alternative way to sign the individual CLA? For example, is there a direct link to the ICLA PDF, or an email-based submission process? I’m happy to complete it as soon as I have access to the correct form.

@tvegas1

tvegas1 commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Maybe wait for now.

@loongson-zn

Copy link
Copy Markdown
Author

Maybe wait for now.

OK, thanks @tvegas1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants