[william-gallagher-nv]

What?

Default to merging rcache regions. Opt out available by setting UCX_RCACHE_MERGE_ADJACENT=n.

Why?

[svc-nvidia-pr-review]

🤖 Starting review — findings will be posted here when done.

[svc-nvidia-pr-review]

for host memory ucp_mem_rcache_can_merge_cb() returns 1 regardless of the underlying allocation, so two unrelated adjacent buffers can become one region, and freeing one of them invalidates the registration of the other. was this measured on host workloads with mixed buffer lifetimes, or only on the merge-friendly case?

[svc-nvidia-pr-review]

minor: now that this is the default, modify_config("RCACHE_MERGE_ADJACENT", "y") in test_ucp_rcache_merge is a no-op and nothing covers the disabled path. can we add a variant with n there?

[svc-ucx]

🤖 CI Triage Agent — UCX PR (Build build_x86_64 centos7) · commit 31921dfc

TL;DR: The job never built anything — it died in the Azure Pipelines "Initialize containers" step because docker inspect on rdmz-harbor.rdmz.labs.mlnx/ucx/centos7:5 returned "No such object" on agent swx-rdmz-instinct01; this is an agent/Docker infrastructure problem, not a code regression from PR #12267, so simply re-run the pipeline (and clean the agent's Docker image cache).

Full analysis

Summary: Build build_x86_64 centos7 (build 143175) failed during container initialization with Exit code 1 returned from process: file name '/scrap/bin/docker', arguments 'inspect --format=... rdmz-harbor.rdmz.labs.mlnx/ucx/centos7:5'.

Root cause: Infrastructure/Docker-state inconsistency on the build agent. The log shows docker pull rdmz-harbor.rdmz.labs.mlnx/ucx/centos7:5 returning Status: Image is up to date (22:01:16.412) — i.e. the pull was a no-op based on the local manifest — but the immediately following docker inspect of the same tag failed with Error: No such object: rdmz-harbor.rdmz.labs.mlnx/ucx/centos7:5 (22:01:19.110). The image reference was therefore not actually resolvable locally: either the local image store is corrupt/partially GC'd, or a concurrent image prune/docker rmi on the shared agent (swx-rdmz-instinct01, slot -05, multiple concurrent jobs do docker network prune/cleanup) removed the image between the pull and the inspect. No source compilation, no test, and not even the git checkout of the PR merge commit happened before the failure (checkout ran only after the container step already errored, and "Stop Containers"/"Finalize Job" followed within 2 seconds). There is no evidence of any UCX code involvement — the whole job lasted ~7 seconds and the total log is pure agent bootstrap. This is also not a timeout: there are no gaps larger than ~3 seconds anywhere in the log.

Implicated commit: unknown — not a code regression; commit 31921df / PR #12267 code was never compiled or executed in this build.

File: unknown (failure is in the Azure agent container-initialization step, before any repo file is used; pipeline definition: buildlib/azure-pipelines-pr.yml)

Suggested fix: Re-run the failed job — this class of failure is transient. If it recurs on swx-rdmz-instinct01, have CI admins clean up that agent's Docker state (docker image prune -a then re-pull rdmz-harbor.rdmz.labs.mlnx/ucx/centos7:5, or docker rmi the stale tag to force a full re-pull) and check for cron/other jobs running docker system prune concurrently with pipeline jobs on that shared host, since the agent's own prune --filter "label=f35170" plus an unfiltered external prune would explain an image disappearing between pull and inspect. As a hardening step, consider adding a pull retry/docker image inspect pre-check (or setting a non-shared image cache per agent slot) in the container resource setup.

Related: #12094 (CI: Jenkins integration testing) — only loosely related CI infrastructure work; no existing issue tracks this specific Docker No such object flake.