Skip to content

UCP/PROTO: Keep latency singles off lanes beyond the fast-path array - #11808

Merged
roiedanino merged 1 commit into
openucx:masterfrom
evgeny-leksikov:ucp_proto_eager_lanes_indexing_quard
Aug 25, 2026
Merged

roiedanino merged 1 commit into
openucx:masterfrom
evgeny-leksikov:ucp_proto_eager_lanes_indexing_quard

Conversation

@evgeny-leksikov

Copy link
Copy Markdown
Contributor

What?

Exclude slow lanes from fast protocols selection

Why?

Progress for AM/TAG/RMA/AMO single protos uses ucp_ep_get_fast_lane(), which cannot index lanes stored in ep->ext.

How?

Exclude those lanes at probe time so BW/multi protocols cover them until failover recovery.

Progress for AM/TAG/RMA/AMO single protos uses ucp_ep_get_fast_lane(), which
cannot index lanes stored in ep->ext. Exclude those lanes at probe time so
BW/multi protocols cover them until failover recovery.
Comment thread src/ucp/rma/amo_sw.c
.super.flags = flags | UCP_PROTO_COMMON_INIT_FLAG_SINGLE_FRAG |
UCP_PROTO_COMMON_INIT_FLAG_CAP_SEG_SIZE,
.super.exclude_map = 0,
.super.exclude_map = ~UCP_MAX_FAST_PATH_LANES_MASK,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amo_sw is lane_type = AM. If all AM lanes < 5 have failed and the only
survivor is a promoted am_bw lane >= 5, AMO now has no protocol at all instead
of using that lane. Clean error or fatal?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same as fast path AM, SW AMO accesses only fast lane:

req->send.lane = ucp_ep_get_am_lane(req->send.ep);

@roiedanino roiedanino left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, consider adding ft test forcing am_lane past first 5

@evgeny-leksikov

Copy link
Copy Markdown
Contributor Author

roved, consider adding ft test forcing am_lane past first

This path will be enabled automatically once we enable ERR_MODE_FAILOVER for AM-short and AM-bcopy protocols.

@roiedanino
roiedanino merged commit 3942d1c into openucx:master Aug 25, 2026
162 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants