Repository navigation
UCT/IB/MLX5: Fix put_sgl_zcopy completion - #11893
Conversation
Associate the completion with the last WQE in an SGL PUT so partial recovery cannot complete the operation prematurely. Remove the unrelated RDMA read resource check from the PUT path.
|
🤖 Starting review — findings will be posted here when done. |
| uct_rc_ep_send_op_completion_handler, comp, sn, | ||
| UCT_RC_IFACE_SEND_OP_FLAG_ZCOPY, NULL, 0, total); | ||
| uct_rc_ep_send_op_completion_handler, comp, | ||
| txwq->sig_pi, UCT_RC_IFACE_SEND_OP_FLAG_ZCOPY, |
There was a problem hiding this comment.
Change 1 — sn → txwq->sig_pi in uct_rc_txqp_add_send_comp: Correct and an improvement. Only the last WQE in the batch carries MLX5_WQE_CTRL_CQ_UPDATE, and its completion reports hw_ci = ntohs(cqe->wqe_counter) equal to that WQE's first BB index (prev_sw_pi after the loop). txwq->sig_pi is set to exactly txwq->prev_sw_pi, so UCS_CIRCULAR_COMPARE16(sig_pi, <=, hw_ci) matches on the right CQE. This aligns the SGL path with the signaled-PI convention used by uct_rc_mlx5_common_post_send, _flush, and _post_check. Removing the now-unused sn local is clean.
|
Residual coverage note: there is no test change in the diff. Since this touches completion-PI matching and a resource-check removal, confirm existing |
|
🤖 Starting review — findings will be posted here when done. |
|
Residual coverage note: The diff contains no test change. This is a completion-PI semantics change on the SGL zcopy path. Existing coverage exists in test/gtest/uct/v2/test_uct_sgl.cc (and the cuda_ipc device test), which exercise uct_ep_put_sgl_zcopy. Worth confirming those tests run multi-element batches with completion callbacks against the RC mlx5 path so the sig_pi-based completion is actually exercised; if the gtest only covers single-element or the plugin/ext path, the base multi-WQE completion change would be untested. |
|
🤖 Starting review — findings will be posted here when done. |
|
Completion PI change looks correct: switching the completion Residual coverage note: the diff contains no test change. This is a completion-PI semantics change plus a resource-check edit on the multi-WQE |
bf497a0 to
022ed4a
Compare
|
🤖 Starting review — findings will be posted here when done. |
|
Residual coverage note (already raised previously): the diff still contains no test change for the multi-WQE |
022ed4a to
48aa836
Compare
48aa836 to
cec4aa2
Compare
|
🤖 Starting review — findings will be posted here when done. |
|
🤖 Review complete — no issues found. |
What?
Associate the completion with the last WQE in an SGL PUT so partial recovery cannot complete the operation prematurely.
Why?
During FT, comp could called before all WQEs consumed, since it's associated with 1st WQE.
How?
Remove the unrelated RDMA read resource check from the PUT path.