[nixl staging buffer] fix decode hung issue when multiple chunks arrived out of order

[fteng-NV]

for nixl enable staging buffer.

Motivation

While investigating SGLang PD performance with nixl backend, a reproducible decode-side hung issue appeared, introduced by staging buffer.

Issue description

SGLang asymmetric PD (prefill tp=4, decode tp=8) with nixl backend, staging buffer enabled, ril=16384, chunked prefill size=8192 (i.e. 2 chunks per request).

When num_prompts >=64, there is a high probability that decode instance hung until router time out.

Root cause analysis

For decode instance, it checks if all chunks completed in staging buffer by _maybe_submit_last_scatter() with condition is_last_chunk is True. This means it assumes the last chunk's notification arrives last.

However when staging buffer enabled, for multiple chunks, there is a high chance that chunks arriving out of order, which means the last chunk cannot guarantee to be the last arrival.

If an earlier chunk arrived later than the last chunk, there is no chance to call _maybe_submit_last_scatter() so decode instance stuck by chunks never completed until router time out.

Modifications

The solution is simple, just move the logic calling _maybe_submit_last_scatter() out of "is_last_chunk is True” condition.

Accuracy Tests

I have verified after the change, hung issue disappeared and decode instance could work normally.

---

CI States

Latest PR Test (Base): ❌ Run #33709685390
Latest PR Test (Extra): ❌ Run #33709685197
Latest PR Test (AMD ROCm 7.2): ❌ Run #33709685368