Cache packed sequence metadata to reduce D2H syncs across layers - #4243

Merged

danielhanchen merged 5 commits into

mainfrom

dh/recover-4173-packing-cache

Mar 12, 2026

Commits on Mar 12, 2026

packing optimziation with cache to reduce D2H copy

ruixiang63
authored and
danielhanchen
committed
2efe2ef
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
danielhanchen
committed
837b5f8
cache per device to avoid race condition for multi-gpu

ruixiang63
authored and
danielhanchen
committed
67930ba
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
danielhanchen
committed
ecc5106
add cache freeing up func

ruixiang63
authored and
danielhanchen
committed
60da93d