Commit 018449a
[KVCache] Fix the reference counter in sequence fork (apache#16666)
This PR fixes a sequence reference counter bug in the KV cache:
when forking a child sequnece from an existing parent sequence,
the reference counter of hte parent sequence was not increased.
This leads to error when the child sequence is removed, where we
will check the parent's reference counter and find it is 0 and is
never changed unexpectedly.
Meanwhile, this PR updates the PagedKVCache tests with some latest
changes, including target-aware tile size selection.1 parent 82f7d57 commit 018449a
File tree
2 files changed
+177
-114
lines changed- src/runtime/relax_vm
- tests/python/relax
2 files changed
+177
-114
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
475 | 475 | | |
476 | 476 | | |
477 | 477 | | |
| 478 | + | |
478 | 479 | | |
479 | 480 | | |
480 | 481 | | |
| |||
0 commit comments