add unit test for prefix cache tracking scorer

[zhengkezhou1]

fix #218

[elevran]

/assign @vMaroon

[elevran]

@zhengkezhou1 kindly please rebase and sign your commits

[elevran]

/cc @vMaroon

[zhengkezhou1]

all commits are signed now.

[elevran]

the commits do not appear to be signed/verified yet.
We're looking for the verified marking which is is still missing. It should look something like the below.

See https://docs.github.com/en/authentication/managing-commit-signature-verification

[zhengkezhou1]

ok, its works now right?

[vMaroon]

The precise prefix-cache scorer is a wrapper of the kvcache.Indexer. The wrapping logic is mostly normalization.

What this test ends up doing is test the latter - which is great, but can be achieved by testing the indexedScoresToNormalizedScoredPods function directly instead of using a mock PodScorer interface.

I think this test should verify correctness of the scorer as-is. This would require:

1. Access to the indexer's backing kvblock.Index
2. Having the test populate the kvblock.Index with kv-block information
3. Testing correctness of the scorer as-is

[zhengkezhou1]

Adding data to the kvblock.Index alone isn't sufficient; we also need to add data to the tokensIndexer. However, the kv-cache-manager currently only allows access to the KVBlockIndex. Can we add a method to the kv-cache-manager to expose the tokensIndexer?

[vMaroon]

You can add data to the tokenization cache by making the calls in a "warmup" step (call once, get no scores). Note that this behavior will change soon once tokenization is synchronous, after-which there would be no need for such a step.

[elevran]

@vMaroon - have your comments been addressed? If so, please approve so we can merge.
@zhengkezhou1 unfortunately this requires another rebase...

[github-actions]

This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity, it will be closed. To prevent this PR from being closed, add a comment or remove the lifecycle/stale label.