Skip to content

Commit 42efa74

Browse files
committed
Update trtllm-bench documentation on setting FP8 KV cache
Signed-off-by: Aurelien Chartier <[email protected]>
1 parent 9f1d9b7 commit 42efa74

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

docs/source/developer-guide/perf-benchmarking.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -460,7 +460,8 @@ If you would like to force the KV cache quantization, you can specify the follow
460460
when the checkpoint precision is `null`:
461461

462462
```yaml
463-
kv_cache_dtype: "fp8"
463+
kv_cache_config:
464+
dtype: fp8
464465
```
465466
466467
```{tip}

0 commit comments

Comments
 (0)