Skip to content

Commit 98726a3

Browse files
authored
[None][chore] Update trtllm-bench documentation on setting FP8 KV cache (#7885)
Signed-off-by: Aurelien Chartier <[email protected]>
1 parent 336c2ef commit 98726a3

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

docs/source/developer-guide/perf-benchmarking.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -460,9 +460,10 @@ If you would like to force the KV cache quantization, you can specify the follow
460460
when the checkpoint precision is `null`:
461461

462462
```yaml
463-
kv_cache_dtype: "fp8"
463+
kv_cache_config:
464+
dtype: fp8
464465
```
465466
466467
```{tip}
467-
The two valid values for `kv_cache_dtype` are `auto` and `fp8`.
468+
The two valid values for `kv_cache_config.dtype` are `auto` and `fp8`.
468469
```

0 commit comments

Comments
 (0)