Skip to content

perf: make the CHUNKED_PREFILL_SIZE=1024 default conditional on context > native (+15…23% prefill at 262144) - #15

Closed
sethforprivacy wants to merge 1 commit into
MiaAI-Lab:mainfrom
sethforprivacy:pr-chunk-native-default
Closed

perf: make the CHUNKED_PREFILL_SIZE=1024 default conditional on context > native (+15…23% prefill at 262144)#15
sethforprivacy wants to merge 1 commit into
MiaAI-Lab:mainfrom
sethforprivacy:pr-chunk-native-default

perf: default CHUNKED_PREFILL_SIZE per context — 4096 at native, 1024…

0919ab0
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs