Skip to content

Commit 12545a3

Browse files
committed
Should use pos_ to text_token_generator
1 parent 7ebb46b commit 12545a3

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

extension/llm/runner/text_llm_runner.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ Error TextLLMRunner::generate(
190190
// Generate max_new_tokens - 1 because prefill already generated 1 token.
191191
auto generate_result = text_token_generator_->generate(
192192
prompt_tokens,
193-
num_prompt_tokens,
193+
pos_,
194194
max_new_tokens - 1,
195195
temperature_ == -1.0f ? config.temperature : temperature_,
196196
wrapped_callback);

0 commit comments

Comments
 (0)