We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 7ebb46b commit 12545a3Copy full SHA for 12545a3
extension/llm/runner/text_llm_runner.cpp
@@ -190,7 +190,7 @@ Error TextLLMRunner::generate(
190
// Generate max_new_tokens - 1 because prefill already generated 1 token.
191
auto generate_result = text_token_generator_->generate(
192
prompt_tokens,
193
- num_prompt_tokens,
+ pos_,
194
max_new_tokens - 1,
195
temperature_ == -1.0f ? config.temperature : temperature_,
196
wrapped_callback);
0 commit comments