You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
---原始邮件---
发件人: ***@***.***>
发送时间: 2024年9月28日(周六) 中午1:40
收件人: ***@***.***>;
抄送: ***@***.******@***.***>;
主题: Re: [labring/FastGPT] 请增加下参数num_ctx用来对接ollama本地模型 (Issue #2812)
看文档
image.png (view on web)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: ***@***.***>
oneapi v0.6.9-alpha版本,增加了2个处理ollama上下文长度和max_token的参数
支持通过max_tokens(对应ollama的num_predict)参数限制输出token数
支持通过num_ctx(ollama原生参数)参数设定更改ollama默认context window大小(默认仅1k/2k上下文)
麻烦fastgpt增加下这个参数,多谢
参考oneapi issue .
songquanpeng/one-api#1694
curl如下
通过curl测试新增加的参数num_ctx,可以正常传递给ollama
curl --location --request POST 'http://172.22.1.39:3001/v1/chat/completions'
--header 'Authorization: Bearer sk-iYrST123'
--header 'Content-Type: application/json'
--data '{
"model": "qwen2:latest",
"stream": false,
"temperature": 1,
"max_tokens": 2000,
"num_ctx":7777,
"messages": [
{
"role": "user",
"content": "你是谁"
}
]
}'
The text was updated successfully, but these errors were encountered: