We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
例行检查
问题描述 本地Qwen2-72B-Instruct-GPTQ-Int8模型,stream=true情况下无返回,stream=false能正常返回。 换gpt-3.5-tubo,两种场景都可以返回。 复现步骤 oneapi配置的gpt-3.5-turbo curl --location '10.81.1.66:3001/v1/chat/completions' --header 'Content-Type: application/json' --header 'Accept: text/event-stream' --header 'Authorization: Bearer sk-dyjZYJ8xdzcFPp8y5597E57eA5354a808bE82dC4D1982515' --data '{ "model": "gpt-3.5-turbo", "temperature": 1, "max_tokens": 512, "stream": true, "messages": [ { "role": "user", "content": "1+98等于几" } ] }'
oneapi配置的qwen2 curl --location '10.81.1.66:3001/v1/chat/completions' --header 'Content-Type: application/json' --header 'Accept: text/event-stream' --header 'Authorization: Bearer sk-dyjZYJ8xdzcFPp8y5597E57eA5354a808bE82dC4D1982515' --data '{ "model": "qwen2-72b-local", "stream": true, "messages": [ { "role": "user", "content": "1+98等于几" } ] }'
预期结果 都能流式正常返回 相关截图 上图是不通过oneapi,直接访问模型,能正常流式输出,结果如下: 上图:通过onenapi,流式访问本地qwen模型,无返回内容
上图通过oneapi访问gpt-3.5-turbo,能正常返回,如下图:
如果没有的话,请删除此节。
The text was updated successfully, but these errors were encountered:
No branches or pull requests
例行检查
问题描述
本地Qwen2-72B-Instruct-GPTQ-Int8模型,stream=true情况下无返回,stream=false能正常返回。
换gpt-3.5-tubo,两种场景都可以返回。
复现步骤
oneapi配置的gpt-3.5-turbo
curl --location '10.81.1.66:3001/v1/chat/completions'
--header 'Content-Type: application/json'
--header 'Accept: text/event-stream'
--header 'Authorization: Bearer sk-dyjZYJ8xdzcFPp8y5597E57eA5354a808bE82dC4D1982515'
--data '{
"model": "gpt-3.5-turbo",
"temperature": 1,
"max_tokens": 512,
"stream": true,
"messages": [
{
"role": "user",
"content": "1+98等于几"
}
]
}'
oneapi配置的qwen2
curl --location '10.81.1.66:3001/v1/chat/completions'
--header 'Content-Type: application/json'
--header 'Accept: text/event-stream'
--header 'Authorization: Bearer sk-dyjZYJ8xdzcFPp8y5597E57eA5354a808bE82dC4D1982515'
--data '{
"model": "qwen2-72b-local",
"stream": true,
"messages": [
{
"role": "user",
"content": "1+98等于几"
}
]
}'
预期结果
![image](https://private-user-images.githubusercontent.com/14832411/343621519-7a05448b-38cf-4aef-ac96-d53bde97bf0a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAzOTU0NjcsIm5iZiI6MTcyMDM5NTE2NywicGF0aCI6Ii8xNDgzMjQxMS8zNDM2MjE1MTktN2EwNTQ0OGItMzhjZi00YWVmLWFjOTYtZDUzYmRlOTdiZjBhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzA3VDIzMzI0N1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTUwMzRmMDg5NzUzNWU1YWU5NjI1NmZlMmE0ZDUzZDRjYmQ2MTAzNThkNDhiYmM3OTBlMDhjYjgzMjZhMzA0ZDAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.Wbta9b3TIlScTOBEgnKyxmUlSRVAqRI7Tsb_JRRHqKs)
![image](https://private-user-images.githubusercontent.com/14832411/343621690-b5d48358-6733-45c2-9ea6-2e45b07a2e52.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAzOTU0NjcsIm5iZiI6MTcyMDM5NTE2NywicGF0aCI6Ii8xNDgzMjQxMS8zNDM2MjE2OTAtYjVkNDgzNTgtNjczMy00NWMyLTllYTYtMmU0NWIwN2EyZTUyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzA3VDIzMzI0N1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRkZWUxMTc3N2M5NzkwMjNiMjkwZGE5MTBjMzkzMzBhYTY0Njk4NDNhODhjMzc0YTI2Njg2MGMwNzViMDQ0NmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.798N1KvopU6LWGD2JP5ZLkB5XmZoAnzZ-b_ywgtLNH4)
![image](https://private-user-images.githubusercontent.com/14832411/343622667-b7e4cf85-89aa-40dc-b808-9c0d0a67a11f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAzOTU0NjcsIm5iZiI6MTcyMDM5NTE2NywicGF0aCI6Ii8xNDgzMjQxMS8zNDM2MjI2NjctYjdlNGNmODUtODlhYS00MGRjLWI4MDgtOWMwZDBhNjdhMTFmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzA3VDIzMzI0N1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTFhY2E1MWRhOTI1ZTNlNjVjYmZlNDAxNDg1YzE2Njg4OGQ3ZjcwOWZlZTI2MDVhYzc3MTkyOWQzYjMxNmJmZWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.V8nKe33EvhNPkUgtvrTMaANHvo-frzUH2i609zoYq9c)
都能流式正常返回
相关截图
上图是不通过oneapi,直接访问模型,能正常流式输出,结果如下:
上图:通过onenapi,流式访问本地qwen模型,无返回内容
上图通过oneapi访问gpt-3.5-turbo,能正常返回,如下图:
如果没有的话,请删除此节。
The text was updated successfully, but these errors were encountered: