Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference endpoint #118

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open

Inference endpoint #118

wants to merge 30 commits into from

Conversation

andimarafioti
Copy link
Member

Code for the deployment to an inference endpoint.

I think here we need to be careful with a few changes to the main models that are good, but need to be consistent. For example, I'm yielding "Done" after the language model and the speech model are done. That allows me to have the client know that the server is done. It could also help us with the local and the ssh server approach. But we need to make sure that it works for them and it doesn't break. Also that all the TTS/language models handle this similarly.

@andimarafioti
Copy link
Member Author

Another thing to consider is that this client and endpoint handler make sense with the webservice that I have over at https://github.com/andimarafioti/speech-to-speech-inference-toolkit/pull/1/files so maybe we want to get those files here as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant