Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BOUNTY - $200] make it so process_prompt is cancellable outside of a download (follow up to #305) #307

Open
AlexCheema opened this issue Oct 7, 2024 · 5 comments · May be fixed by #407
Open

[BOUNTY - $200] make it so process_prompt is cancellable outside of a download (follow up to #305) #307

AlexCheema opened this issue Oct 7, 2024 · 5 comments · May be fixed by #407

Comments

@AlexCheema
Copy link
Contributor

See #305

Currently the fix is to shield process_prompt however this also makes it so that if a chatgpt api request times out, it never gets cancelled even when a download is not in progress.

This can lead to behaviour where exo keeps generating even after a chatgpt api request has timed out.

Gotchas: make sure however exo works right is actually cancellable end-to-end. If an inference is in progress, that needs to be cancellable - InferenceEngines need to support this behaviour.
@AlexCheema AlexCheema mentioned this issue Oct 7, 2024
Closed
@ji-cryptocafe
Copy link

Started working on this. But since holidays are near, it could take a few days.

@rahat2134
Copy link
Contributor

Hi @AlexCheema,

I'd like to work on this issue. I understand the main goal is to make process_prompt cancellable when not downloading while maintaining download protection from #305.

Would you like me to proceed with the implementation, so can you assign this issue to me?

@rahat2134
Copy link
Contributor

@AlexCheema PTAL!

@rahat2134
Copy link
Contributor

@AlexCheema

@rahat2134
Copy link
Contributor

@AlexCheema , I have started working on this. Please assign this to me.

@rahat2134 rahat2134 linked a pull request Nov 3, 2024 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make process_prompt Cancellable Outside Downloads rahat2134/exo
3 participants
@AlexCheema @ji-cryptocafe @rahat2134