-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
I expect some might want this for demos with an aspect of realism to the response. If set (cli or env) we could add a random inject into time.sleep to cause modulations of lag in response, especially in stream responses. This would better element an LLM , a slight pause after sending the prompt, followed by a stream 1 to 2 seconds later, with micro lags mid stream.
@danbarr , useful?
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request