Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream LLM response when editing apps #430

Merged
merged 7 commits into from
Oct 29, 2024
Merged

Stream LLM response when editing apps #430

merged 7 commits into from
Oct 29, 2024

Conversation

benjreinhart
Copy link
Contributor

Follow ups:

  • Change creation flow immediately redirect to created app and use edit flow to get streaming support
  • Stream partial plan description markdown text
  • Break response into more granular pieces we can display on client for more responsiveness

@nichochar
Copy link
Contributor

LGTM. Given how this works, we should be able to populate the DiffBox temporarily with "updating ${filename}..."

Although we need to be careful to not make that clickable otherwise if users click on the file we'll interrupt the request

@nichochar
Copy link
Contributor

I realized that we could also push the file streams another way:

  • we could be pushing them through the websocket "file:updated"
  • this would mean they update real time
  • at the end of the request, we could still send the total diff plan like we did before
    I think this might work well, but might present issues based on the ordering of the LLM's response tags

@benjreinhart
Copy link
Contributor Author

Yes for sure, we can improve this a lot beyond this first pass.

@benjreinhart benjreinhart merged commit a6bfde7 into main Oct 29, 2024
2 checks passed
@benjreinhart benjreinhart deleted the streaming branch October 29, 2024 23:22
nichochar added a commit that referenced this pull request Oct 30, 2024
nichochar added a commit that referenced this pull request Oct 30, 2024
nichochar added a commit that referenced this pull request Oct 31, 2024
* Streaming WIP

* More wip

* Fix up the streaming parser and the tests

* Remove extra debug log

* Remove unused file

* Fix more parser bugs

* Hook up frontend

---------

Co-authored-by: Nicholas Charriere <[email protected]>
nichochar added a commit that referenced this pull request Oct 31, 2024
* Stream LLM response when editing apps (#430)

* Streaming WIP

* More wip

* Fix up the streaming parser and the tests

* Remove extra debug log

* Remove unused file

* Fix more parser bugs

* Hook up frontend

---------

Co-authored-by: Nicholas Charriere <[email protected]>

* Add changes to bugfix

* Saving progress. Some things work but tags are wonky

* Fix streaming bugs

* Remove debug log

---------

Co-authored-by: Ben Reinhart <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants