Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix starvation with async server and interleaving optimization #13

Merged
merged 9 commits into from
Mar 27, 2024
Merged

Fix starvation with async server and interleaving optimization #13

merged 9 commits into from
Mar 27, 2024

Commits on Mar 15, 2024

  1. Fix starvation with async server

    @JoeZijunZhou
    JoeZijunZhou committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    cf00e88 View commit details
    Browse the repository at this point in the history

  2. keep grpc option

    @JoeZijunZhou
    JoeZijunZhou committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    cf09132 View commit details
    Browse the repository at this point in the history

  3. rm default for return_channel

    @JoeZijunZhou
    JoeZijunZhou committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    fde895e View commit details
    Browse the repository at this point in the history

  4. revert rm default for return_channel

    @JoeZijunZhou
    JoeZijunZhou committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    bb96d36 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Using blocking fixed size queue to block and yield threads efficiently

    @JoeZijunZhou
    JoeZijunZhou committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    a4d7f32 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. fix AsyncMultifuture

    @JoeZijunZhou
    JoeZijunZhou committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ab4277b View commit details
    Browse the repository at this point in the history

  2. complete fix - optimized interleaving prefill, insert, and generate

    @JoeZijunZhou
    JoeZijunZhou committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    77dd514 View commit details
    Browse the repository at this point in the history

  3. fix unit test and pytype

    @JoeZijunZhou
    JoeZijunZhou committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    09e0e65 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. add TODO

    @JoeZijunZhou
    JoeZijunZhou committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    a9a9a9d View commit details
    Browse the repository at this point in the history