Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update /props with "total_slots" value #5373

Merged
merged 6 commits into from
Feb 7, 2024
Merged

Update /props with "total_slots" value #5373

merged 6 commits into from
Feb 7, 2024

Conversation

jparkerweb
Copy link
Contributor

  • update server/README /props endpoint section
  • remove num_slots from default_generation_settings_for_props
  • add total_slots to /props json directly:
    json data = {
        { "user_name",      llama.name_user.c_str() },
        { "assistant_name", llama.name_assistant.c_str() },
        { "default_generation_settings", llama.default_generation_settings_for_props },
        { "total_slots",    llama.params.n_parallel }
    };

image

@jparkerweb jparkerweb mentioned this pull request Feb 6, 2024
Merged
z80maniac
z80maniac reviewed Feb 6, 2024
View reviewed changes
examples/server/README.md Outdated
@@ -276,13 +276,15 @@ Notice that each `probs` is an array of length `n_probs`.
{
"assistant_name": "",
"user_name": "",
"default_generation_settings": { ... }
"default_generation_settings": { ... },
"total_slots": ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nitpick, but it should probably be "total_slots": 1, since it's an integer, not a string.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@ggerganov ggerganov merged commit f3e2b4f into ggerganov:master Feb 7, 2024
53 checks passed
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Mar 13, 2024
@jparkerweb @jordankanter
server : update /props with "total_slots" value (ggerganov#5373)
fe73ead 
* include total "num_slots" in default_generation_settings_for_props

* cleanup total_slots return value in /props endpoint

* update /props endpoint docs with total_slots

* remove num_slots from default_generation_settings_for_props

* update /props endpoint section
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
@jparkerweb @hodlen
server : update /props with "total_slots" value (ggerganov#5373)
7808c1b 
* include total "num_slots" in default_generation_settings_for_props

* cleanup total_slots return value in /props endpoint

* update /props endpoint docs with total_slots

* remove num_slots from default_generation_settings_for_props

* update /props endpoint section
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@z80maniac z80maniac z80maniac left review comments

@justin-parker justin-parker justin-parker left review comments

@ggerganov ggerganov ggerganov approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jparkerweb @ggerganov @z80maniac @justin-parker