Skip to content

Commit

Permalink
server : include total "num_slots" in props endpoint (ggerganov#5349)
Browse files Browse the repository at this point in the history
  • Loading branch information
jparkerweb authored and jordankanter committed Mar 13, 2024
1 parent 2f89416 commit d7ca5d5
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions examples/server/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,7 @@ struct llama_server_context
}

default_generation_settings_for_props = get_formated_generation(slots.front());
default_generation_settings_for_props["num_slots"] = params.n_parallel;
default_generation_settings_for_props["seed"] = -1;

batch = llama_batch_init(n_ctx, 0, params.n_parallel);
Expand Down

0 comments on commit d7ca5d5

Please sign in to comment.