-
Notifications
You must be signed in to change notification settings - Fork 893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefer global limit over model-specific default #1893
Prefer global limit over model-specific default #1893
Conversation
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose the main question is who has already implemented this. It might be easy to miss by SDK implementors.
Otherwise I think that model-explicit > global-explicit > model-default > global-default
sounds good. Unless more people think that in general global limits are too dangerous, and we'd rather only have the model-specific limits.
We implemented this in the Python SDK and most people who reviewed it were surprised by the fact that it prefers model-specific default over user specified global. |
Right, so when I was writing that rule I considered both cases, and went model-specific defaults, so there's a smaller chance of a user shooting themselves in the foot, by setting something globally and not understanding consequences. But good to hear that it was tested in practice, and now we now which way is preferred. |
Co-authored-by: Nikita Salnikov-Tarnovski <[email protected]>
1f963e2
to
011141e
Compare
Fixes #1878
Changes
Prefer global limit over model-specific limit default when model-specific limit is not set.