Improve role fetching and unmarshaling#42007
Merged
espadolini merged 2 commits intomasterfrom May 24, 2024
Merged
Conversation
strideynet
approved these changes
May 24, 2024
tigrato
approved these changes
May 24, 2024
Contributor
Author
|
For the record: the existing implementation sits at 27651 ns/op, this one with |
zmb3
approved these changes
May 24, 2024
|
@espadolini See the table below for backport results.
|
espadolini
added a commit
that referenced
this pull request
May 27, 2024
* Reduce default ListRoles page size * Avoid stdjson in UnmarshalRoleV6
espadolini
added a commit
that referenced
this pull request
May 27, 2024
* Reduce default ListRoles page size * Avoid stdjson in UnmarshalRoleV6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the default page size for
ListRoles(that the API client uses by default, and by extent the cache initialization process) from 1000 to 100; roles can require a lot of effort during unmarshaling (as we validate expressions, which requires parsing them) and after a cluster event such as an auth server restart, a lot of nodes will all connect at the same time and get roles 1000 at a time, which can spike the memory used by the auth server. The default iteration page size of 1000 was picked somewhat arbitrarily and works well for resources that are just data with very little effort required during unmarshaling, but roles can be pretty big and hang around for a while, so tuning the page size down can help mitigate these memory requirement bursts.In addition, this PR changes the version checking for roles to use
jsoniter.Getto access the one field we need, rather than usingencoding/json's unmarshaler for a whole resource header. A simple benchmark that unmarshals a reasonably "average" role with onewhereexpression shows a 28% improvement in time spent inservices.UnmarshalRoleV6, with a tiny reduction in allocations as well.Benchmark result:
CPU profile flame graph for a relative comparison (with FastUnmarshal and ValidateRole being the same in both):
changelog: reduced memory and cpu usage after control plane restarts in clusters with a high number of roles