Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memoise calls to fullyQualifiedNameToSwaggerName to speed it up for large registries #421

Merged
merged 2 commits into from
Jul 24, 2017

Conversation

peterebden
Copy link
Contributor

We're observing some very slow compile times (on the order of ~1 minute) for reasonably large protos, containing hundreds of messages or so.

This implements a very simple cache of calls to determine Swagger names inside fullyQualifiedNameToSwaggerName which appears to be taking most of the time; after this change compilation time is well under 1 second.

@tmc
Copy link
Collaborator

tmc commented Jun 26, 2017

@peterebden thank you for your contribution, have you signed the CLA?

@peterebden
Copy link
Contributor Author

Yes I have - I can see it on cla.developers.google.com.

@tmc
Copy link
Collaborator

tmc commented Jun 27, 2017

@peterebden thanks, for whatever reason the cla bot needs a comment to kick in.

// lastRegistry is used to remember the last registry passed into fullyQualifiedNameToSwaggerName;
// we use that to memoise calls to resolveFullyQualifiedNamesToSwaggerNames since it can be
// significantly slow for registries of hundreds of entries.
var lastRegistry *descriptor.Registry
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we instead do something like var fqmnCache = map[*descriptor.Registry]map[string]string so we can avoid the extra 'last' concept and just compute the cache once per registry seen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done. I did think about doing something like that at first but in the end got a little bit lazy with it :)

@tmc
Copy link
Collaborator

tmc commented Jul 16, 2017

@peterebden ping

@peterebden
Copy link
Contributor Author

I don't seem to be able to merge this - maybe cla-bot needs another message?

@achew22
Copy link
Collaborator

achew22 commented Jul 24, 2017

Looks like the CLA bot is happy now. Merging (sorry for the delay)

@achew22 achew22 merged commit f2862b4 into grpc-ecosystem:master Jul 24, 2017
adasari pushed a commit to adasari/grpc-gateway that referenced this pull request Apr 9, 2020
…arge registries (grpc-ecosystem#421)

Memoise calls to fullyQualifiedNameToSwaggerName for all registries ever seen instead of just the last one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants