Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module loader: Reduce parallelism to 1/1 in all cases #752

Merged
merged 1 commit into from
Jan 5, 2022

Conversation

radeksimko
Copy link
Member

@radeksimko radeksimko commented Jan 4, 2022

This will cause indexing to take longer to finish on machines with >2 CPUs, but it should flatten the CPU spikes some users have reported in #733

It is worth noting that this puts a bit more pressure on resolving #5 / #150

@radeksimko radeksimko changed the title Reduce parallelism to 1/1 in all cases module loader: Reduce parallelism to 1/1 in all cases Jan 4, 2022
@radeksimko radeksimko requested a review from a team January 4, 2022 14:33
Copy link
Member

@dbanck dbanck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Were you able to identify the module loader as source of the high cpu usage from the profiling results?

@radeksimko
Copy link
Member Author

radeksimko commented Jan 5, 2022

Were you able to identify the module loader as source of the high cpu usage from the profiling results?

In broader sense yes, based on a provided CPU profile from a user I believe the main root cause is the command(s) we execute for each (initialized) module. I reckon the ultimate solution will be sourcing the schemas from a (future) Registry API, which should consume far less CPU than running terraform providers schema -json. That is however more involved solution for long term. I assume that "slowing down" the execution should reduce the aggregate CPU usage at any given time.

Screenshot 2022-01-05 at 10 32 23

@radeksimko radeksimko merged commit 07a4962 into main Jan 5, 2022
@radeksimko radeksimko deleted the f-reduce-parallelism branch January 5, 2022 11:01
@github-actions
Copy link

github-actions bot commented Feb 5, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants