Skip to content

Conversation

@nghialv
Copy link
Member

@nghialv nghialv commented Jun 28, 2021

What this PR does / why we need it:

This PR continues implementing the PlanPreview feature described in this RFC.

In this PR, I have added a new handler component that will run when piped started up.
This handler periodically fetches all un-handled BuildPlanPreview commands and split them into a pool of workers.
Each worker handles command serially by making a builder instance to build appropriate PlanPreview results for the command.

Which are the main parts that should be reviewed:

  • the handler struct and its behaviour defined inside planpreview package

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Copy link
Collaborator

@pipecd-bot pipecd-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GO_LINTER

Some issues were detected while linting go source files in your changes.

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 32.60%. This pull request increases coverage by 0.15%.

File Function Base Head Diff
pkg/app/piped/apistore/commandstore/store.go store.ListBuildPlanPreviewCommands -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go newBuilder -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go builder.Build -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go builder.listApplications -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithWorkerNum -- 100.00% +100.00%
pkg/app/piped/planpreview/handler.go WithCommandQueueBufferSize -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithCommandCheckInterval -- 100.00% +100.00%
pkg/app/piped/planpreview/handler.go WithCommandHandleTimeout -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithLogger -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go NewHandler -- 85.71% +85.71%
pkg/app/piped/planpreview/handler.go Handler.Run -- 94.44% +94.44%
pkg/app/piped/planpreview/handler.go Handler.enqueueNewCommands -- 93.75% +93.75%
pkg/app/piped/planpreview/handler.go Handler.handleCommand -- 43.48% +43.48%
pkg/app/piped/apistore/commandstore/store.go store.sync 0.00% 0.00% +0.00%

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 32.60%. This pull request increases coverage by 0.15%.

File Function Base Head Diff
pkg/app/piped/apistore/commandstore/store.go store.ListBuildPlanPreviewCommands -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go newBuilder -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go builder.Build -- 0.00% +0.00%
pkg/app/piped/planpreview/builder.go builder.listApplications -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithWorkerNum -- 100.00% +100.00%
pkg/app/piped/planpreview/handler.go WithCommandQueueBufferSize -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithCommandCheckInterval -- 100.00% +100.00%
pkg/app/piped/planpreview/handler.go WithCommandHandleTimeout -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go WithLogger -- 0.00% +0.00%
pkg/app/piped/planpreview/handler.go NewHandler -- 85.71% +85.71%
pkg/app/piped/planpreview/handler.go Handler.Run -- 94.44% +94.44%
pkg/app/piped/planpreview/handler.go Handler.enqueueNewCommands -- 93.75% +93.75%
pkg/app/piped/planpreview/handler.go Handler.handleCommand -- 43.48% +43.48%
pkg/app/piped/apistore/commandstore/store.go store.sync 0.00% 0.00% +0.00%

@nakabonne
Copy link
Member

I know it's a bit tedious and likely to have no sense, but it would be helpful to review if you could write down a brief summary or where you want us to look over if there is. No need to do it every time, but it's gonna make for a good discussion if you could give it a try as much as possible 😃

if err != nil {
return nil, fmt.Errorf("failed to clone git repository %s", cmd.RepositoryId)
}
defer repo.Clean()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know there is such a helper.

@nakabonne
Copy link
Member

Cool. there you go!
/lgtm

@nghialv
Copy link
Member Author

nghialv commented Jun 28, 2021

I know it's a bit tedious and likely to have no sense, but it would be helpful to review if you could write down a brief summary or where you want us to look over if there is. No need to do it every time, but it's gonna make for a good discussion if you could give it a try as much as possible 😃

Sure. I have just updated the PR body to explain more about this PR.

@nakabonne
Copy link
Member

Thank you!

Comment on lines 96 to 99
workerNum: 3,
commandQueueBufferSize: 10,
commandCheckInterval: 5 * time.Second,
commandHandleTimeout: 5 * time.Minute,
Copy link
Member

@nakabonne nakabonne Jun 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: to be more clear they should be defined as const like default~, some other places don't be so though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, make them as const variables of this planpreview package is LGTM 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Applied this. 👍

@pipecd-bot pipecd-bot removed the lgtm label Jun 28, 2021
@pipecd-bot
Copy link
Collaborator

TODO

The following ISSUES will be created once got merged. If you want me to skip creating the issue, you can use /todo skip command.

Details

1. Implement planpreview builder.

https://github.com/pipe-cd/pipe/blob/b7ea8c324e897d036739fc6827e0fc6d067631b8/pkg/app/piped/planpreview/builder.go#L67-L70

This was created by todo plugin since "TODO:" was found in b7ea8c3 when #2143 was merged. cc: @nghialv.

2. Add a new field to show why command was failed.

https://github.com/pipe-cd/pipe/blob/b7ea8c324e897d036739fc6827e0fc6d067631b8/pkg/model/command.proto#L81-L84

This was created by todo plugin since "TODO:" was found in b7ea8c3 when #2143 was merged. cc: @nghialv.

@khanhtc1202
Copy link
Member

Nice 🚀
/approve

@pipecd-bot
Copy link
Collaborator

APPROVE

This pull request is APPROVED by khanhtc1202.

Approvers can cancel the approval by writing /approve cancel in a comment. Any additional commits also will change this pull request to be not-approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants