Skip to content

Improve insight collector #1340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pipecd-bot merged 19 commits into from
Jan 6, 2021
Merged

Improve insight collector #1340

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
902e5f1
Improve insight collector
sanposhiho Dec 27, 2020
2d01d13
Add error hadling and fix some small issues from linter
sanposhiho Dec 27, 2020
16ac4bf
Add option to specify retryTime and retryInterval
sanposhiho Dec 27, 2020
709e19a
Remove unuse function
sanposhiho Dec 27, 2020
8abc72a
Fix handling on chunks already updated
sanposhiho Dec 27, 2020
7404079
Fix process project chunk
sanposhiho Dec 27, 2020
261b654
Fix import order
sanposhiho Dec 27, 2020
0e99357
Fix error handling
sanposhiho Dec 27, 2020
26322b5
Add descriptions for functions and rename some functions
sanposhiho Dec 27, 2020
0ba2717
Fix descriptions on functions
sanposhiho Dec 27, 2020
8bca2b1
Rename some variables and functions
sanposhiho Jan 4, 2021
06c1d3f
PutMilestone only updateErr is not nil
sanposhiho Jan 4, 2021
83958c0
Rename functions to export
sanposhiho Jan 4, 2021
b35358d
Use backoff package to retry
sanposhiho Jan 4, 2021
2b494d0
Set default value on config for insight collector
sanposhiho Jan 6, 2021
f8a5fa8
Fix logging messages
sanposhiho Jan 6, 2021
0c76835
Add successProcess flag
sanposhiho Jan 6, 2021
3fedbfd
Rename some functions and variables
sanposhiho Jan 6, 2021
0c89dc2
Merge branch 'master' into improve-insight-collector
sanposhiho Jan 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/pipecd/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ go_library(
"//pkg/app/api/stagelogstore:go_default_library",
"//pkg/app/ops/handler:go_default_library",
"//pkg/app/ops/insightcollector:go_default_library",
"//pkg/backoff:go_default_library",
"//pkg/cache/rediscache:go_default_library",
"//pkg/cli:go_default_library",
"//pkg/config:go_default_library",
Expand Down
35 changes: 23 additions & 12 deletions cmd/pipecd/ops.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import (
"github.com/pipe-cd/pipe/pkg/admin"
"github.com/pipe-cd/pipe/pkg/app/ops/handler"
"github.com/pipe-cd/pipe/pkg/app/ops/insightcollector"
"github.com/pipe-cd/pipe/pkg/backoff"
"github.com/pipe-cd/pipe/pkg/cli"
"github.com/pipe-cd/pipe/pkg/datastore"
"github.com/pipe-cd/pipe/pkg/version"
Expand Down Expand Up @@ -102,18 +103,28 @@ func (s *ops) run(ctx context.Context, t cli.Telemetry) error {
collector := insightcollector.NewInsightCollector(ds, fs, t.Logger)
c := cron.New(cron.WithLocation(time.UTC))
_, err := c.AddFunc(cfg.InsightCollector.Schedule, func() {
start := time.Now()
if err := collector.CollectProjectsInsight(ctx); err != nil {
t.Logger.Error("failed to run the project insight collector", zap.Error(err))
} else {
t.Logger.Info("project insight collector successfully finished", zap.Duration("duration", time.Since(start)))
}

start = time.Now()
if err := collector.CollectApplicationInsight(ctx); err != nil {
t.Logger.Error("failed to run the application insight collector", zap.Error(err))
} else {
t.Logger.Info("application insight collector successfully finished", zap.Duration("duration", time.Since(start)))
retry := backoff.NewRetry(cfg.InsightCollector.RetryTime, backoff.NewConstant(time.Duration(cfg.InsightCollector.RetryIntervalHour)*time.Hour))
for retry.WaitNext(ctx) {
var failed bool
start := time.Now()
if err = collector.ProcessNewlyCompletedDeployments(ctx); err != nil {

@nghialv nghialv Jan 5, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you skip calling this function when it was completed successfully while ProcessNewlyCreatedDeployments was failed? I know that the insight data will not be affected but we can save some unnecessary DB calls by skip calling this one.

t.Logger.Error("failed to process the insight collector with completedAt", zap.Error(err))

@nghialv nghialv Jan 5, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: failed to process the newly completed deployments while accumulating insight data

failed = true
} else {
t.Logger.Info("processing the insight collector with completedAt successfully finished", zap.Duration("duration", time.Since(start)))

@nghialv nghialv Jan 5, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: successfully processed the newly completed deployments while accumulating insight data

}

start = time.Now()
if err = collector.ProcessNewlyCreatedDeployments(ctx); err != nil {
t.Logger.Error("failed to process the insight collector with createdAt", zap.Error(err))

@nghialv nghialv Jan 5, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: failed to process the newly created deployments while accumulating insight data

failed = true
} else {
t.Logger.Info("processing the insight collector with createdAt successfully finished", zap.Duration("duration", time.Since(start)))

@nghialv nghialv Jan 5, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: successfully processed the newly created deployments while accumulating insight data

}

if !failed {
return
}
}
})
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ require (
github.com/prometheus/client_golang v1.6.0
github.com/prometheus/client_model v0.2.0
github.com/prometheus/common v0.9.1
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/robfig/cron/v3 v3.0.1
github.com/spf13/cobra v0.0.5
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.5.1
Expand Down
1 change: 0 additions & 1 deletion go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,6 @@ github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsT
github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/prometheus/procfs v0.0.11 h1:DhHlBtkHWPYi8O2y31JkK0TF+DGM+51OopZjH/Ia5qI=
github.com/prometheus/procfs v0.0.11/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/robfig/cron v1.2.0 h1:ZjScXvvxeQ63Dbyxy76Fj3AT3Ut0aKsyd2/tl3DTMuQ=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
Expand Down
Loading