[WIP] Add ES Auto Index Cleaner #6425
Conversation
Signed-off-by: Alok Kumar Singh <[email protected]>
Signed-off-by: Alok Kumar Singh <[email protected]>
Signed-off-by: Alok Kumar Singh <[email protected]>
akstron
force-pushed
the
es-index-cleaner
branch
from
8feef9c to
6370ca0
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6425 +/- ##
===========================================
- Coverage 96.20% 34.64% -61.57%
===========================================
Files 356 192 -164
Lines 20416 11643 -8773
===========================================
- Hits 19642 4034 -15608
- Misses 585 7301 +6716
- Partials 189 308 +119
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
| i.logger.Info("Indices before this date will be deleted", zap.String("date", deleteIndicesBefore.Format(time.RFC3339))) | ||
|
|
||
| indices = filter.ByDate(indices, deleteIndicesBefore) | ||
|
|
There was a problem hiding this comment.
Will add check to not delete indices with write alias.
| @@ -41,6 +44,32 @@ type IndicesClient struct { | |||
| MasterTimeoutSeconds int | |||
| } | |||
|
|
|||
| // Create the indices only client using config.Configuration | |||
| func CreateIndicesClient(c *config.Configuration, logger *zap.Logger) (*IndicesClient, error) { | |||
There was a problem hiding this comment.
I don't follow why we need this function. The pattern in es.Factory is to use config.NewClient, which takes configuration.
There was a problem hiding this comment.
This is because config.NewClient creates a client using the elastic library, while the current implementation of cleaner uses IndicesClient which is a wrapper around the http client. I wanted to use the already implemented IndicesClient for the functionality.
There was a problem hiding this comment.
I don't understand that explanation. You are not changing the functionality of IndicesClient struct, you are just using the functions that are already implemented in it. In order to construct this struct you need to do call config.NewClient and then set that as Client property of the struct.
There was a problem hiding this comment.
But struct Client property of IndicesClient is not same as es.Client which we get from config.NewClient So, we can't actually set it to Client of IndicesClient.
| }() | ||
| } | ||
|
|
||
| func (i *IndexCleaner) Clean() error { |
| year, month, day := time.Now().UTC().Date() | ||
| tomorrowMidnight := time.Date(year, month, day, 0, 0, 0, 0, time.UTC).AddDate(0, 0, 1) | ||
| deleteIndicesBefore := tomorrowMidnight.Add(-1 * i.timePeriod) | ||
| i.logger.Info("Indices before this date will be deleted", zap.String("date", deleteIndicesBefore.Format(time.RFC3339))) | ||
|
|
||
| indices = filter.ByDate(indices, deleteIndicesBefore) |
There was a problem hiding this comment.
doesn't this logic already exist in the existing es_cleaner? I expect we reuse whatever exist there. We can move the logic if necessary, but then es_cleaner binary should change to use the moved logic, so that our integration tests would validate the changes.
Which problem is this PR solving?
Description of the changes
How was this change tested?
Checklist
jaeger:make lint testjaeger-ui:npm run lintandnpm run test