LG-6668 safeguard failed message

[peggles2]

This pull request adds a safe guard so that if the alert had failed and passed, that the failed alert is logged and not written over. The chances currently of this happening is zero because the 'Passed alerts are processed first' but as a safeguard, I added a check to ensure failed is always logged and updated the test.

Currently a work in progress and not review for review just yet.

[kbighorse]

It looks like perhaps the identical #log_alerts methods can be DRYed up somehow? Or perhaps that can wait for a dedicated issue if there are other such duplicated methods.

[zachmargolis]

It looks like perhaps the identical #log_alerts methods can be DRYed up somehow? Or perhaps that can wait for a dedicated issue if there are other such duplicated methods.

+1 I think it would be nice, could have a Helper class with a static method or a mixin shared across both places

[peggles2]

Ok so now the code has been changed so that the values are comma delimited, this means passed and failed can be stored in the same alert as well.
For example
{ visible_pattern: { no_side: 'Passed, Failed, Skipped, Caution' }, document_classification: { no_side: 'Failed' } },
or
visible_pattern: { no_side: 'Passed, Failed' }

Nothing is written over. CW query will use a like to wildcard the searches for the result values.

[zachmargolis]

Ok so now the code has been changed so that the values are comma delimited, this means passed and failed can be stored in the same alert as well. For example { visible_pattern: { no_side: 'Passed, Failed, Skipped, Caution' }, document_classification: { no_side: 'Failed' } }, or visible_pattern: { no_side: 'Passed, Failed' }

Nothing is written over. CW query will use a like to wildcard the searches for the result values.

I think adding commas like that makes things hard to aggregate. Yes, we coud use a "LIKE" query in CW, but that still harms the | stats count(*) by ... approach

Can we do the monitor-only version of this for a week or so, and the re-evaluate based on how often multiple errors occur?

[peggles2]

Refactored duplicate methods into one service class method. Changed the code so that if there is a duplicate for now to just output to the log file so we can see how frequently this occurs first.

[jmhooper]

LGTM! Let's throw a changelog commit on here. Looks like CI is upset about that