[Security Solution][Alerts] fixes merge fields with source in Detection Engine on Alerts creation

[vitaliidm]

Summary

• fixes [Security Solution][Alerts] Merge _source and fields correctly in alert creation logic #147389
• mergeMissingFieldsWithSource and mergeAllFieldsWithSource method will not be merging anymore multi field values into source.

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[marshallmain]

There's another edge case that we may want to fix in the future, but it should be rare so we don't have to worry about fixing it for 8.7 at this point.

test.only('does the right stuff', () => {
        const _source: SignalSourceHit['_source'] = {
          '@timestamp': '2023-02-10T10:15:50Z',
          'process.command_line': { text: 'a' },
        };

        const fields: SignalSourceHit['fields'] = {
          'process.command_line.text': ['string longer than 10 characters'],
          '@timestamp': ['2023-02-10T10:15:50.000Z'],
        };
        const doc: SignalSourceHit = { ...emptyEsResult(), _source, fields };
        const merged = mergeMissingFieldsWithSource({ doc, ignoreFields: [] })._source;
        expect(merged).toEqual<ReturnTypeMergeFieldsWithSource>(_source);
      });

If, for some reason, source documents mix the dot and nested notations, then mergeMissingFields logic can't find the original field in the _source and still merges the value from fields into it.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: b7f37de

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
securitySolution	428	430	+2

Total ESLint disabled count

id	before	after	diff
securitySolution	506	508	+2

History

• 💛 Build #110605 was flaky 254ccf2
• 💔 Build #110574 failed a970544
• 💚 Build #107716 succeeded e5d9bd2
• 💔 Build #107650 failed 371c4c8
• 💚 Build #107570 succeeded 36f55e0

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @vitaliidm

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.7	Backport failed because of merge conflicts You might need to backport the following PRs to 8.7: - [Security Solution] Re-enable a skipped url state e2e test (#152075)

Manual backport

To create the backport manually run:

node scripts/backport --pr 151004

Questions ?

Please refer to the Backport tool documentation

[vitaliidm]

There's another edge case that we may want to fix in the future, but it should be rare so we don't have to worry about fixing it for 8.7 at this point.

Created an issue for this one #152446

[vitaliidm]

💚 All backports created successfully

Status	Branch	Result
✅	8.7

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation