[Security Solution] Rule Diff Phase 2 components

[dplumlee]

Summary

Addresses #166489
Docs issue: elastic/security-docs#4783

Adds per-field diffs for the rule upgrade flyout

Acceptance Criteria

• The tab with per-field diffs is hidden behind a new feature flag. When the flag is off, the tab does not appear in the flyout. The tab should work regardless of the value of jsonPrebuiltRulesDiffingEnabled.
• Per-field diffs are read-only components. We don't need to let the user "merge" differences using these components.
• Diffs for complex fields are rendered as JSON diffs using the same component used for rendering the JSON diff for the whole rule. This means this component should be abstracted away and should accept unknown values in props instead of RuleResponse.
• Diffs for related fields are grouped or rendered close to each other. For example:
  • Index patterns + Data view id
  • Custom query + Filters + Language + Saved query id
• The tab uses the response from the upgrade/_review API endpoint and doesn't need any other API calls to render itself.
• The tab renders itself under 150ms.

Manual testing

We are currently going over every field returned by the rule diff api to locate any issues (either in the UI or endpoint) that occur when certain fields are added/deleted/modified

To test certain fields:

Use either a PATCH or PUT method on the /api/detection_engine/rules endpoint route with a rule object that contains at least a rule_id (not id) and a lower version number than the current rule's version. This will allow you to see the prebuilt package update in the Rule Updates tab on the rules overview page

Example

{
  "rule_id": "some_rule_id_12345",
  "version": 1,
  "test_field": "test value",
  ...
}

Manually tested fields

Field name	Has been verified	Endpoint issue
version	✔️
name	✔️
description	✔️
author	✔️
building_block	✔️
severity	✔️
severity_mapping	✔️
risk_score	✔️
risk_score_mapping	✔️
references	✔️
false_positives	✔️
investigation_fields	✔️
license	✔️
rule_name_override	✔️
threat	✔️
threat_indicator_path	✔️
timestamp_override	✔️
tags	✔️
data_source	✔️
type	✔️
kql_query	✔️
eql_query	✔️
event_category_override	✔️
timestamp_field	✔️
tiebreaker_field	✔️
esql_query	✔️
anomaly_threshold	✔️
machine_learning_job_id	✔️
related_integrations	✔️
required_fields	✔️
timeline_template	✔️
threshold	✔️
threat_index	✔️
threat_mapping	✔️
threat_filters	✔️
threat_query	✔️
threat_indicator_path	✔️
concurrent_searches	✔️
items_per_search	✔️
alert_suppression	✔️
new_terms_fields	✔️
history_window_start	✔️
rule_schedule	✔️
setup	✔️
note	✔️
max_signals	✔️

Screenshots

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[jpdjere]

We're going to need a dictionary to translate field names as they exist in the rule schema to their UI-version. For example, note is actually Investigation guide. I guess we can create a dictionary only for the one that are completely different, and for the others, not found in the dict, just take the field name and camel case it as you are doing now.

[dplumlee]

Started in x-pack/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/diff_components/translations.ts

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[dplumlee]

buildkite test this

[jpdjere]

This one is an interesting one:

Change of Rule Type: Query ---> ESQL

[jpdjere]

Threat Indicator match changes:

Additional data_source change

Concurrent items_per_search and concurrent_searches

[jpdjere]

Threshold rules

[jpdjere]

Machine Learning rule

[jpdjere]

Tested all types, looking good! LGTM ✅

[banderror]

@dplumlee Reviewed changes in the code and left a few suggestions, most of them for further improvements in follow-up PRs.

But overall it looks great: components are clean and separated into logical pieces, the implementation is easy to comprehend. LGTM 👍

[banderror]

Not sure I get the current folder structure and where what components are supposed to be kept:

/rule_management/components/rule_details
  /diff_components/*
  /json_diff/*
  /per_field_diff/*
  /per_field_rule_diff_tab.tsx
  /rule_diff_tab.tsx

I'd like to propose a structure that would represent the actual components' hierarchy. Something like that:

/rule_management/components/rule_details
  /sections
    /rule_about_section.tsx
    /rule_definition_section.tsx
    /etc
  /flyout
    /rule_details_flyout.tsx
    /use_rule_details_flyout.tsx
    /diffs
      /common diff components and utils go here
    /tabs
      /json_diff_tab
      /per_field_diff_tab
      /overview_tab

If that makes sense, I'd suggest to do it in a separate PR.

[banderror]

This file contains a lot of code (461 lines) and most of it looks very repetitive. If you take a closer look, most of the functions take some object in parameters, and for each of its properties yield either a FieldDiff or nothing. I feel like we could replace most of the code in this file with some kind of generic implementation for calculating a FieldDiff[] for a given ThreeWayDiff input.

Again, I'd suggest trying to tackle this in a separate PR with refactoring and probably after covering the current implementation with tests.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: faaec7f

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	4960	4970	+10

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	11.4MB	11.4MB	+16.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	70.6KB	70.7KB	+39.0B

History

• 💛 Build #192605 was flaky fe1112fb07a0020579a1f6197613d6542272533e
• 💛 Build #192134 was flaky 4f7171e
• 💛 Build #191801 was flaky bc20d45

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dplumlee