[ti_anomali] Modify field type to eliminate field conflicts within the data streams.

[brijesh-elastic]

Proposed commit message

ti_anomali: resolve field data type conflicts between data streams

This introduces a breaking change by altering the field type from 'long' to
'keyword' for the 'import_session_id', 'trusted_circle_ids', and 'update_id' fields,
and from 'short' to 'long' for the 'confidence' field.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

• Clone integrations repo.
• Install elastic package locally.
• Start elastic stack using elastic-package.
• Move to integrations/packages/ti_anomali directory.
• Run the following command to run tests.

elastic-package test

Related issues

• Closes [ti_anomali] Field data type conflicts between data streams #13585

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[kcreddy]

During integration upgrade, the source data stream-backed indices are rolled over, which doesn't cause conflicts during indexing.
But the destination indices aren't data stream backed, hence they wouldn't roll over. This will cause indexing failures because of mapping conflicts. You will also need to update transform's destination index suffix, and also fleet_transform_version to actually update the transform.

Please also test the upgrade process.

[navnit-elastic]

We tried upgrading the integration without changing destination index and fleet_transform_version and noticed that the destination index mapping has changed and newer documents ingested without any error. However, the newer documents still indexed with the previous field type, which is unusual. After changing the destination index name and fleet_transform_version it ingested documents with modified field types.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: c58347a

History

• 💚 Build #26717 succeeded a2f86a2
• 💚 Build #25986 succeeded e6b1f72
• 💚 Build #25954 succeeded 2dae054

cc @brijesh-elastic

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
84.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elastic-vault-github-plugin-prod]

Package ti_anomali - 2.0.0 containing this change is available at https://epr.elastic.co/package/ti_anomali/2.0.0/