[ResponseOps][Reporting] reapply index mappings of reporting index at startup

[pmuellr]

resolves: #231200

This PR adds code run at startup to:

• read the .kibana-reporting index template to get the mappings
• apply those mappings directly to the write index of the .kibana-reporting data stream

This is needed for those times when the mappings change, for instance in this PR: elastic/elasticsearch#129061. The mappings do get updated, but are not applied to the data stream, so in fact the "new" fields (when updates happen) are not actually mapped in the existing data stream.

I was hoping to actually roll over the index, but it's not clear how we could determine if we need to, and we certainly don't want to do that on every restart. Re-applying the mappings will ensure that any changes to the index template's mapping are made to the current index.

Release note

Startup processing for reporting has changed to apply the current index template mappings to the kibana reporting index, if it exists. This fixes a problem where a new field added to the index template, however it was not applied, so the field could not reliably used. The field was space_id, which could cause reports in non-default spaces to not appear in the reports list.

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[elasticmachine]

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

• Click to trigger kibana-pull-request for this PR!
• Click to trigger kibana-deploy-project-from-pr for this PR!
• Click to trigger kibana-deploy-cloud-from-pr for this PR!

[pmuellr]

Converted this to a draft. I think the problem with this approach is that if the bad mapping has already been applied, this will generate a warning message at startup, that's going to be noisy.

To fix that, we'd need to do a query on the mappings of the data stream (or a backing index) to see if it's "bad" or not. But at that point, we could also just roll-over ... which is what we wanted to do anyway.

So, rethinking the current code - will prolly open another PR if we go another route. But thinking we check the mapping for the "bad" field, if it's there, roll the datastream.

Clearly, in the future, we're going to have to be smarter about this. Perhaps we can store the "last template version applied" in the DS meta data, when we actually apply | roll it. At startup, we can read that and the template's version, to compare - if different, roll/apply.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 4d79b15

Failed CI Steps

• FTR Configs #45
• FTR Configs #62

Test Failures

• [job] [logs] FTR Configs #62 / Fleet packages test Automatic agent upgrades should take agents marked and ready for retry into account and upgrade them

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/reporting-server	90	91	+1

Unknown metric groups

API count

id	before	after	diff
@kbn/reporting-server	90	91	+1

History

• 💔 Build #328126 failed b409994

[pmuellr]

I'm going to close this, as I think it's not appropriate. Most specifically, if a report has been generated since the upgrade that added a new field, applying the mappings will actually fail, and there's not really any way to fix it.

I've been thinking a better approach is this - start storing the version field from the index template, as a _meta field in the data stream itself; for instance version_index_mappings. We can check that at startup - if the field is in the data stream and matches the index template, there's nothing to do. If the field is not in the data stream or doesn't match, roll the data stream, and create/update the version_index_mappings field in the data stream _meta to the index template version.