user-defined custom field formats for index patterns can get overwritten by Malcolm

[mmguero]

found by @ee-hex-ee

If a user goes into the index pattern and sets custom formatting:

I think what is happening is that if templates have been imported (based on the existing templates' hashes not matching the "standard" template hashes) then the index pattern gets ovewritten, which loses those settings.

I want to create/update the index pattern, but we should see if we can somehow pull out and save existing field format settings beforehand, then reapply them after the creation if they did.

[mmguero]

The field format map can be retrieved like this, externally (same curl command inside the container only without the docker compose stuff):

$ docker compose exec api curl -sSL 'dashboards:5601/dashboards/api/saved_objects/index-pattern/arkime_sessions3-*' | jq '.attributes.fieldFormatMap'
"{\"zeek.dns.query\":{\"id\":\"color\",\"params\":{\"parsedUrl\":{\"origin\":\"https://malcolm.seven.local.lan\",\"pathname\":\"/dashboards/app/dashboards\",\"basePath\":\"/dashboards\"},\"fieldType\":\"string\",\"colors\":[{\"range\":\"-Infinity:Infinity\",\"regex\":\"*chromecast*\",\"text\":\"#ff0000\",\"background\":\"#ffffff\"}]}}}"

[mmguero]

and stripping some of the parsedUrl stuff we wouldn't want to save:

$ docker compose exec api curl -sSL 'dashboards:5601/dashboards/api/saved_objects/index-pattern/arkime_sessions3-*' | jq -r '.attributes.fieldFormatMap' | jq -c 'with_entries(.value.params.parsedUrl? = null | del(.value.params.parsedUrl))'  | jq '@json'

results in:

"{\"zeek.dns.query\":{\"id\":\"color\",\"params\":{\"fieldType\":\"string\",\"colors\":[{\"range\":\"-Infinity:Infinity\",\"regex\":\"*chromecast*\",\"text\":\"#ff0000\",\"background\":\"#ffffff\"}]}},\"related.role\":{\"id\":\"color\",\"params\":{\"fieldType\":\"string\",\"colors\":[{\"range\":\"-Infinity:Infinity\",\"regex\":\"*HMI*\",\"text\":\"#54B399\",\"background\":\"#ffffff\"}]}}}"