[Fleet][EPM] Don't roll back on saved objects conflict errors.

[skh]

Summary

This refines the implementation of #84190 and implements #84651 . See also #84656 for a bit of discussion.

This changes the behavior of _installPackage() so that

• when a concurrent installation is detected, a ConcurrentInstallOperationError is thrown (instead of returning a list of installed assets which may or may not be complete)
• when a version conflict on a saved object write operation is thrown in any of the install*() methods called by _installPackage(), this is also wrapped in a ConcurrentInstallOperationError
• all other errors are thrown as before
• higher up in the call chain, ConcurrentInstallOperationError will not trigger a rollback. This fixes the bug that occurs when a second installation/upgrade operation aborts because of a saved object version conflict, and therefore rolls back the installation that a first installation operation just completed successfully, potentially resulting in follow-up errors and a broken installation
• ConcurrentInstallOperationError will cause the handler to return a 409 HTTP response with a message stating on which package the concurrent installation was detected, and that the operation was aborted.

This is still a rather optimistic way of handling this situation: when a concurrent installation is detected, the running installation is aborted and no attempts are made to clean up after it. This is possible because the install*() methods (installing kibana assets, pipelines, templates etc.) are idempotent. Indeed it is still perfectly possible that two parallel installations run successfully, installing everything twice, or that they only run into the saved object conflict at the very end, after almost everything was installed twice.

This may have effects on other users of the install package code flow, namely endpoint security. (cc @jonathan-buttner )

How to test this

• Try to get the installed package into a broken state. To do that, try to trigger a race condition by installing the same package several times at once, and observe if the race condition is handled correctly. https://gist.github.com/skh/cc695952031c9e349874b898c7066e42 may be helpful for this -- I had to set WAIT_TIME_REINSTALL in that script to 0 and run it a few times.

• Try to break it in any other way.

[elasticmachine]

Pinging @elastic/ingest-management (Feature:EPM)

[skh]

@elasticmachine merge upstream

[kibanamachine]

⏳ Build in-progress, with failures

• continuous-integration/kibana-ci/pull-request
• Commit: 69d125c
• This comment will update when the build is complete

Failed CI Steps

• serverIntegration
• Execute kibana-ciGroup9
• Execute kibana-ciGroup4
• Execute kibana-ciGroup3
• Execute kibana-ciGroup7
• Execute oss-accessibility
• Execute oss-accessibility
• Execute oss-firefox
• Execute kibana-ciGroup1
• Execute kibana-ciGroup11
• Execute kibana-ciGroup8
• Execute kibana-ciGroup5
• Execute kibana-ciGroup2
• Verify NOTICE
• Execute kibana-ciGroup10

History

• 💚 Build #94004 succeeded 69d125c
• 💚 Build #92330 succeeded 846781cf735fdf641803a536e8da96dbdbc41224
• 💚 Build #92321 succeeded a66d2c3527df6bffe921bdfd1bae1d65a8560c32

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream