[UII] Update package spec min version to 3.0 for serverless projects

[jen-huang]

Summary

Resolves #182827

As the title says :)

This also corrects spec.max to 3.1, which it should have been all along.

Integrations available for Observability projects after change:
Click here

Integrations available for Security projects after change:
Click here

[ghost]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[jen-huang]

/ci

[jen-huang]

The Observability serverless tests are failing because it is expecting apache package to be available & installable. However, apache does not appear because spec.max and spec.min are both set to 3.0, so only packages that are exactly on that format_version are shown on Serverless. There is no version of apache that is exactly on 3.0, it was migrated to 3.1.4 right away: https://github.com/elastic/integrations/pull/9818/files#diff-c1bfeefb3a7a71c324e3d6ee4ea6ed2b3cb6fd640e040b146c03582c3216cf37

It seems that there are four packages which are now on package spec > 3.0:
https://epr.elastic.co/search?spec.min=3.1
(apache, azure_metrics, kubernetes, lumos)

For serverless, I propose to either:

• Remove xpack.fleet.internal.registry.spec.max restriction
• Bump xpack.fleet.internal.registry.spec.max to 3.99 (probably safer)

@jsoriano @kpollich WDYT?

[kpollich]

Remove xpack.fleet.internal.registry.spec.max restriction

I'm sure we have a good reason for keeping this around, but I am a little surprised that we don't just have a min in place. My immediate instinct is to remove the max constraint, but I might be missing some important context.

Bump xpack.fleet.internal.registry.spec.max to 3.99 (probably safer)

I agree that this is safer, and we aren't likely to reach such a high minor format_version value for the package spec before introducing a 4.0 version. This seems like the best short-term path forward to me.

[jsoriano]

spec.max should be set to 3.1 at this point. We use this versioning to ensure that a given deployment of kibana only uses versions of the spec it implements.

[jsoriano]

I am a little surprised that we don't just have a min in place.

Regarding this, we could not set a min version because there were many packages using old versions of the spec.

[jen-huang]

I am a little surprised that we don't just have a min in place.

Regarding this, we could not set a min version because there were many packages using old versions of the spec.

I think Kyle mean just having the min and no max. Regardless, I pushed up the change to set spec.max to 3.1. Thanks for the clarifications!

[jen-huang]

/ci

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jen-huang]

I would like some folks from Observability and Security to approve this change before merging. For the folks below, you can see the Elastic Agent integrations available (including beta ones) after this change in the screenshots in this PR description, or you can view them in the serverless deployments for this PR that was created by CI (grab the URLs and creds from Buildkite or DM me).

@vinaychandrasekhar Can you review the Observability integrations?

@jamiehynds @cpascale43 Can you review the Security integrations?

[jen-huang]

After review with @cpascale43, we found that these integrations are missing after this change because they are not updated to v3:

• elasticsearch @elastic/stack-monitoring
• nginx_ingress_controller @elastic/obs-cloud-monitoring
• journald @elastic/elastic-agent-data-plane
• elastic-agent @elastic/elastic-agent
• windows @elastic/elastic-agent-data-plane

[jsoriano]

I think we should update the max version, independently with what we do with the min version by now. I have opened a separate PR for this #192493.

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: bc4f65f
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-184792-bc4f65f3bb7c
• Security Deployment

Failed CI Steps

Metrics [docs]

✅ unchanged

History

• 💛 Build #215613 was flaky 133230c
• 💔 Build #215542 failed 3f551c4
• 💛 Build #215307 was flaky c5b80b1
• 💛 Build #214746 was flaky d839b67

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jen-huang