Initial definition of content pack[age]s#777
Merged
jsoriano merged 11 commits intoelastic:mainfrom Aug 27, 2024
Merged
Conversation
jsoriano
force-pushed
the
content-pack-ages
branch
from
6114b36 to
6900576
Compare
jsoriano
force-pushed
the
content-pack-ages
branch
from
6900576 to
c6687d6
Compare
kpollich
reviewed
Jul 23, 2024
spec/content/manifest.spec.yml
Outdated
| type: object | ||
| additionalProperties: false | ||
| definitions: | ||
| interested_on: |
Member
There was a problem hiding this comment.
This seems like a tricky thing to name. What do you think about discovery_fields instead, in the interest of being more explicit about the behavior associated with this property?
Member
Author
There was a problem hiding this comment.
Yeah, I am not super happy with the name, so thanks for the suggestion!
I wouldn't include fields at this level because maybe we can support other things, like other packages, but discovery could fit.
discovery:
fields:
- name: "process.pid"
discovery:
packages:
- name: "system"
Member
There was a problem hiding this comment.
I like leaving room for driving recommendations for other packages vs fields - I think this approach is solid.
mrodm
reviewed
Jul 31, 2024
jsoriano
commented
Aug 26, 2024
| Processed error | ||
| Removed error | ||
|
|
||
| UnusedProcessors []Processor |
Member
Author
There was a problem hiding this comment.
Added detection of unused rules, used by now only on tests, we could warn in the future about this.
Member
Author
|
Opening for review so we can start testing it. |
mrodm
reviewed
Aug 26, 2024
Contributor
mrodm
left a comment
mrodm
left a comment
There was a problem hiding this comment.
LGTM!
Added some questions about the limits for this new type of package
spec/content/spec.yml
Outdated
Comment on lines
+10
to
+11
| totalSizeLimit: 50MB | ||
| sizeLimit: 30MB |
Contributor
There was a problem hiding this comment.
Are these limits related to the whole package ? As these are going to be mainly assets, probably we could set the same limits as the integration packages. WDYT ?
totalSizeLimit: 250MB
sizeLimit: 150MB
Member
Author
There was a problem hiding this comment.
Ok, in any case we will revisit this, as content packages are intended to support bigger sizes than other packages.
💚 Build Succeeded
History
cc @jsoriano |
This was referenced Aug 27, 2024
jen-huang
added a commit
to elastic/kibana
that referenced
this pull request
Oct 9, 2024
## Summary Related to #192484. This PR adding [new content package types and schemas](elastic/package-spec#777) so that content packages can be returned correctly from EPR to unblock development of those packages. The only current content package is `kubernetes_otel`. You will need to bump up the max allowed spec version and search with beta (prerelease) packages enabled to find it: ``` xpack.fleet.internal.registry.spec.max: '3.4' ``` Tests will come with the rest of work for #192484 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
jen-huang
added a commit
to jen-huang/kibana
that referenced
this pull request
Oct 9, 2024
## Summary Related to elastic#192484. This PR adding [new content package types and schemas](elastic/package-spec#777) so that content packages can be returned correctly from EPR to unblock development of those packages. The only current content package is `kubernetes_otel`. You will need to bump up the max allowed spec version and search with beta (prerelease) packages enabled to find it: ``` xpack.fleet.internal.registry.spec.max: '3.4' ``` Tests will come with the rest of work for elastic#192484 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit bd6533f) # Conflicts: # oas_docs/bundle.json # oas_docs/bundle.serverless.json # oas_docs/output/kibana.serverless.staging.yaml # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.staging.yaml # oas_docs/output/kibana.yaml # x-pack/plugins/fleet/server/types/rest_spec/epm.ts
jen-huang
added a commit
to elastic/kibana
that referenced
this pull request
Oct 10, 2024
…195690) # Backport This will backport the following commits from `main` to `8.x`: - [[UII] Add types to return content packages correctly (#195505)](#195505) Note that this includes manual changes to openapi spec files. This change is not present on the original PR as the main branch no longer uses openapi spec files. <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jen Huang","email":"its.jenetic@gmail.com"},"sourceCommit":{"committedDate":"2024-10-09T20:08:18Z","message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","backport:prev-minor"],"number":195505,"url":"https://github.com/elastic/kibana/pull/195505","mergeCommit":{"message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195505","number":195505,"mergeCommit":{"message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c"}}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add the initial definition for content packages.
On this initial version, they only support the distribution of kibana assets, like dashboards.
This package type introduces a discovery mechanism based on the data the package can work with. For example the following definition would indicate that the package provides resources that are useful when used with data that includes the field
process.pid:The package registry could add support to look for packages that have support for certain fields. For example a request like the following one could be used to discover packages that include resources that can leverage the
process.pidor theuser.idfields.Why is it important?
There are some use cases where developers want to distribute resources that can be useful with some data, without coupling them to the mechanism used to collect this data.
These resources could be for example dashboards that can be used with data collected with other integrations or even with other agents as the OTEL collector.
Paves the foundations for the use cases described in #351.
Checklist
test/packagesthat prove my change is effective.spec/changelog.yml.Related issues