Skip to content

[Streams 🌊] Introduce GroupStreams#208126

Merged
simianhacker merged 39 commits intoelastic:mainfrom
simianhacker:streams/grouped
Feb 4, 2025
Merged

[Streams 🌊] Introduce GroupStreams#208126
simianhacker merged 39 commits intoelastic:mainfrom
simianhacker:streams/grouped

Conversation

@simianhacker
Copy link
Member

@simianhacker simianhacker commented Jan 23, 2025 •
edited by kibanamachine
Loading

🍒 Summary

This PR adds support to /api/streams endpoints for the GroupStreamDefinition type. Group streams are simply a list of member streams along with dashboards. An example of the definition looks like:

{
  "name": "nginx-logs",
  "stream": {
    "grouped": {
      "description": "A collection of streams for Nginx",
      "members": [
        "logs",
        "logs.nginx"
      ]
    }
  },
  "dashboards": []
}

The following APIs support GroupStreamDefinition:

  • GET /api/streams
  • GET /api/streams/{id}
  • PUT /api/streams/{id}
  • DELETE /api/streams/{id}
  • GET /api/streams/{id}/_details

This PR only includes the support to the APIs, I will submit a follow PR for the UI.

@simianhacker simianhacker changed the title [Streams] Introduce GroupedStream [Streams 🌊] Introduce GroupedStream Jan 23, 2025
@simianhacker simianhacker marked this pull request as ready for review January 28, 2025 13:56
@simianhacker simianhacker requested review from a team as code owners January 28, 2025 13:56
@simianhacker simianhacker added release_note:feature Makes this part of the condensed release notes v9.0.0 Feature:Streams This is the label for the Streams Project backport:prev-minor labels Jan 28, 2025
rmyz
rmyz approved these changes Jan 28, 2025
View reviewed changes
Copy link
Contributor

@rmyz rmyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x-pack/solutions/observability/packages/utils_server/es/storage/types.ts changes LGTM!

flash1293
flash1293 reviewed Jan 29, 2025
View reviewed changes
Copy link
Contributor

@flash1293 flash1293 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks mostly good to me, biggest question to me is the semantics of what should and shouldn't be allowed:

  • Can you reference a non-existing stream in a group (but one you would have access to)?
  • Can you reference a stream you don't have access to in a group?
  • Can you delete group streams which contain streams you don't have access to?
  • Can you see group streams which contain streams you don't have access to?

Right now, the answers are:

  • No
  • No
  • Yes
  • Yes

For dashboard links (we are kind of similar), the answers are:

  • Yes
  • Yes
  • Yes (if you have access to the stream you are deleting)
  • Yes (if you have access to the stream you are deleting)

Not sure about the correct behavior. As this is a bit of a product question, wdyt @LucaWintergerst ?

@flash1293
Copy link
Contributor

flash1293 commented Jan 29, 2025

Another thing - to stay consistent with the ingest streams, could we introduce another route PUT /api/streams/<name>/_grouped to only update the group stream settings? We have the same for ingest and it would make it easier to wire it up in the UI.

miltonhultgren
miltonhultgren approved these changes Feb 4, 2025
View reviewed changes
Copy link
Contributor

@miltonhultgren miltonhultgren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@simianhacker simianhacker merged commit 2092c3d into elastic:main Feb 4, 2025
1 check passed
@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 4, 2025

Starting backport for target branches: 9.0

https://github.com/elastic/kibana/actions/runs/13133118391

@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 4, 2025

💔 All backports failed

Status Branch Result
❌ 9.0 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 208126

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Feb 5, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 5, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 208126 locally

@kibanamachine kibanamachine mentioned this pull request Feb 5, 2025
Merged
@tonyghiani tonyghiani added backport:version Backport to applied version labels v8.19.0 and removed backport missing Added to PRs automatically when the are determined to be missing a backport. backport:prev-minor labels Feb 5, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 5, 2025

Starting backport for target branches: 8.x, 9.0

https://github.com/elastic/kibana/actions/runs/13156800497

@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 5, 2025

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [lens][datatable] Remove eui datagrid styles (#204057)
- [Security Solution] [AI Assistant] security assistant content references tour (#208775)
- [Upgrade Assistant] Critical Kibana API deprecations should not block upgrades (#209128)
- [[ES
❌ 9.0 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 208126

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Feb 5, 2025
Merged
@simianhacker simianhacker mentioned this pull request Feb 5, 2025
Merged
simianhacker added a commit to simianhacker/kibana that referenced this pull request Feb 5, 2025
@simianhacker
[Streams 🌊] Introduce GroupStreams (elastic#208126)
5af1f35 
## 🍒  Summary

This PR adds support to `/api/streams` endpoints for the
`GroupStreamDefinition` type. Group streams are simply a list of member
streams along with dashboards. An example of the definition looks like:

```JSON
{
  "name": "nginx-logs",
  "stream": {
    "grouped": {
      "description": "A collection of streams for Nginx",
      "members": [
        "logs",
        "logs.nginx"
      ]
    }
  },
  "dashboards": []
}
```
The following APIs support `GroupStreamDefinition`:

- `GET /api/streams`
- `GET /api/streams/{id}`
- `PUT /api/streams/{id}`
- `DELETE /api/streams/{id}`
- `GET /api/streams/{id}/_details`

This PR only includes the support to the APIs, I will submit a follow PR
for the UI.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
(cherry picked from commit 2092c3d)

# Conflicts:
#	x-pack/solutions/observability/plugins/streams_app/public/components/stream_detail_overview/index.tsx
#	x-pack/solutions/observability/plugins/streams_app/public/components/stream_detail_view/index.tsx
#	x-pack/test/api_integration/deployment_agnostic/apis/observability/streams/lifecycle.ts
@simianhacker
Copy link
Member Author

simianhacker commented Feb 5, 2025

💔 Some backports could not be created

Status Branch Result
✅ 9.0
❌ 8.x An unhandled error occurred. Please see the logs for details

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 208126

Questions ?

Please refer to the Backport tool documentation

simianhacker added a commit that referenced this pull request Feb 5, 2025
@simianhacker
[9.0] [Streams 🌊] Introduce GroupStreams (#208126) (#209776)
56e51b8 
# Backport

This will backport the following commits from `main` to `9.0`:
- [[Streams 🌊] Introduce GroupStreams
(#208126)](#208126)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Chris
Cowan","email":"chris@elastic.co"},"sourceCommit":{"committedDate":"2025-02-04T10:12:35Z","message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","release_note:feature","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[Streams
🌊] Introduce
GroupStreams","number":208126,"url":"https://github.com/elastic/kibana/pull/208126","mergeCommit":{"message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208126","number":208126,"mergeCommit":{"message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
@simianhacker simianhacker mentioned this pull request Feb 5, 2025
Merged
@simianhacker
Copy link
Member Author

simianhacker commented Feb 5, 2025

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

simianhacker added a commit to simianhacker/kibana that referenced this pull request Feb 5, 2025
@simianhacker
[Streams 🌊] Introduce GroupStreams (elastic#208126)
e2e7de0 
## 🍒  Summary

This PR adds support to `/api/streams` endpoints for the
`GroupStreamDefinition` type. Group streams are simply a list of member
streams along with dashboards. An example of the definition looks like:

```JSON
{
  "name": "nginx-logs",
  "stream": {
    "grouped": {
      "description": "A collection of streams for Nginx",
      "members": [
        "logs",
        "logs.nginx"
      ]
    }
  },
  "dashboards": []
}
```
The following APIs support `GroupStreamDefinition`:

- `GET /api/streams`
- `GET /api/streams/{id}`
- `PUT /api/streams/{id}`
- `DELETE /api/streams/{id}`
- `GET /api/streams/{id}/_details`

This PR only includes the support to the APIs, I will submit a follow PR
for the UI.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
(cherry picked from commit 2092c3d)

# Conflicts:
#	api_docs/kbn_streams_schema.devdocs.json
simianhacker added a commit that referenced this pull request Feb 6, 2025
@simianhacker @miltonhultgren @kibanamachine
[8.x] [Streams 🌊] Introduce GroupStreams (#208126) (#209871)
dd80846 
# Backport

This will backport the following commits from `main` to `8.x`:
- [[Streams 🌊] Introduce GroupStreams
(#208126)](#208126)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Chris
Cowan","email":"chris@elastic.co"},"sourceCommit":{"committedDate":"2025-02-04T10:12:35Z","message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","release_note:feature","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[Streams
🌊] Introduce
GroupStreams","number":208126,"url":"https://github.com/elastic/kibana/pull/208126","mergeCommit":{"message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/209776","number":209776,"state":"MERGED","mergeCommit":{"sha":"56e51b892b6fa6b21954afd61dd828cf80e1c262","message":"[9.0]
[Streams 🌊] Introduce GroupStreams (#208126) (#209776)\n\n#
Backport\n\nThis will backport the following commits from `main` to
`9.0`:\n- [[Streams 🌊] Introduce
GroupStreams\n(#208126)](https://github.com/elastic/kibana/pull/208126)\n\n<!---
Backport version: 9.6.4 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Chris\nCowan\",\"email\":\"chris@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-04T10:12:35Z\",\"message\":\"[Streams\n🌊]
Introduce GroupStreams (#208126)\\n\\n## 🍒 Summary\\r\\n \\r\\nThis PR
adds\nsupport to `/api/streams` endpoints for
the\\r\\n`GroupStreamDefinition`\ntype. Group streams are simply a list
of member\\r\\nstreams along with\ndashboards. An example of the
definition looks\nlike:\\r\\n\\r\\n```JSON\\r\\n{\\r\\n \\\"name\\\":
\\\"nginx-logs\\\",\\r\\n \\\"stream\\\":\n{\\r\\n \\\"grouped\\\":
{\\r\\n \\\"description\\\": \\\"A collection of streams
for\nNginx\\\",\\r\\n \\\"members\\\": [\\r\\n \\\"logs\\\",\\r\\n
\\\"logs.nginx\\\"\\r\\n ]\\r\\n\n}\\r\\n },\\r\\n \\\"dashboards\\\":
[]\\r\\n}\\r\\n```\\r\\nThe following APIs\nsupport
`GroupStreamDefinition`:\\r\\n\\r\\n- `GET /api/streams`\\r\\n-
`GET\n/api/streams/{id}`\\r\\n- `PUT /api/streams/{id}`\\r\\n-
`DELETE\n/api/streams/{id}`\\r\\n- `GET
/api/streams/{id}/_details`\\r\\n\\r\\nThis PR\nonly includes the
support to the APIs, I will submit a follow PR\\r\\nfor\nthe
UI.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<42973632+kibanamachine@users.noreply.github.com>\\r\\nCo-authored-by:
Joe\nReuter\n<johannes.reuter@elastic.co>\",\"sha\":\"2092c3d4ca2ae917420f992686fc7a704b393ea4\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"v9.0.0\",\"release_note:feature\",\"backport:version\",\"Feature:Streams\",\"v9.1.0\",\"v8.19.0\"],\"title\":\"[Streams\n🌊]
Introduce\nGroupStreams\",\"number\":208126,\"url\":\"https://github.com/elastic/kibana/pull/208126\",\"mergeCommit\":{\"message\":\"[Streams\n🌊]
Introduce GroupStreams (#208126)\\n\\n## 🍒 Summary\\r\\n \\r\\nThis PR
adds\nsupport to `/api/streams` endpoints for
the\\r\\n`GroupStreamDefinition`\ntype. Group streams are simply a list
of member\\r\\nstreams along with\ndashboards. An example of the
definition looks\nlike:\\r\\n\\r\\n```JSON\\r\\n{\\r\\n \\\"name\\\":
\\\"nginx-logs\\\",\\r\\n \\\"stream\\\":\n{\\r\\n \\\"grouped\\\":
{\\r\\n \\\"description\\\": \\\"A collection of streams
for\nNginx\\\",\\r\\n \\\"members\\\": [\\r\\n \\\"logs\\\",\\r\\n
\\\"logs.nginx\\\"\\r\\n ]\\r\\n\n}\\r\\n },\\r\\n \\\"dashboards\\\":
[]\\r\\n}\\r\\n```\\r\\nThe following APIs\nsupport
`GroupStreamDefinition`:\\r\\n\\r\\n- `GET /api/streams`\\r\\n-
`GET\n/api/streams/{id}`\\r\\n- `PUT /api/streams/{id}`\\r\\n-
`DELETE\n/api/streams/{id}`\\r\\n- `GET
/api/streams/{id}/_details`\\r\\n\\r\\nThis PR\nonly includes the
support to the APIs, I will submit a follow PR\\r\\nfor\nthe
UI.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<42973632+kibanamachine@users.noreply.github.com>\\r\\nCo-authored-by:
Joe\nReuter\n<johannes.reuter@elastic.co>\",\"sha\":\"2092c3d4ca2ae917420f992686fc7a704b393ea4\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.x\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/208126\",\"number\":208126,\"mergeCommit\":{\"message\":\"[Streams\n🌊]
Introduce GroupStreams (#208126)\\n\\n## 🍒 Summary\\r\\n \\r\\nThis PR
adds\nsupport to `/api/streams` endpoints for
the\\r\\n`GroupStreamDefinition`\ntype. Group streams are simply a list
of member\\r\\nstreams along with\ndashboards. An example of the
definition looks\nlike:\\r\\n\\r\\n```JSON\\r\\n{\\r\\n \\\"name\\\":
\\\"nginx-logs\\\",\\r\\n \\\"stream\\\":\n{\\r\\n \\\"grouped\\\":
{\\r\\n \\\"description\\\": \\\"A collection of streams
for\nNginx\\\",\\r\\n \\\"members\\\": [\\r\\n \\\"logs\\\",\\r\\n
\\\"logs.nginx\\\"\\r\\n ]\\r\\n\n}\\r\\n },\\r\\n \\\"dashboards\\\":
[]\\r\\n}\\r\\n```\\r\\nThe following APIs\nsupport
`GroupStreamDefinition`:\\r\\n\\r\\n- `GET /api/streams`\\r\\n-
`GET\n/api/streams/{id}`\\r\\n- `PUT /api/streams/{id}`\\r\\n-
`DELETE\n/api/streams/{id}`\\r\\n- `GET
/api/streams/{id}/_details`\\r\\n\\r\\nThis PR\nonly includes the
support to the APIs, I will submit a follow PR\\r\\nfor\nthe
UI.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<42973632+kibanamachine@users.noreply.github.com>\\r\\nCo-authored-by:
Joe\nReuter\n<johannes.reuter@elastic.co>\",\"sha\":\"2092c3d4ca2ae917420f992686fc7a704b393ea4\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208126","number":208126,"mergeCommit":{"message":"[Streams
🌊] Introduce GroupStreams (#208126)\n\n## 🍒 Summary\r\n \r\nThis PR adds
support to `/api/streams` endpoints for the\r\n`GroupStreamDefinition`
type. Group streams are simply a list of member\r\nstreams along with
dashboards. An example of the definition looks
like:\r\n\r\n```JSON\r\n{\r\n \"name\": \"nginx-logs\",\r\n \"stream\":
{\r\n \"grouped\": {\r\n \"description\": \"A collection of streams for
Nginx\",\r\n \"members\": [\r\n \"logs\",\r\n \"logs.nginx\"\r\n ]\r\n
}\r\n },\r\n \"dashboards\": []\r\n}\r\n```\r\nThe following APIs
support `GroupStreamDefinition`:\r\n\r\n- `GET /api/streams`\r\n- `GET
/api/streams/{id}`\r\n- `PUT /api/streams/{id}`\r\n- `DELETE
/api/streams/{id}`\r\n- `GET /api/streams/{id}/_details`\r\n\r\nThis PR
only includes the support to the APIs, I will submit a follow PR\r\nfor
the UI.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Joe
Reuter
<johannes.reuter@elastic.co>","sha":"2092c3d4ca2ae917420f992686fc7a704b393ea4"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: Milton Hultgren <milton.hultgren@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
@simianhacker simianhacker mentioned this pull request Feb 6, 2025
Merged
drewdaemon pushed a commit to drewdaemon/kibana that referenced this pull request Feb 6, 2025
@simianhacker @kibanamachine
@flash1293 @drewdaemon
[Streams 🌊] Introduce GroupStreams (elastic#208126)
0a83d0e 
## 🍒  Summary
 
This PR adds support to `/api/streams` endpoints for the
`GroupStreamDefinition` type. Group streams are simply a list of member
streams along with dashboards. An example of the definition looks like:

```JSON
{
  "name": "nginx-logs",
  "stream": {
    "grouped": {
      "description": "A collection of streams for Nginx",
      "members": [
        "logs",
        "logs.nginx"
      ]
    }
  },
  "dashboards": []
}
```
The following APIs support `GroupStreamDefinition`:

- `GET /api/streams`
- `GET /api/streams/{id}`
- `PUT /api/streams/{id}`
- `DELETE /api/streams/{id}`
- `GET /api/streams/{id}/_details`

This PR only includes the support to the APIs, I will submit a follow PR
for the UI.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
simianhacker added a commit that referenced this pull request Feb 20, 2025
@simianhacker @flash1293
[Streams 🌊] Enable /api/streams/{id}/_group endpoints for GroupStre…
8c90076 
…ams (#210114)

## Summary

This PR implements 2 endpoints as a follow up to
#208126 for working directly with
the `group` object for `GroupStreamDefinition`:

- `PUT /api/streams/{id}/_group`
- `GET /api/streams/{id}/_group`

---------

Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
simianhacker added a commit to simianhacker/kibana that referenced this pull request Feb 21, 2025
@simianhacker @flash1293
[Streams 🌊] Enable /api/streams/{id}/_group endpoints for GroupStre…
a0c70c0 
…ams (elastic#210114)

## Summary

This PR implements 2 endpoints as a follow up to
elastic#208126 for working directly with
the `group` object for `GroupStreamDefinition`:

- `PUT /api/streams/{id}/_group`
- `GET /api/streams/{id}/_group`

---------

Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
(cherry picked from commit 8c90076)
simianhacker added a commit that referenced this pull request Feb 25, 2025
@simianhacker
[8.x] [Streams 🌊] Enable /api/streams/{id}/_group endpoints for Gro…
ddaef9b 
…upStreams (#210114) (#212114)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Streams 🌊] Enable `/api/streams/{id}/_group` endpoints for
GroupStreams (#210114)](#210114)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Chris
Cowan","email":"chris@elastic.co"},"sourceCommit":{"committedDate":"2025-02-20T16:03:06Z","message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com//pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for
GroupStreams","number":210114,"url":"https://github.com/elastic/kibana/pull/210114","mergeCommit":{"message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com//pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210114","number":210114,"mergeCommit":{"message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com//pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
SoniaSanzV pushed a commit to SoniaSanzV/kibana that referenced this pull request Mar 4, 2025
@simianhacker @SoniaSanzV
[8.x] [Streams 🌊] Enable /api/streams/{id}/_group endpoints for Gro…
03adc30 
…upStreams (elastic#210114) (elastic#212114)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Streams 🌊] Enable `/api/streams/{id}/_group` endpoints for
GroupStreams (elastic#210114)](elastic#210114)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Chris
Cowan","email":"chris@elastic.co"},"sourceCommit":{"committedDate":"2025-02-20T16:03:06Z","message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(elastic#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com/elastic/pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for
GroupStreams","number":210114,"url":"https://github.com/elastic/kibana/pull/210114","mergeCommit":{"message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(elastic#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com/elastic/pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210114","number":210114,"mergeCommit":{"message":"[Streams
🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams
(elastic#210114)\n\n## Summary\n\nThis PR implements 2 endpoints as a follow up
to\nhttps://github.com/elastic/pull/208126 for working directly
with\nthe `group` object for `GroupStreamDefinition`:\n\n- `PUT
/api/streams/{id}/_group`\n- `GET
/api/streams/{id}/_group`\n\n---------\n\nCo-authored-by: Joe Reuter
<johannes.reuter@elastic.co>","sha":"8c90076d803177b0c1a9d17d00e78b242d5758ef"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Mar 22, 2025
@simianhacker @flash1293 @CAWilson94
[Streams 🌊] Enable /api/streams/{id}/_group endpoints for GroupStre…
25ffa23 
…ams (elastic#210114)

## Summary

This PR implements 2 endpoints as a follow up to
elastic#208126 for working directly with
the `group` object for `GroupStreamDefinition`:

- `PUT /api/streams/{id}/_group`
- `GET /api/streams/{id}/_group`

---------

Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@miltonhultgren miltonhultgren miltonhultgren approved these changes

@rmyz rmyz rmyz approved these changes

@flash1293 flash1293 Awaiting requested review from flash1293

Assignees

No one assigned

Labels

backport:version Backport to applied version labels Feature:Streams This is the label for the Streams Project release_note:feature Makes this part of the condensed release notes v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@simianhacker @flash1293 @elasticmachine @kibanamachine @miltonhultgren @rmyz @tonyghiani