Skip to content

Data Stream fields: Move to stage 3 #1212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ruflin merged 3 commits into from
Mar 18, 2021
Merged

Data Stream fields: Move to stage 3 #1212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b3f9302
Data Stream fields: Move to stage 3
ruflin Jan 6, 2021
17f24d9
Update rfcs/text/0009-data_stream-fields.md
ruflin Mar 18, 2021
9cfa3db
Fix rendering of restricted data_stream fields
felixbarny Mar 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions rfcs/text/0009-data_stream-fields.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# 0009: Data stream fields
<!-- Leave this ID at 0000. The ECS team will assign a unique, contiguous RFC number upon merging the initial stage of this RFC. -->

- Stage: **2 (draft)** <!-- Update to reflect target stage. See https://elastic.github.io/ecs/stages.html -->
- Stage: **3 (finished)** <!-- Update to reflect target stage. See https://elastic.github.io/ecs/stages.html -->
- Date: **2021-01-04** <!-- The ECS team sets this date at merge time. This is the date of the latest stage advancement. -->

When introducing the new indexing strategy for Elastic Agent which uses data streams, we found that adding a few [constant_keyword](https://www.elastic.co/guide/en/elasticsearch/reference/master/keyword.html#constant-keyword-field-type) fields corresponding to the central components in the new indexing strategy would be advantageous.
Expand Down Expand Up @@ -35,7 +35,7 @@ The fields can be found in `rfcs/text/0009/data_stream.yml`.

### Restrictions on values

Due to the fact that the values of the `data_stream` fields make up the data stream name, the restrictions on data stream names also apply to values for the `data_stream` fields. As an example, they cannot include \, /, *, ?, ", <, >, |, ` `. Please see the Elasticsearch reference for [restrictions on index/data stream names](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params). Here follows the _additional_ restrictions imposed on the data stream fields:
Due to the fact that the values of the `data_stream` fields make up the data stream name, the restrictions on data stream names also apply to values for the `data_stream` fields. As an example, they cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, ` `, `,`, `#`. Please see the Elasticsearch reference for [restrictions on index/data stream names](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params). Here follows the _additional_ restrictions imposed on the data stream fields:

**data_stream.type**

Expand Down Expand Up @@ -246,7 +246,7 @@ e.g.:
* Previous discussion on [dataset fields](https://github.com/elastic/ecs/pull/845)
* Discussion on [field value restrictions](https://github.com/elastic/kibana/issues/75846)
* Restrictions on [index names](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html)

* Blog post: [An introduction to the Elastic data stream naming scheme](https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme)

### RFC Pull Requests

Expand Down
2 changes: 1 addition & 1 deletion rfcs/text/0009/data_stream.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
description: >
The data_stream fields are part defining the new data stream naming scheme.

In the new data stream naming scheme the value of the data stream fields combine to the name of the actual data stream in the following manner `{data_stream.type}-{data_stream.dataset}-{data_stream.namespace}`. This means the fields can only contain characters that are valid as part of names of data streams. More details about this can be found in this blog post. TODO: Add link to blog post
In the new data stream naming scheme the value of the data stream fields combine to the name of the actual data stream in the following manner `{data_stream.type}-{data_stream.dataset}-{data_stream.namespace}`. This means the fields can only contain characters that are valid as part of names of data streams. More details about this can be found in this [blog post](https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme).

Due to the fact that the values of the `data_stream` fields make up the data stream name, the restrictions on data stream names also apply to values for the `data_stream` fields. As an example, they cannot include \, /, *, ?, ", <, >, |, ` `. Please see the Elasticsearch reference for [restrictions on index/data stream names](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params).

Expand Down