Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
7472d90
Add mapping network event guidance doc (#969)
ebeahan Sep 29, 2020
1ac936a
Removing unneeded link under `Additional Information` (#984)
ebeahan Sep 29, 2020
fd822f0
Add discrete attribute to field details page headers (#989)
ebeahan Sep 30, 2020
b106d3a
[RFC] Multiple users in an event (#914)
Oct 2, 2020
9c562b0
Add --oss flag to the ECS generator script (#991)
Oct 2, 2020
35ea308
Uniformity across domain name breakdown fields (#981)
ebeahan Oct 2, 2020
c7422e8
[RFC] Wildcard - stage 2 proposal (#970)
ebeahan Oct 2, 2020
e29b7b1
Fix the month in the stage advancement date for RFC-0007 (#996)
Oct 2, 2020
1d32671
Add network directions ingress and egress (#945)
Oct 2, 2020
76ec52d
Mention ECS Mapper in the main documentation (#987)
DanRoscigno Oct 5, 2020
08b1a08
Introduce experimental artifacts (#993)
ebeahan Oct 5, 2020
9212952
Cut 1.7 changelog (#1010) (#1013)
Oct 6, 2020
f78f1f9
Clarify that file extension should exclude the dot. (#1016)
Oct 7, 2020
203bb01
Add usage docs section (#988)
ebeahan Oct 8, 2020
b976f98
Add 1.7 as a backport target branch (#1025)
ebeahan Oct 8, 2020
b8d008c
[RFC] Add host metric fields to ECS (#950)
kaiyan-sheng Oct 13, 2020
954f294
Set the correct merge date for RFC 0005 stage 1 (#1031)
Oct 15, 2020
20ebe35
[RFC] Threat intelligence (#986)
shimonmodi Oct 15, 2020
78c8549
feat: include alias path when generating template (#877)
rgmz Oct 16, 2020
9704ec3
Add TLDR example to USAGE.md and full example in `usage-example/` (#1…
Oct 19, 2020
40ffbcf
Update Jinja2 section to reflect template refactoring (#1018)
ebeahan Oct 21, 2020
4562e11
New default make task that generates main and experimental artifacts.…
Oct 21, 2020
260a3eb
Add support for `scaling_factor` in the generator (#1042)
Oct 21, 2020
13a12c0
Correct experimental usage example (#1045)
ebeahan Oct 22, 2020
472041e
Fix the discuss link in the GH issue templates (#1047)
Oct 26, 2020
d3546c9
Add fallback for constant_keyword (#1046)
Oct 26, 2020
f868575
Add event.category session. (#1049)
Oct 27, 2020
d75353a
Add event.category registry (#1040)
Oct 27, 2020
dfd5c5a
Add wildcard type support to go code generator (#1050)
ebeahan Oct 27, 2020
0fb41a1
Add constant_keyword to OSS fallback types list (#1058)
ebeahan Oct 28, 2020
67a596e
Change the index pattern in the sample template. (#1048)
Nov 2, 2020
ec19b97
Prepare link to Logs docs changing with the 7.10 release in "products…
Nov 4, 2020
9008a54
Prepare link to Logs docs changing with the 7.10 release in "getting-…
Nov 4, 2020
cba2dfa
[RFC] Cyber Threat Intelligence Fields Stage 1 (#1037)
shimonmodi Nov 9, 2020
c1eae26
Correct advancement date for RFC 0008 Stage 1 (#1100)
ebeahan Nov 10, 2020
d9df1ab
Add --ref support for experimental artifacts (#1063)
ebeahan Nov 10, 2020
18fc6af
Remove experimental event.original definition (#1053)
ebeahan Nov 10, 2020
552abeb
Add missing `process.thread.name` to experimental definitions (#1103)
ebeahan Nov 10, 2020
124dbfb
[RFC] Wildcard - stage 3 proposal (#1015)
ebeahan Nov 11, 2020
fad7fa8
[RFC] data_stream fields (#980)
roncohen Nov 11, 2020
6cbb544
[RFC] Multiple users in an event, stage 3 PR (#1017)
Nov 11, 2020
42bb9c7
Remove index parameter for wildcard fields (#1115)
ebeahan Nov 12, 2020
881da08
Add dns.answer object into experimental schema (#1118)
ebeahan Nov 12, 2020
cc674c8
Clarify x509 definition guidance for network events with only one cer…
Nov 12, 2020
4ee5c21
Indicate when artifacts include experimental changes (#1117)
Nov 12, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .backportrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"branches": [
{ "name": "master", "checked": true },
{ "name": "1.x", "checked": true },
"1.7",
"1.6",
"1.5",
"1.4",
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Question
url: https://discuss.elastic.co/tag/elastic-common-schema
url: https://discuss.elastic.co/tag/ecs-elastic-common-schema
about: Ask (and answer) questions here.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/schema-issue.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ labels: "bug"
<!--
GitHub is reserved for bug reports and feature requests; it is not the place
for general questions. If you have a general question, please visit the
[forums](https://discuss.elastic.co/tag/elastic-common-schema).
discuss forums: https://discuss.elastic.co/tag/ecs-elastic-common-schema.

Please fill in the following details to help us triage the issue:
-->
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/tooling-bug.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ labels: "bug"
<!--
GitHub is reserved for bug reports and feature requests; it is not the place
for general questions. If you have a general question, please visit the
[forums](https://discuss.elastic.co/tag/elastic-common-schema).
discuss forums: https://discuss.elastic.co/tag/ecs-elastic-common-schema.

Please fill in the following details to help us reproduce the bug:
-->
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ build
.idea
*.iml
.vscode/*

# experimental exclusions
experimental/generated/elasticsearch/6
experimental/generated/docs
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,59 @@
# CHANGELOG
All notable changes to this project will be documented in this file based on the [Keep a Changelog](http://keepachangelog.com/) Standard. This project adheres to [Semantic Versioning](http://semver.org/).

## [1.7.0](https://github.com/elastic/ecs/compare/v1.6.0...v1.7.0)

### Schema Changes

#### Bugfixes

* The `protocol` allowed value under `event.type` should not have the `expected_event_types` defined. #964
* Clarify the definition of `file.extension` (no dots). #1016

#### Added

* Added Mime Type fields to HTTP request and response. #944
* Added network directions ingress and egress. #945
* Added `threat.technique.subtechnique` to capture MITRE ATT&CK® subtechniques. #951
* Added `configuration` as an allowed `event.category`. #963
* Added a new directory with experimental artifacts, which includes all changes
from RFCs that have reached stage 2. #993, #1053, #1115, #1117, #1118

#### Improvements

* Expanded field set definitions for `source.*` and `destination.*`. #967
* Provided better guidance for mapping network events. #969
* Added the field `.subdomain` under `client`, `destination`, `server`, `source`
and `url`, to match its presence at `dns.question.subdomain`. #981
* Clarified ambiguity in guidance on how to use x509 fields for connections with
only one certificate. #1114

### Tooling and Artifact Changes

#### Breaking changes

* Changed the index pattern of the sample Elasticsearch template from `ecs-*` to
`try-ecs-*` to avoid conflicting with Logstash' `ecs-logstash-*`. #1048

#### Bugfixes

* Addressed issue where foreign reuses weren't using the user-supplied `as` value for their destination. #960
* Experimental artifacts failed to install due to `event.original` index setting. #1053

#### Added

* Introduced `--strict` flag to perform stricter schema validation when running the generator script. #937
* Added check under `--strict` that ensures composite types in example fields are quoted. #966
* Added `ignore_above` and `normalizer` support for keyword multi-fields. #971
* Added `--oss` flag for users who want to generate ECS templates for use on OSS clusters. #991

#### Improvements

* Field details Jinja2 template components have been consolidated into one template #897
* Add `[discrete]` marker before each section header in field details. #989
* `--ref` now loads `experimental/schemas` based on git ref in addition to `schemas`. #1063


## [1.6.0](https://github.com/elastic/ecs/compare/v1.5.0...v1.6.0)

### Schema Changes
Expand Down
21 changes: 7 additions & 14 deletions CHANGELOG.next.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,13 @@ Thanks, you're awesome :-) -->

#### Bugfixes

* The `protocol` allowed value under `event.type` should not have the `expected_event_types` defined. #964

#### Added

* Added Mime Type fields to HTTP request and response. #944
* Added `threat.technique.subtechnique` to capture MITRE ATT&CK® subtechniques. #951
* Added `configuration` as an allowed `event.category`. #963
* Added `event.category` "registry". #1040
* Added `event.category` "session". #1049

#### Improvements

* Expanded field set definitions for `source.*` and `destination.*`. #967

#### Deprecated

### Tooling and Artifact Changes
Expand All @@ -34,18 +29,16 @@ Thanks, you're awesome :-) -->

#### Bugfixes

* Addressed issue where foreign reuses weren't using the user-supplied `as` value for their destination. #960

#### Added

* Introduced `--strict` flag to perform stricter schema validation when running the generator script. #937
* Added check under `--strict` that ensures composite types in example fields are quoted. #966
* Added `ignore_above` and `normalizer` support for keyword multi-fields. #971
* Added ability to supply free-form usage documentation per fieldset. #988
* Added the `path` key when type is `alias`, to support the [alias field type](https://www.elastic.co/guide/en/elasticsearch/reference/current/alias.html). #877
* Added support for `scaled_float`'s mandatory parameter `scaling_factor`. #1042
* Added ability for --oss flag to fall back `constant_keyword` to `keyword`. #1046
* Added support in the generated Go source go for `wildcard`, `version`, and `constant_keyword` data types. #1050

#### Improvements

* Field details Jinja2 template components have been consolidated into one template #897

#### Deprecated


Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ The following files are generated based on the current schema using [Jinja](http
| ------------------ | -------- |
| [fields.asciidoc](docs/fields.asciidoc) | [fields_template.j2](scripts/templates/fields_template.j2) |
| [fields-values.asciidoc](docs/field-values.asciidoc) | [field_values_template.j2](scripts/templates/field_values_template.j2) |
| [field-details.asciidoc](docs/field-details.asciidoc) | [field_details directory](scripts/templates/field_details) |
| [field-details.asciidoc](docs/field-details.asciidoc) | [field_details.j2](scripts/templates/field_details.j2) |

Running `make` will update these files using the [scripts/generators/asciidoc_fields.py](scripts/generators/asciidoc_fields.py) generator. These doc files should *not* be modified directly. Any changes as a result of a schema update and subsequent run of `make` *should* be committed.

Expand Down
17 changes: 13 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#
# Variables
#
.DEFAULT_GOAL := generate
.DEFAULT_GOAL := all
FIND := find . -type f -not -path './build/*' -not -path './.git/*'
FORCE_GO_MODULES := GO111MODULE=on
OPEN_DOCS ?= "--open"
Expand All @@ -12,10 +12,14 @@ VERSION := $(shell cat version)
# Targets (sorted alphabetically)
#

# Default build generates main and experimental artifacts
.PHONY: all
all: generate experimental

# Check verifies that all of the committed files that are generated are
# up-to-date.
.PHONY: check
check: generate test fmt misspell makelint check-license-headers
check: generate experimental test fmt misspell makelint check-license-headers
# Check if diff is empty.
git diff | cat
git update-index --refresh
Expand Down Expand Up @@ -44,7 +48,12 @@ docs:
if [ ! -d $(PWD)/build/docs ]; then \
git clone --depth=1 https://github.com/elastic/docs.git ./build/docs ; \
fi
./build/docs/build_docs --asciidoctor --doc ./docs/index.asciidoc --chunk=1 $(OPEN_DOCS) --out ./build/html_docs
./build/docs/build_docs --asciidoctor --doc ./docs/index.asciidoc --chunk=2 $(OPEN_DOCS) --out ./build/html_docs

# Alias to generate experimental artifacts
.PHONY: experimental
experimental: ve
$(PYTHON) scripts/generator.py --include experimental/schemas --out experimental

# Format code and files in the repo.
.PHONY: fmt
Expand All @@ -55,7 +64,7 @@ fmt: ve

# Alias to generate everything.
.PHONY: generate
generate: legacy_use_cases codegen generator
generate: generator legacy_use_cases codegen
$(PYTHON) --version

# Run the new generator
Expand Down
Loading