Skip to content

[7.x] [SIEM][Detection Engine] Converts from joi to use io-ts and moves the types to common (#68127)#68611

Merged
FrankHassanabad merged 1 commit intoelastic:7.xfrom
FrankHassanabad:backport/7.x/pr-68127
Jun 9, 2020
Merged

[7.x] [SIEM][Detection Engine] Converts from joi to use io-ts and moves the types to common (#68127)#68611
FrankHassanabad merged 1 commit intoelastic:7.xfrom
FrankHassanabad:backport/7.x/pr-68127

Conversation

@FrankHassanabad
Copy link
Copy Markdown
Contributor

Backports the following commits to 7.x:

… types to common (elastic#68127)

## Summary
* https://github.com/elastic/siem-team/issues/646
* Converts the detection rules and REST to use io-ts
* Removes their joi counterparts
* Updates all tests to use it
* Fixes a bug with the risk_score that was being sent in as a string from the UI instead of a number
* Fixes a bug within the exactCheck validating where it can now accept null value types for optional body messages.
* Fixes a bug in the FindRoute where it did not send down fields from REST
* Changes the lists plugin to utilize the io-ts types from siem rather than having them duplicated.
* Makes some stronger validations
* Adds a lot of codecs

**Things to look out for:**

* Generic testing to ensure I didn't break something that was not part of the tests.
* Fix for the risk_score from string to number is in:
```
x-pack/plugins/security_solution/public/alerts/components/rules/step_about_rule/index.test.tsx
```
* Fix for the exact check (unit tests are written and added)
```
x-pack/plugins/security_solution/public/alerts/components/rules/step_about_rule/index.test.tsx
```
* Within all the types I added are there any misspelled things or copy-pasta mistakes with strings:
x-pack/plugins/security_solution/common/detection_engine/schemas/types
* Fix for `find_rules_route.ts:58`
```
x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/find_rules_route.ts
```

**Follow on things that this PR doesn't do we need to:**
* Add linter rule to forbid NodeJS code within common section
* The `[object Object]` formatter issues seen in the code such as:
```
// TODO: Fix/Change the formatErrors to be better able to handle objects
'Invalid value "[object Object]" supplied to "note"',
```
* Formatter issues such as: `'Invalid value "" supplied to ""'`
* Remove the hapi server object from lists plugin

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
@FrankHassanabad FrankHassanabad added the backport This PR is a backport of another PR label Jun 9, 2020
@kibanamachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / kibana-oss-agent / Firefox UI Functional Tests.test/functional/apps/home/_home·js.homepage app Kibana takes you home clicking on kibana logo should take you to home page

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:11:32]         └-: homepage app
[00:11:32]           └-> "before all" hook
[00:11:32]           └-> "before all" hook
[00:11:32]           └-: Kibana takes you home
[00:11:32]             └-> "before all" hook
[00:11:32]             └-> clicking on kibana logo should take you to home page
[00:11:32]               └-> "before each" hook: global before each
[00:11:32]               │ debg navigating to settings url: http://localhost:6111/app/management
[00:11:32]               │ debg navigate to: http://localhost:6111/app/management
[00:11:33]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:11:33]               │ debg ... sleep(700) start
[00:11:33]               │ debg ... sleep(700) end
[00:11:33]               │ debg returned from get, calling refresh
[00:11:35]               │ debg browser[info] "INFO: 2020-06-09T02:25:50Z\n  Adding connection to http://localhost:6111/elasticsearch\n\n"
[00:11:35]               │ERROR browser[error] (new TypeError("NetworkError when attempting to fetch resource.", ""))
[00:11:35]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:11:35]               │ debg currentUrl = http://localhost:6111/app/management
[00:11:35]               │          appUrl = http://localhost:6111/app/management
[00:11:35]               │ debg TestSubjects.find(kibanaChrome)
[00:11:35]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:11:39]               │ debg browser[info] "INFO: 2020-06-09T02:25:54Z\n  Adding connection to http://localhost:6111/elasticsearch\n\n"
[00:11:40]               │ debg ... sleep(501) start
[00:11:40]               │ debg ... sleep(501) end
[00:11:40]               │ debg in navigateTo url = http://localhost:6111/app/management
[00:11:40]               │ debg TestSubjects.exists(statusPageContainer)
[00:11:40]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:11:43]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:11:43]               │ debg TestSubjects.click(headerGlobalNav > logo)
[00:11:43]               │ debg Find.clickByCssSelector('[data-test-subj="headerGlobalNav"] [data-test-subj="logo"]') with timeout=10000
[00:11:43]               │ debg Find.findByCssSelector('[data-test-subj="headerGlobalNav"] [data-test-subj="logo"]') with timeout=10000
[00:11:43]               │ERROR browser[error] (new Error("ScopedHistory instance has fell out of navigation scope for basePath: /app/management", "http://localhost:6111/32273/bundles/core/core.entry.js", 76))
[00:11:43]               │ERROR browser[error] (new Error("ScopedHistory instance has fell out of navigation scope for basePath: /app/management", "http://localhost:6111/32273/bundles/core/core.entry.js", 76))
[00:11:43]               │ debg isGlobalLoadingIndicatorVisible
[00:11:43]               │ debg TestSubjects.exists(globalLoadingIndicator)
[00:11:43]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:11:43]               │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:11:43]               │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:13:23]               │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/homepage app Kibana takes you home clicking on kibana logo should take you to home page.png"
[00:13:23]               │ info Current URL is: http://localhost:6111/app/home#/
[00:13:24]               │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/homepage app Kibana takes you home clicking on kibana logo should take you to home page.html
[00:13:24]               └- ✖ fail: "homepage app Kibana takes you home clicking on kibana logo should take you to home page"
[00:13:24]               │

Stack Trace

Error: expected testSubject(globalLoadingIndicator-hidden) to exist
    at TestSubjects.existOrFail (test/functional/services/common/test_subjects.ts:62:15)
    at process._tickCallback (internal/process/next_tick.js:68:7)

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@FrankHassanabad FrankHassanabad merged commit a1e979a into elastic:7.x Jun 9, 2020
@FrankHassanabad FrankHassanabad deleted the backport/7.x/pr-68127 branch June 9, 2020 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants