Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync cloud 4.0 #10115

Merged
merged 38 commits into from
Feb 23, 2021
Merged

Sync cloud 4.0 #10115

merged 38 commits into from
Feb 23, 2021

Conversation

edmundoa
Copy link
Contributor

@edmundoa edmundoa commented Feb 22, 2021

Update cloud-4.0 with the latest 4.0 changes.

/jenkins-pr-deps Graylog2/graylog-plugin-enterprise#2070,Graylog2/graylog-plugin-enterprise-integrations#477,Graylog2/graylog-plugin-integrations#748,Graylog2/graylog-plugin-cloud#749

kmerz and others added 30 commits December 16, 2020 14:33
## Motivation
Prior to this change, a dashboard installed by a content pack had a
tabs in a random order. Since we cannot change the order of the tabs
this is quite inconvinient for the user.

## Description
This change will do mutiple steps to fix that:
- Use a LinkedHashMap to ensure the order stays fix during content pack
  creation.
- Instead of using the serach QueryIds we use the views QueryIds which
  we previously fixed in the order.
- When loading a view and selecting the activeQuery we also use the
  view.state queryids instead of the search ones.

Fixes #8264
* Display the timezone settings to the user

## Motivation
Prior to this change, the user was not able to change his own timezone
settings since they were hidden to him.

## Description
This change will show the settings to the user so he can edit them.

* Fix settings section test
This provides a fix for the following security issue:
GHSA-3329-pjwv-fjpg

Backport of #9857 for 4.0
…ings. (#9877) (#9887)

* Consolidating backend-agnostic query parsing/metadata creation.

* Fixing test.

* Adding test cases for bug.

* Adjusting regex for query string parser.

* Removing unused parameter.

Co-authored-by: Dennis Oelkers <[email protected]>
The new paginated endpoint in StreamResource is using mongojack to
deserialize the database objects. In some setups the AlertReceivers
object in the database might not be complete.

Change AlertReceivers to use defaults for missing values.

Fixes #9637

(cherry picked from commit 1e456af)
The `PageSizeSelect` component passes an event as argument to its
`onChange` function but but we were expecting a string containing
a number. This change fixes that by extracting the value from the
received event.

Fixes #9629

Co-authored-by: Edmundo Alvarez <[email protected]>
* Avoid using a stage number that is already used

 Check if selected stage number already exists in pipeline, displaying
 an error in that case and not allowing to submit the form. This will
 prevent users of accidentally overriding one of their existing stages.

 Additionally improve the initial suggestion for new stages. Before we
 always displayed `0` when creating a stage, now we base that value in
 the existing stages for the pipeline.

 Fixes #6806

* Do not submit form when using an existing stage
* Retry read timeouts for Elasticsearch.

Before this change, the retry strategy used for ES requests did not
retry for instances of `SocketTimeoutException`, raised in case of read
timeouts. This resulted in exceptions being raised when individual nodes
were not responding and node discovery was not used. This was e.g. being
noticable when performing searches and 1 of 3 nodes was failing, so
every 3rd search request was timing out and returning an error instead
of being retried and successful and just taking longer.

This change is adding this class to the exceptions being retried, so any
request that goes out to a node which is not responding on time is
retried.

* Removing duplicate license header.
* Fix date format specifiers in index mapping templates for ES7.

Before this change, the pre-ES7 date format specifiers were reused for
ES7 to define the format of date fields in index mapping templates.

Unfortunately, this results in warnings being returned in responses to
searches operating on date fields, due to a [breaking change introduced
in ES7](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#_joda_based_date_formatters_are_replaced_with_java_ones).

This change is now splitting up the date format specifiers and uses joda
syntax for ES6 and `java.time` syntax for ES7. In addition, the overly
lax exposition of the date format specifiers from the `Tools` class was
locked down to prevent them leaking into new places.

Fixes #9690.

* Adjusting tests/expected results.

* Adjusting single date format in GIM Mapping for v7 too.

* Prefixing date format for ES6 to disable deprecation notice.

As we are using a deprecated formatter according to the [ES breaking changes](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#_joda_based_date_formatters_are_replaced_with_java_ones),
this commit is adding the `8` prefix to prevent a warning.

* Prefixing legacy date formats in GIM mapping.

* Adjusting tests to use `8yyyy`.
Before this change we were mutating this.state on input changes directly which resulted in a strange behaviour.
One result was that the stream rule form did not reset after creating a stream rule, even though the component is unmounting correctly.
Have a look at #9956 for more information.
* Adding test to check if indexing retries for invalid alias targets.

* Adding indexing listener to bulk indexing to improve tests.

* Suppressing specific warnings.

* Introducing `InvalidWriteTargetException`.

* Retrying bulk indexing upon `InvalidWriteTargetException`.

* Relaxing ES exception regex.

* Throw specific exception if parsed exception indices invalid write target.

* Fixing error parsing for ESS6, extracting target for exception.

* Reverting changes to `ElasticsearchException`.

* Adding related test case for ES exception parser.

* Fix implicit mutation of retryer builder when listener is present.

Fixes #9955
Fixes #9962
## Motivation
Prior to this change, when loading the new dashboard containing the
previous search we used a not qualified route, which did not take the
prefix in account.

## Description
This change will use `Routes` to set the new pathname for history.push.
…map (#10007)

* Provide default position in case the widget isn't found in positions map (#9841)

* Provide default position in case the widget isn't found in positions map

## Motivation
Prior to this change, a widget copied from a search was unable to be
copied to a different page since the position was missing in
the positions map.

## Description
This change will add a safty net in case the position is missing we
provide the widgets default size.

* Use AddNewWidgetsToPosition for CopyWidgetToDashboard

That way we can saftly assume that future copied widgets will not have a missing position

The NaN values in the test can be ignored, since the widgets are not
existing anymore.

* Fix linter warning

* Fix license header

* Clear positions from stale entries

* Provide defaultHeight/Width to test

* Update snapshots

* Refactor position calculation for new widgets

* Fix linter warnings

* Create new id when copying a widget

* Reuse old sizes if possible

* Rename last leftover import

* Fix linter

* Don't mix responsibilities

* Update snapshots

* Fix merge errors

* Add missing flow annotation

* Remove unused files

* Fix tests
This shouldn't have been released and can cause a
high load on MongoDB.

Fixes #10043

(cherry picked from commit 40fbb82)
* Creating integration test for `Indices#numberOfMessages`.

* Correcting json path, adding safeguard if not present.
* Add missing GRN type for outputs

Fixes #10088

* Add output permissions to the capabilities

Without this, the dependency check doesn't run properly
because it filters out any permission that isn't handled via grants.
That's another bug we should be fixing..

* Add TODO comment

* Hide some dependencies that are not relevant for sharing entities

* Use Set instead of List for ignored dependcies lookup

(cherry picked from commit 8eec5db)
…0098)

* Use custom timeout when optimizing index for ES7.

* Actually return value, use correct import.

* Validating config setting to make sure it can be safely casted to int.

* Actually use 24 days as maximum.
dennisoelkers and others added 5 commits February 22, 2021 19:06
* Adding test case to check if index optimization throws NPE.

* If request config is `null`, create new one instead of copying.

* Check it `requestOptions` is `null`.
@ousmaneo
Copy link
Contributor

Hi @edmundoa, it seems there is already a new 4.0.6-SNAPSHOT

Copy link
Contributor

@Gaya Gaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I'll leave it up to the Java specialists 😊

@edmundoa
Copy link
Contributor Author

Updated all PRs to use 4.0.6-SNAPSHOT, please re-check after tests are green :)

Copy link
Contributor

@ousmaneo ousmaneo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legit !

@ousmaneo ousmaneo merged commit bf38aee into cloud-4.0 Feb 23, 2021
@ousmaneo ousmaneo deleted the sync-cloud-4.0 branch February 23, 2021 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants