diff --git a/docs/discover/kuery.asciidoc b/docs/discover/kuery.asciidoc index d25d1a71c9738..e82575ada1fc4 100644 --- a/docs/discover/kuery.asciidoc +++ b/docs/discover/kuery.asciidoc @@ -3,22 +3,32 @@ experimental[This functionality is experimental and may be changed or removed completely in a future release.] +[NOTE] +============ +Breaking changes were made to Kuery's experimental syntax in 6.3. Read on for details of the new syntax. +============ + Kuery is a new query language built specifically for Kibana. It aims to simplify the search experience in Kibana and enable the creation of helpful features like auto-complete, seamless migration of saved searches, additional query types, and more. Kuery is a basic experience today but we're hard at work building these additional features on top of the foundation Kuery provides. -Kueries are built with functions. Many functions take a field name as their first argument. Extremely common functions have shorthand notations. +If you're familiar with Kibana's old lucene query syntax, you should feel right at home with Kuery. Both languages +are very similar, but there are some differences we'll note along the way. -`is("response", 200)` will match documents where the response field matches the value 200. -`response:200` does the same thing. `:` is an alias for the `is` function. +`response:200` will match documents where the response field matches the value 200. -Multiple search terms are separated by whitespace. +Quotes around a search term will initiate a phrase search. For example, `message:"Quick brown fox"` will search +for the phrase "quick brown fox" in the message field. Without the quotes, your query will get broken down into tokens via +the message field's configured analyzer and will match documents that contain those tokens, regardless of the order in which +they appear. This means documents with "quick brown fox" will match, but so will "quick fox brown". Remember to use quotes if you want +to search for a phrase. -`response:200 extension:php` will match documents where response matches 200 and extension matches php. +Unlike lucene, Kuery will not split on whitespace. Multiple search terms must be separated by explicit +boolean operators. Note that boolean operators in Kuery are not case sensitive. -*All terms must match by default*. The language supports boolean logic with and/or operators. The above query is equivalent to `response:200 and extension:php`. -This is a departure from the Lucene query syntax where all terms are optional by default. +`response:200 extension:php` in lucene would become `response:200 and extension:php`. + This will match documents where response matches 200 and extension matches php. We can make terms optional by using `or`. @@ -32,85 +42,40 @@ We can override the default precedence with grouping. `response:200 and (extension:php or extension:css)` will match documents where response is 200 and extension is either php or css. -Terms can be inverted by prefixing them with `!`. +A shorthand exists that allows us to easily search a single field for multiple values. + +`response:(200 or 404)` searches for docs where the `response` field matches 200 or 404. We can also search for docs +with multi-value fields that contain a list of terms, for example: `tags:(success and info and security)` -`!response:200` will match all documents where response is not 200. +Terms can be inverted by prefixing them with `not`. + +`not response:200` will match all documents where response is not 200. Entire groups can also be inverted. -`response:200 and !(extension:php or extension:css)` +`response:200 and not (extension:php or extension:css)` + +Ranges in Kuery are similar to lucene with a small syntactical difference. + +Instead of `bytes:>1000`, Kuery omits the colon: `bytes > 1000`. + +`>, >=, <, <=` are all valid range operators. + +Exist queries are simple and do not require a special operator. `response:*` will find all docs where the response +field exists. -Some query functions have named arguments. +Wildcard queries are available. `machine.os:win*` would match docs where the machine.os field starts with "win", which +would match values like "windows 7" and "windows 10". -`range("bytes", gt=1000, lt=8000)` will match documents where the bytes field is greater than 1000 and less than 8000. +Wildcards also allow us to search multiple fields at once. This can come in handy when you have both `text` and `keyword` +versions of a field. Let's say we have `machine.os` and `machine.os.keyword` fields and we want to check both for the term +"windows 10". We can do it like this: `machine.os*:windows 10". -Quotes are generally optional if your terms don't have whitespace or special characters. `range(bytes, gt=1000, lt=8000)` -would also be a valid query. [NOTE] ============ -Terms without fields will be matched against all fields. For example, a query for `response:200` will search for the value 200 +Terms without fields will be matched against the default field in your index settings. If a default field is not +set these terms will be matched against all fields. For example, a query for `response:200` will search for the value 200 in the response field, but a query for just `200` will search for 200 across all fields in your index. ============ -==== Function Reference - -[horizontal] -Function Name:: Description - -and:: -Purpose::: Match all given sub-queries -Alias::: `and` as a binary operator -Examples::: -* `and(response:200, extension:php)` -* `response:200 and extension:php` - -or:: -Purpose::: Match one or more sub-queries -Alias::: `or` as a binary operator -Examples::: -* `or(extension:css, extension:php)` -* `extension:css or extension:php` - -not:: -Purpose::: Negates a sub-query -Alias::: `!` as a prefix operator -Examples::: -* `not(response:200)` -* `!response:200` - -is:: -Purpose::: Matches a field with a given term -Alias::: `:` -Examples::: -* `is("response", 200)` -* `response:200` - -range:: -Purpose::: Match a field against a range of values. -Alias::: `:[]` -Examples::: -* `range("bytes", gt=1000, lt=8000)` -* `bytes:[1000 to 8000]` -Named arguments::: -* `gt` - greater than -* `gte` - greater than or equal to -* `lt` - less than -* `lte` - less than or equal to - -exists:: -Purpose::: Match documents where a given field exists -Examples::: `exists("response")` - -geoBoundingBox:: -Purpose::: Creates a geo_bounding_box query -Examples::: -* `geoBoundingBox("coordinates", topLeft="40.73, -74.1", bottomRight="40.01, -71.12")` (whitespace between lat and lon is ignored) -Named arguments::: -* `topLeft` - the top left corner of the bounding box as a "lat, lon" string -* `bottomRight` - the bottom right corner of the bounding box as a "lat, lon" string - -geoPolygon:: -Purpose::: Creates a geo_polygon query given 3 or more points as "lat, lon" -Examples::: -* `geoPolygon("geo.coordinates", "40.97, -127.26", "24.20, -84.375", "40.44, -66.09")` \ No newline at end of file diff --git a/src/core_plugins/kibana/public/dashboard/dashboard_app.html b/src/core_plugins/kibana/public/dashboard/dashboard_app.html index 7f124c25a979c..ad4180aea3fb5 100644 --- a/src/core_plugins/kibana/public/dashboard/dashboard_app.html +++ b/src/core_plugins/kibana/public/dashboard/dashboard_app.html @@ -39,7 +39,6 @@ ng-show="showFilterBar()" state="state" index-patterns="indexPatterns" - ng-if="['lucene', 'kql'].includes(model.query.language)" >
diff --git a/src/core_plugins/kibana/public/visualize/editor/editor.html b/src/core_plugins/kibana/public/visualize/editor/editor.html index c885e06d2fb07..5506dc1afc4b5 100644 --- a/src/core_plugins/kibana/public/visualize/editor/editor.html +++ b/src/core_plugins/kibana/public/visualize/editor/editor.html @@ -50,7 +50,7 @@ diff --git a/src/core_plugins/tile_map/public/coordinate_maps_visualization.js b/src/core_plugins/tile_map/public/coordinate_maps_visualization.js index 05a78410a2be2..1b4dfe08a5b5e 100644 --- a/src/core_plugins/tile_map/public/coordinate_maps_visualization.js +++ b/src/core_plugins/tile_map/public/coordinate_maps_visualization.js @@ -117,39 +117,11 @@ export function CoordinateMapsVisualizationProvider(Notifier, Private) { const indexPatternName = agg.vis.indexPattern.id; const field = agg.fieldName(); - const query = this.vis.API.queryManager.getQuery(); - const language = query.language; + const filter = { meta: { negate: false, index: indexPatternName } }; + filter[filterName] = { ignore_unmapped: true }; + filter[filterName][field] = filterData; - if (['lucene', 'kql'].includes(language)) { - const filter = { meta: { negate: false, index: indexPatternName } }; - filter[filterName] = { ignore_unmapped: true }; - filter[filterName][field] = filterData; - - this.vis.API.queryFilter.addFilters([filter]); - } - else if (language === 'kuery') { - const { fromKueryExpression, toKueryExpression, nodeTypes } = this.vis.API.kuery; - let newQuery; - - if (filterName === 'geo_bounding_box') { - newQuery = nodeTypes.function.buildNode('geoBoundingBox', field, _.mapKeys(filterData, (value, key) => _.camelCase(key))); - } - else if (filterName === 'geo_polygon') { - newQuery = nodeTypes.function.buildNode('geoPolygon', field, filterData.points); - } - else { - throw new Error(`Kuery does not support ${filterName} queries`); - } - - const allQueries = _.isEmpty(query.query) - ? [newQuery] - : [fromKueryExpression(query.query), newQuery]; - - this.vis.API.queryManager.setQuery({ - query: toKueryExpression(nodeTypes.function.buildNode('and', allQueries, 'implicit')), - language: 'kuery' - }); - } + this.vis.API.queryFilter.addFilters([filter]); this.vis.updateState(); } diff --git a/src/test_utils/public/stub_index_pattern.js b/src/test_utils/public/stub_index_pattern.js index daf1e1bcd82e3..8f366cdb4d334 100644 --- a/src/test_utils/public/stub_index_pattern.js +++ b/src/test_utils/public/stub_index_pattern.js @@ -16,6 +16,7 @@ export default function (Private) { function StubIndexPattern(pattern, timeField, fields) { this.id = pattern; + this.title = pattern; this.popularizeField = sinon.stub(); this.timeFieldName = timeField; this.getNonScriptedFields = sinon.spy(IndexPattern.prototype.getNonScriptedFields); diff --git a/src/ui/public/courier/data_source/build_query/__tests__/build_es_query.js b/src/ui/public/courier/data_source/build_query/__tests__/build_es_query.js index 3c684d829c703..018005918c1bb 100644 --- a/src/ui/public/courier/data_source/build_query/__tests__/build_es_query.js +++ b/src/ui/public/courier/data_source/build_query/__tests__/build_es_query.js @@ -36,7 +36,7 @@ describe('build query', function () { it('should combine queries and filters from multiple query languages into a single ES bool query', function () { const queries = [ - { query: 'foo:bar', language: 'kuery' }, + { query: 'extension:jpg', language: 'kuery' }, { query: 'bar:baz', language: 'lucene' }, ]; const filters = [ @@ -53,7 +53,7 @@ describe('build query', function () { { match_all: {} }, ], filter: [ - toElasticsearchQuery(fromKueryExpression('foo:bar'), indexPattern), + toElasticsearchQuery(fromKueryExpression('extension:jpg'), indexPattern), ], should: [], must_not: [], diff --git a/src/ui/public/courier/data_source/build_query/__tests__/from_kuery.js b/src/ui/public/courier/data_source/build_query/__tests__/from_kuery.js index 9b81423208e39..296f32300493d 100644 --- a/src/ui/public/courier/data_source/build_query/__tests__/from_kuery.js +++ b/src/ui/public/courier/data_source/build_query/__tests__/from_kuery.js @@ -2,6 +2,7 @@ import { buildQueryFromKuery } from '../from_kuery'; import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; import ngMock from 'ng_mock'; import { expectDeepEqual } from '../../../../../../test_utils/expect_deep_equal.js'; +import expect from 'expect.js'; import { fromKueryExpression, toElasticsearchQuery } from '../../../../kuery'; let indexPattern; @@ -28,8 +29,8 @@ describe('build query', function () { it('should transform an array of kuery queries into ES queries combined in the bool\'s filter clause', function () { const queries = [ - { query: 'foo:bar', language: 'kuery' }, - { query: 'bar:baz', language: 'kuery' }, + { query: 'extension:jpg', language: 'kuery' }, + { query: 'machine.os:osx', language: 'kuery' }, ]; const expectedESQueries = queries.map( @@ -43,6 +44,14 @@ describe('build query', function () { expectDeepEqual(result.filter, expectedESQueries); }); + it('should throw a useful error if it looks like query is using an old, unsupported syntax', function () { + const oldQuery = { query: 'is(foo, bar)', language: 'kuery' }; + + expect(buildQueryFromKuery).withArgs(indexPattern, [oldQuery]).to.throwError( + /It looks like you're using an outdated Kuery syntax./ + ); + }); + }); }); diff --git a/src/ui/public/courier/data_source/build_query/build_es_query.js b/src/ui/public/courier/data_source/build_query/build_es_query.js index 1026bed66221d..00e133268b3e7 100644 --- a/src/ui/public/courier/data_source/build_query/build_es_query.js +++ b/src/ui/public/courier/data_source/build_query/build_es_query.js @@ -1,6 +1,6 @@ import { groupBy, has } from 'lodash'; import { DecorateQueryProvider } from '../_decorate_query'; -import { buildQueryFromKuery, buildQueryFromKql } from './from_kuery'; +import { buildQueryFromKuery } from './from_kuery'; import { buildQueryFromFilters } from './from_filters'; import { buildQueryFromLucene } from './from_lucene'; @@ -17,16 +17,15 @@ export function BuildESQueryProvider(Private) { const queriesByLanguage = groupBy(validQueries, 'language'); const kueryQuery = buildQueryFromKuery(indexPattern, queriesByLanguage.kuery); - const kqlQuery = buildQueryFromKql(indexPattern, queriesByLanguage.kql); const luceneQuery = buildQueryFromLucene(queriesByLanguage.lucene, decorateQuery); const filterQuery = buildQueryFromFilters(filters, decorateQuery, indexPattern); return { bool: { - must: [].concat(kueryQuery.must, kqlQuery.must, luceneQuery.must, filterQuery.must), - filter: [].concat(kueryQuery.filter, kqlQuery.filter, luceneQuery.filter, filterQuery.filter), - should: [].concat(kueryQuery.should, kqlQuery.should, luceneQuery.should, filterQuery.should), - must_not: [].concat(kueryQuery.must_not, kqlQuery.must_not, luceneQuery.must_not, filterQuery.must_not), + must: [].concat(kueryQuery.must, luceneQuery.must, filterQuery.must), + filter: [].concat(kueryQuery.filter, luceneQuery.filter, filterQuery.filter), + should: [].concat(kueryQuery.should, luceneQuery.should, filterQuery.should), + must_not: [].concat(kueryQuery.must_not, luceneQuery.must_not, filterQuery.must_not), } }; } diff --git a/src/ui/public/courier/data_source/build_query/from_kuery.js b/src/ui/public/courier/data_source/build_query/from_kuery.js index b04f3d36d5c92..574e05486d2e4 100644 --- a/src/ui/public/courier/data_source/build_query/from_kuery.js +++ b/src/ui/public/courier/data_source/build_query/from_kuery.js @@ -1,13 +1,25 @@ -import _ from 'lodash'; -import { fromKueryExpression, fromKqlExpression, toElasticsearchQuery, nodeTypes } from '../../../kuery'; +import { fromLegacyKueryExpression, fromKueryExpression, toElasticsearchQuery, nodeTypes } from '../../../kuery'; +import { documentationLinks } from '../../../documentation_links'; -export function buildQueryFromKuery(indexPattern, queries) { - const queryASTs = _.map(queries, query => fromKueryExpression(query.query)); - return buildQuery(indexPattern, queryASTs); -} +const queryDocs = documentationLinks.query; -export function buildQueryFromKql(indexPattern, queries) { - const queryASTs = _.map(queries, query => fromKqlExpression(query.query)); +export function buildQueryFromKuery(indexPattern, queries = []) { + const queryASTs = queries.map((query) => { + try { + return fromKueryExpression(query.query); + } + catch (parseError) { + try { + fromLegacyKueryExpression(query.query); + } + catch (legacyParseError) { + throw parseError; + } + throw new Error( + `It looks like you're using an outdated Kuery syntax. See what changed in the [docs](${queryDocs.kueryQuerySyntax})!` + ); + } + }); return buildQuery(indexPattern, queryASTs); } diff --git a/src/ui/public/doc_table/__tests__/actions/filter.js b/src/ui/public/doc_table/__tests__/actions/filter.js index 7f001f969241f..26d9485c94e15 100644 --- a/src/ui/public/doc_table/__tests__/actions/filter.js +++ b/src/ui/public/doc_table/__tests__/actions/filter.js @@ -38,38 +38,6 @@ describe('doc table filter actions', function () { expect(filterManager.add.calledWith(...args)).to.be(true); }); - it('should add an operator style "is" function to kuery queries', function () { - const state = { - query: { query: '', language: 'kuery' } - }; - addFilter('foo', 'bar', '+', indexPattern, state, filterManager); - expect(state.query.query).to.be('"foo":"bar"'); - }); - - it('should combine the new clause with any existing query clauses using an implicit "and"', function () { - const state = { - query: { query: 'foo', language: 'kuery' } - }; - addFilter('foo', 'bar', '+', indexPattern, state, filterManager); - expect(state.query.query).to.be('foo "foo":"bar"'); - }); - - it('should support creation of negated clauses', function () { - const state = { - query: { query: 'foo', language: 'kuery' } - }; - addFilter('foo', 'bar', '-', indexPattern, state, filterManager); - expect(state.query.query).to.be('foo !"foo":"bar"'); - }); - - it('should add an exists query when the provided field name is "_exists_"', function () { - const state = { - query: { query: 'foo', language: 'kuery' } - }; - addFilter('_exists_', 'baz', '+', indexPattern, state, filterManager); - expect(state.query.query).to.be('foo exists("baz")'); - }); - }); diff --git a/src/ui/public/doc_table/actions/filter.js b/src/ui/public/doc_table/actions/filter.js index 2101f6d2d2431..b9049519e3767 100644 --- a/src/ui/public/doc_table/actions/filter.js +++ b/src/ui/public/doc_table/actions/filter.js @@ -1,36 +1,7 @@ -import _ from 'lodash'; -import { toKueryExpression, fromKueryExpression, nodeTypes } from 'ui/kuery'; - export function addFilter(field, values = [], operation, index, state, filterManager) { - const fieldName = _.isObject(field) ? field.name : field; - if (!Array.isArray(values)) { values = [values]; } - if (['lucene', 'kql'].includes(state.query.language)) { - filterManager.add(field, values, operation, index); - } - - if (state.query.language === 'kuery') { - const negate = operation === '-'; - const isExistsQuery = fieldName === '_exists_'; - - const newQueries = values.map((value) => { - const newQuery = isExistsQuery - ? nodeTypes.function.buildNode('exists', value) - : nodeTypes.function.buildNode('is', fieldName, value); - - return negate ? nodeTypes.function.buildNode('not', newQuery) : newQuery; - }); - - const allQueries = _.isEmpty(state.query.query) - ? newQueries - : [fromKueryExpression(state.query.query), ...newQueries]; - - state.query = { - query: toKueryExpression(nodeTypes.function.buildNode('and', allQueries, 'implicit')), - language: 'kuery' - }; - } + filterManager.add(field, values, operation, index); } diff --git a/src/ui/public/filter_bar/filter_bar_click_handler.js b/src/ui/public/filter_bar/filter_bar_click_handler.js index 84bde607c96bb..a1af664f1f02e 100644 --- a/src/ui/public/filter_bar/filter_bar_click_handler.js +++ b/src/ui/public/filter_bar/filter_bar_click_handler.js @@ -3,10 +3,8 @@ import { dedupFilters } from './lib/dedup_filters'; import { uniqFilters } from './lib/uniq_filters'; import { findByParam } from 'ui/utils/find_by_param'; import { toastNotifications } from 'ui/notify'; -import { AddFiltersToKueryProvider } from './lib/add_filters_to_kuery'; -export function FilterBarClickHandlerProvider(Private) { - const addFiltersToKuery = Private(AddFiltersToKueryProvider); +export function FilterBarClickHandlerProvider() { return function ($state) { return function (event, simulate) { @@ -63,17 +61,7 @@ export function FilterBarClickHandlerProvider(Private) { filters = dedupFilters($state.filters, uniqFilters(filters), { negate: true }); if (!simulate) { - if (['lucene', 'kql'].includes($state.query.language)) { - $state.$newFilters = filters; - } - else if ($state.query.language === 'kuery') { - addFiltersToKuery($state, filters) - .then(() => { - if (_.isFunction($state.save)) { - $state.save(); - } - }); - } + $state.$newFilters = filters; } return filters; } diff --git a/src/ui/public/filter_bar/lib/__tests__/add_filters_to_kuery.js b/src/ui/public/filter_bar/lib/__tests__/add_filters_to_kuery.js deleted file mode 100644 index 8c92cd852ae97..0000000000000 --- a/src/ui/public/filter_bar/lib/__tests__/add_filters_to_kuery.js +++ /dev/null @@ -1,94 +0,0 @@ -import { AddFiltersToKueryProvider } from '../add_filters_to_kuery'; -import { FilterManagerProvider } from 'ui/filter_manager'; -import NoDigestPromises from 'test_utils/no_digest_promises'; -import expect from 'expect.js'; -import ngMock from 'ng_mock'; -import sinon from 'sinon'; -import moment from 'moment'; - -describe('addFiltersToKuery', function () { - NoDigestPromises.activateForSuite(); - - let addFiltersToKuery; - let filterManager; - let timefilter; - - beforeEach(ngMock.module( - 'kibana', - 'kibana/courier', - function ($provide) { - $provide.service('courier', require('fixtures/mock_courier')); - } - )); - - beforeEach(ngMock.inject(function (Private, _timefilter_) { - timefilter = _timefilter_; - addFiltersToKuery = Private(AddFiltersToKueryProvider); - filterManager = Private(FilterManagerProvider); - sinon.stub(filterManager, 'add'); - })); - - - const filters = [{ - meta: { - index: 'logstash-*', - type: 'phrase', - key: 'machine.os', - params: { - query: 'osx' - }, - }, - query: { - match: { - 'machine.os': { - query: 'osx', - type: 'phrase' - } - } - } - }]; - - it('should return a Promise', function () { - const state = { - query: { query: '', language: 'lucene' } - }; - expect(addFiltersToKuery(state, filters)).to.be.a(Promise); - }); - - it('should add a query clause equivalent to the given filter', function () { - const state = { - query: { query: '', language: 'kuery' } - }; - return addFiltersToKuery(state, filters) - .then(() => { - expect(state.query.query).to.be('"machine.os":"osx"'); - }); - }); - - it('time field filters should update the global time filter instead of modifying the query', function () { - const startTime = moment('1995'); - const endTime = moment('1996'); - const state = { - query: { query: '', language: 'kuery' } - }; - const timestampFilter = { - meta: { - index: 'logstash-*', - }, - range: { - time: { - gt: startTime.valueOf(), - lt: endTime.valueOf(), - } - } - }; - return addFiltersToKuery(state, [timestampFilter]) - .then(() => { - expect(state.query.query).to.be(''); - expect(startTime.isSame(timefilter.time.from)).to.be(true); - expect(endTime.isSame(timefilter.time.to)).to.be(true); - }); - }); - - -}); diff --git a/src/ui/public/filter_bar/lib/add_filters_to_kuery.js b/src/ui/public/filter_bar/lib/add_filters_to_kuery.js deleted file mode 100644 index 13eb0e104a173..0000000000000 --- a/src/ui/public/filter_bar/lib/add_filters_to_kuery.js +++ /dev/null @@ -1,39 +0,0 @@ -import _ from 'lodash'; -import { FilterBarLibMapAndFlattenFiltersProvider } from 'ui/filter_bar/lib/map_and_flatten_filters'; -import { FilterBarLibExtractTimeFilterProvider } from 'ui/filter_bar/lib/extract_time_filter'; -import { FilterBarLibChangeTimeFilterProvider } from 'ui/filter_bar/lib/change_time_filter'; -import { FilterBarLibFilterOutTimeBasedFilterProvider } from 'ui/filter_bar/lib/filter_out_time_based_filter'; -import { toKueryExpression, fromKueryExpression, nodeTypes, filterToKueryAST } from 'ui/kuery'; - -export function AddFiltersToKueryProvider(Private) { - const mapAndFlattenFilters = Private(FilterBarLibMapAndFlattenFiltersProvider); - const extractTimeFilter = Private(FilterBarLibExtractTimeFilterProvider); - const changeTimeFilter = Private(FilterBarLibChangeTimeFilterProvider); - const filterOutTimeBasedFilter = Private(FilterBarLibFilterOutTimeBasedFilterProvider); - - return async function addFiltersToKuery(state, filters) { - return mapAndFlattenFilters(filters) - .then((results) => { - extractTimeFilter(results) - .then((timeFilter) => { - if (timeFilter) { - changeTimeFilter(timeFilter); - } - }); - return results; - }) - .then(filterOutTimeBasedFilter) - .then((results) => { - const newQueries = results.map(filterToKueryAST); - const allQueries = _.isEmpty(state.query.query) - ? newQueries - : [fromKueryExpression(state.query.query), ...newQueries]; - - state.query = { - query: toKueryExpression(nodeTypes.function.buildNode('and', allQueries, 'implicit')), - language: 'kuery' - }; - }); - - }; -} diff --git a/src/ui/public/kuery/ast/__tests__/ast.js b/src/ui/public/kuery/ast/__tests__/ast.js index fb782fa8af772..33810d0119923 100644 --- a/src/ui/public/kuery/ast/__tests__/ast.js +++ b/src/ui/public/kuery/ast/__tests__/ast.js @@ -8,8 +8,8 @@ import { expectDeepEqual } from '../../../../../test_utils/expect_deep_equal.js' // Helpful utility allowing us to test the PEG parser by simply checking for deep equality between // the nodes the parser generates and the nodes our constructor functions generate. -function fromKueryExpressionNoMeta(text) { - return ast.fromKueryExpression(text, { includeMetadata: false }); +function fromLegacyKueryExpressionNoMeta(text) { + return ast.fromLegacyKueryExpression(text, { includeMetadata: false }); } let indexPattern; @@ -21,10 +21,10 @@ describe('kuery AST API', function () { indexPattern = Private(StubbedLogstashIndexPatternProvider); })); - describe('fromKueryExpression', function () { + describe('fromLegacyKueryExpression', function () { it('should return location and text metadata for each AST node', function () { - const notNode = ast.fromKueryExpression('!foo:bar'); + const notNode = ast.fromLegacyKueryExpression('!foo:bar'); expect(notNode).to.have.property('text', '!foo:bar'); expect(notNode.location).to.eql({ min: 0, max: 8 }); @@ -42,19 +42,19 @@ describe('kuery AST API', function () { it('should return a match all "is" function for whitespace', function () { const expected = nodeTypes.function.buildNode('is', '*', '*'); - const actual = fromKueryExpressionNoMeta(' '); + const actual = fromLegacyKueryExpressionNoMeta(' '); expectDeepEqual(actual, expected); }); it('should return an "and" function for single literals', function () { - const expected = nodeTypes.function.buildNode('and', [nodeTypes.literal.buildNode('foo')], 'implicit'); - const actual = fromKueryExpressionNoMeta('foo'); + const expected = nodeTypes.function.buildNode('and', [nodeTypes.literal.buildNode('foo')]); + const actual = fromLegacyKueryExpressionNoMeta('foo'); expectDeepEqual(actual, expected); }); it('should ignore extraneous whitespace at the beginning and end of the query', function () { - const expected = nodeTypes.function.buildNode('and', [nodeTypes.literal.buildNode('foo')], 'implicit'); - const actual = fromKueryExpressionNoMeta(' foo '); + const expected = nodeTypes.function.buildNode('and', [nodeTypes.literal.buildNode('foo')]); + const actual = fromLegacyKueryExpressionNoMeta(' foo '); expectDeepEqual(actual, expected); }); @@ -62,8 +62,8 @@ describe('kuery AST API', function () { const expected = nodeTypes.function.buildNode('and', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'implicit'); - const actual = fromKueryExpressionNoMeta('foo bar'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('foo bar'); expectDeepEqual(actual, expected); }); @@ -71,8 +71,8 @@ describe('kuery AST API', function () { const expected = nodeTypes.function.buildNode('and', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'operator'); - const actual = fromKueryExpressionNoMeta('foo and bar'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('foo and bar'); expectDeepEqual(actual, expected); }); @@ -81,7 +81,7 @@ describe('kuery AST API', function () { nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), ], 'function'); - const actual = fromKueryExpressionNoMeta('and(foo, bar)'); + const actual = fromLegacyKueryExpressionNoMeta('and(foo, bar)'); expectDeepEqual(actual, expected); }); @@ -89,8 +89,8 @@ describe('kuery AST API', function () { const expected = nodeTypes.function.buildNode('or', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'operator'); - const actual = fromKueryExpressionNoMeta('foo or bar'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('foo or bar'); expectDeepEqual(actual, expected); }); @@ -98,8 +98,8 @@ describe('kuery AST API', function () { const expected = nodeTypes.function.buildNode('or', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'function'); - const actual = fromKueryExpressionNoMeta('or(foo, bar)'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('or(foo, bar)'); expectDeepEqual(actual, expected); }); @@ -108,8 +108,8 @@ describe('kuery AST API', function () { nodeTypes.function.buildNode('or', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'function'), 'operator'); - const actual = fromKueryExpressionNoMeta('!or(foo, bar)'); + ])); + const actual = fromLegacyKueryExpressionNoMeta('!or(foo, bar)'); expectDeepEqual(actual, expected); }); @@ -120,11 +120,11 @@ describe('kuery AST API', function () { nodeTypes.function.buildNode('and', [ nodeTypes.literal.buildNode('bar'), nodeTypes.literal.buildNode('baz'), - ], 'operator'), + ]), nodeTypes.literal.buildNode('qux'), ]) - ], 'operator'); - const actual = fromKueryExpressionNoMeta('foo or bar and baz or qux'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('foo or bar and baz or qux'); expectDeepEqual(actual, expected); }); @@ -133,16 +133,16 @@ describe('kuery AST API', function () { nodeTypes.function.buildNode('or', [ nodeTypes.literal.buildNode('foo'), nodeTypes.literal.buildNode('bar'), - ], 'operator'), + ]), nodeTypes.literal.buildNode('baz'), - ], 'operator'); - const actual = fromKueryExpressionNoMeta('(foo or bar) and baz'); + ]); + const actual = fromLegacyKueryExpressionNoMeta('(foo or bar) and baz'); expectDeepEqual(actual, expected); }); it('should support a shorthand operator syntax for "is" functions', function () { - const expected = nodeTypes.function.buildNode('is', 'foo', 'bar', 'operator'); - const actual = fromKueryExpressionNoMeta('foo:bar'); + const expected = nodeTypes.function.buildNode('is', 'foo', 'bar', true); + const actual = fromLegacyKueryExpressionNoMeta('foo:bar'); expectDeepEqual(actual, expected); }); @@ -152,43 +152,223 @@ describe('kuery AST API', function () { nodeTypes.literal.buildNode(1000), nodeTypes.literal.buildNode(8000), ]; - const expected = nodeTypes.function.buildNodeWithArgumentNodes('range', argumentNodes, 'operator'); - const actual = fromKueryExpressionNoMeta('bytes:[1000 to 8000]'); + const expected = nodeTypes.function.buildNodeWithArgumentNodes('range', argumentNodes); + const actual = fromLegacyKueryExpressionNoMeta('bytes:[1000 to 8000]'); expectDeepEqual(actual, expected); }); it('should support functions with named arguments', function () { - const expected = nodeTypes.function.buildNode('range', 'bytes', { gt: 1000, lt: 8000 }, 'function'); - const actual = fromKueryExpressionNoMeta('range(bytes, gt=1000, lt=8000)'); + const expected = nodeTypes.function.buildNode('range', 'bytes', { gt: 1000, lt: 8000 }); + const actual = fromLegacyKueryExpressionNoMeta('range(bytes, gt=1000, lt=8000)'); expectDeepEqual(actual, expected); }); it('should throw an error for unknown functions', function () { - expect(ast.fromKueryExpression).withArgs('foo(bar)').to.throwException(/Unknown function "foo"/); + expect(ast.fromLegacyKueryExpression).withArgs('foo(bar)').to.throwException(/Unknown function "foo"/); }); }); - describe('toKueryExpression', function () { + describe('fromKueryExpression', function () { - it('should return the given node type\'s kuery string representation', function () { - const node = nodeTypes.function.buildNode('exists', 'foo'); - const expected = nodeTypes.function.toKueryExpression(node); - const result = ast.toKueryExpression(node); - expectDeepEqual(result, expected); + it('should return a match all "is" function for whitespace', function () { + const expected = nodeTypes.function.buildNode('is', '*', '*'); + const actual = ast.fromKueryExpression(' '); + expectDeepEqual(actual, expected); }); - it('should return an empty string for undefined nodes and unknown node types', function () { - expect(ast.toKueryExpression()).to.be(''); + it('should return an "is" function with a null field for single literals', function () { + const expected = nodeTypes.function.buildNode('is', null, 'foo'); + const actual = ast.fromKueryExpression('foo'); + expectDeepEqual(actual, expected); + }); - const noTypeNode = nodeTypes.function.buildNode('exists', 'foo'); - delete noTypeNode.type; - expect(ast.toKueryExpression(noTypeNode)).to.be(''); + it('should ignore extraneous whitespace at the beginning and end of the query', function () { + const expected = nodeTypes.function.buildNode('is', null, 'foo'); + const actual = ast.fromKueryExpression(' foo '); + expectDeepEqual(actual, expected); + }); - const unknownTypeNode = nodeTypes.function.buildNode('exists', 'foo'); - unknownTypeNode.type = 'notValid'; - expect(ast.toKueryExpression(unknownTypeNode)).to.be(''); + it('should not split on whitespace', function () { + const expected = nodeTypes.function.buildNode('is', null, 'foo bar'); + const actual = ast.fromKueryExpression('foo bar'); + expectDeepEqual(actual, expected); + }); + + it('should support "and" as a binary operator', function () { + const expected = nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('is', null, 'foo'), + nodeTypes.function.buildNode('is', null, 'bar'), + ]); + const actual = ast.fromKueryExpression('foo and bar'); + expectDeepEqual(actual, expected); + }); + + it('should support "or" as a binary operator', function () { + const expected = nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', null, 'foo'), + nodeTypes.function.buildNode('is', null, 'bar'), + ]); + const actual = ast.fromKueryExpression('foo or bar'); + expectDeepEqual(actual, expected); + }); + + it('should support negation of queries with a "not" prefix', function () { + const expected = nodeTypes.function.buildNode('not', + nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', null, 'foo'), + nodeTypes.function.buildNode('is', null, 'bar'), + ]) + ); + const actual = ast.fromKueryExpression('not (foo or bar)'); + expectDeepEqual(actual, expected); }); + it('"and" should have a higher precedence than "or"', function () { + const expected = nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', null, 'foo'), + nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('is', null, 'bar'), + nodeTypes.function.buildNode('is', null, 'baz'), + ]), + nodeTypes.function.buildNode('is', null, 'qux'), + ]) + ]); + const actual = ast.fromKueryExpression('foo or bar and baz or qux'); + expectDeepEqual(actual, expected); + }); + + it('should support grouping to override default precedence', function () { + const expected = nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', null, 'foo'), + nodeTypes.function.buildNode('is', null, 'bar'), + ]), + nodeTypes.function.buildNode('is', null, 'baz'), + ]); + const actual = ast.fromKueryExpression('(foo or bar) and baz'); + expectDeepEqual(actual, expected); + }); + + it('should support matching against specific fields', function () { + const expected = nodeTypes.function.buildNode('is', 'foo', 'bar'); + const actual = ast.fromKueryExpression('foo:bar'); + expectDeepEqual(actual, expected); + }); + + it('should also not split on whitespace when matching specific fields', function () { + const expected = nodeTypes.function.buildNode('is', 'foo', 'bar baz'); + const actual = ast.fromKueryExpression('foo:bar baz'); + expectDeepEqual(actual, expected); + }); + + it('should treat quoted values as phrases', function () { + const expected = nodeTypes.function.buildNode('is', 'foo', 'bar baz', true); + const actual = ast.fromKueryExpression('foo:"bar baz"'); + expectDeepEqual(actual, expected); + }); + + it('should support a shorthand for matching multiple values against a single field', function () { + const expected = nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', 'foo', 'bar'), + nodeTypes.function.buildNode('is', 'foo', 'baz'), + ]); + const actual = ast.fromKueryExpression('foo:(bar or baz)'); + expectDeepEqual(actual, expected); + }); + + it('should support "and" and "not" operators and grouping in the shorthand as well', function () { + const expected = nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('or', [ + nodeTypes.function.buildNode('is', 'foo', 'bar'), + nodeTypes.function.buildNode('is', 'foo', 'baz'), + ]), + nodeTypes.function.buildNode('not', + nodeTypes.function.buildNode('is', 'foo', 'qux') + ), + ]); + const actual = ast.fromKueryExpression('foo:((bar or baz) and not qux)'); + expectDeepEqual(actual, expected); + }); + + it('should support exclusive range operators', function () { + const expected = nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('range', 'bytes', { + gt: 1000, + }), + nodeTypes.function.buildNode('range', 'bytes', { + lt: 8000, + }), + ]); + const actual = ast.fromKueryExpression('bytes > 1000 and bytes < 8000'); + expectDeepEqual(actual, expected); + }); + + it('should support inclusive range operators', function () { + const expected = nodeTypes.function.buildNode('and', [ + nodeTypes.function.buildNode('range', 'bytes', { + gte: 1000, + }), + nodeTypes.function.buildNode('range', 'bytes', { + lte: 8000, + }), + ]); + const actual = ast.fromKueryExpression('bytes >= 1000 and bytes <= 8000'); + expectDeepEqual(actual, expected); + }); + + it('should support wildcards in field names', function () { + const expected = nodeTypes.function.buildNode('is', 'machine*', 'osx'); + const actual = ast.fromKueryExpression('machine*:osx'); + expectDeepEqual(actual, expected); + }); + + it('should support wildcards in values', function () { + const expected = nodeTypes.function.buildNode('is', 'foo', 'ba*'); + const actual = ast.fromKueryExpression('foo:ba*'); + expectDeepEqual(actual, expected); + }); + + it('should create an exists "is" query when a field is given and "*" is the value', function () { + const expected = nodeTypes.function.buildNode('is', 'foo', '*'); + const actual = ast.fromKueryExpression('foo:*'); + expectDeepEqual(actual, expected); + }); + + }); + + describe('fromLiteralExpression', function () { + + it('should create literal nodes for unquoted values with correct primitive types', function () { + const stringLiteral = nodeTypes.literal.buildNode('foo'); + const booleanFalseLiteral = nodeTypes.literal.buildNode(false); + const booleanTrueLiteral = nodeTypes.literal.buildNode(true); + const numberLiteral = nodeTypes.literal.buildNode(42); + + expectDeepEqual(ast.fromLiteralExpression('foo'), stringLiteral); + expectDeepEqual(ast.fromLiteralExpression('true'), booleanTrueLiteral); + expectDeepEqual(ast.fromLiteralExpression('false'), booleanFalseLiteral); + expectDeepEqual(ast.fromLiteralExpression('42'), numberLiteral); + }); + + it('should allow escaping of special characters with a backslash', function () { + const expected = nodeTypes.literal.buildNode('\\():<>"*'); + // yo dawg + const actual = ast.fromLiteralExpression('\\\\\\(\\)\\:\\<\\>\\"\\*'); + expectDeepEqual(actual, expected); + }); + + it('should support double quoted strings that do not need escapes except for quotes', function () { + const expected = nodeTypes.literal.buildNode('\\():<>"*'); + const actual = ast.fromLiteralExpression('"\\():<>\\"*"'); + expectDeepEqual(actual, expected); + }); + + it('should detect wildcards and build wildcard AST nodes', function () { + const expected = nodeTypes.wildcard.buildNode('foo*bar'); + const actual = ast.fromLiteralExpression('foo*bar'); + expectDeepEqual(actual, expected); + }); }); describe('toElasticsearchQuery', function () { @@ -216,38 +396,4 @@ describe('kuery AST API', function () { }); - describe('symmetry of to/fromKueryExpression', function () { - - it('toKueryExpression and fromKueryExpression should be inverse operations', function () { - function testExpression(expression) { - expect(ast.toKueryExpression(ast.fromKueryExpression(expression))).to.be(expression); - } - - testExpression(''); - testExpression(' '); - testExpression('foo'); - testExpression('foo bar'); - testExpression('foo 200'); - testExpression('bytes:[1000 to 8000]'); - testExpression('bytes:[1000 TO 8000]'); - testExpression('range(bytes, gt=1000, lt=8000)'); - testExpression('range(bytes, gt=1000, lte=8000)'); - testExpression('range(bytes, gte=1000, lt=8000)'); - testExpression('range(bytes, gte=1000, lte=8000)'); - testExpression('response:200'); - testExpression('"response":200'); - testExpression('response:"200"'); - testExpression('"response":"200"'); - testExpression('is(response, 200)'); - testExpression('!is(response, 200)'); - testExpression('foo or is(tic, tock) or foo:bar'); - testExpression('or(foo, is(tic, tock), foo:bar)'); - testExpression('foo is(tic, tock) foo:bar'); - testExpression('foo and is(tic, tock) and foo:bar'); - testExpression('(foo or is(tic, tock)) and foo:bar'); - testExpression('!(foo or is(tic, tock)) and foo:bar'); - }); - - }); - }); diff --git a/src/ui/public/kuery/ast/ast.js b/src/ui/public/kuery/ast/ast.js index c9ef74327f4b9..c553bfe930457 100644 --- a/src/ui/public/kuery/ast/ast.js +++ b/src/ui/public/kuery/ast/ast.js @@ -1,21 +1,32 @@ -import grammar from 'raw-loader!./kuery.peg'; -import kqlGrammar from 'raw-loader!./kql.peg'; +import legacyKueryGrammar from 'raw-loader!./legacy_kuery.peg'; +import kueryGrammar from 'raw-loader!./kuery.peg'; import PEG from 'pegjs'; import _ from 'lodash'; import { nodeTypes } from '../node_types/index'; -const kueryParser = PEG.buildParser(grammar); -const kqlParser = PEG.buildParser(kqlGrammar); +const legacyKueryParser = PEG.buildParser(legacyKueryGrammar); +const kueryParser = PEG.buildParser(kueryGrammar, { + allowedStartRules: ['start', 'Literal'], +}); + +export function fromLiteralExpression(expression, parseOptions) { + parseOptions = { + ...parseOptions, + startRule: 'Literal', + }; -export function fromKueryExpression(expression, parseOptions) { return fromExpression(expression, parseOptions, kueryParser); } -export function fromKqlExpression(expression, parseOptions) { - return fromExpression(expression, parseOptions, kqlParser); +export function fromLegacyKueryExpression(expression, parseOptions) { + return fromExpression(expression, parseOptions, legacyKueryParser); +} + +export function fromKueryExpression(expression, parseOptions) { + return fromExpression(expression, parseOptions, kueryParser); } -function fromExpression(expression, parseOptions = {}, parser = kqlParser) { +function fromExpression(expression, parseOptions = {}, parser = kueryParser) { if (_.isUndefined(expression)) { throw new Error('expression must be a string, got undefined instead'); } @@ -28,14 +39,6 @@ function fromExpression(expression, parseOptions = {}, parser = kqlParser) { return parser.parse(expression, parseOptions); } -export function toKueryExpression(node) { - if (!node || !node.type || !nodeTypes[node.type]) { - return ''; - } - - return nodeTypes[node.type].toKueryExpression(node); -} - export function toElasticsearchQuery(node, indexPattern) { if (!node || !node.type || !nodeTypes[node.type]) { return toElasticsearchQuery(nodeTypes.function.buildNode('and', [])); diff --git a/src/ui/public/kuery/ast/index.js b/src/ui/public/kuery/ast/index.js index 57eaebe354fe8..441c20ee0f5d6 100644 --- a/src/ui/public/kuery/ast/index.js +++ b/src/ui/public/kuery/ast/index.js @@ -1 +1 @@ -export { fromKueryExpression, fromKqlExpression, toKueryExpression, toElasticsearchQuery } from './ast'; +export { fromLegacyKueryExpression, fromKueryExpression, fromLiteralExpression, toElasticsearchQuery } from './ast'; diff --git a/src/ui/public/kuery/ast/kuery.peg b/src/ui/public/kuery/ast/kuery.peg index 8dc5411b854bf..40a7ee676a054 100644 --- a/src/ui/public/kuery/ast/kuery.peg +++ b/src/ui/public/kuery/ast/kuery.peg @@ -1,150 +1,179 @@ -/* - * Kuery parser - */ - -/* - * Initialization block - */ +// Initialization block { - var nodeTypes = options.helpers.nodeTypes; - - if (options.includeMetadata === undefined) { - options.includeMetadata = true; - } - - function addMeta(source, text, location) { - if (options.includeMetadata) { - return Object.assign( - {}, - source, - { - text: text, - location: simpleLocation(location), - } - ); - } - - return source; + const { nodeTypes } = options.helpers; + const buildFunctionNode = nodeTypes.function.buildNodeWithArgumentNodes; + const buildLiteralNode = nodeTypes.literal.buildNode; + const buildWildcardNode = nodeTypes.wildcard.buildNode; + const buildNamedArgNode = nodeTypes.namedArg.buildNode; + + function trimLeft(string) { + return string.replace(/^[\s\uFEFF\xA0]+/g, ''); } - function simpleLocation(location) { - // Returns an object representing the position of the function within the expression, - // demarcated by the position of its first character and last character. We calculate these values - // using the offset because the expression could span multiple lines, and we don't want to deal - // with column and line values. - return { - min: location.start.offset, - max: location.end.offset - } + function trimRight(string) { + return string.replace(/[\s\uFEFF\xA0]+$/g, ''); } } start - = space? query:OrQuery space? { - if (query.type === 'literal') { - return addMeta(nodeTypes.function.buildNode('and', [query], 'implicit'), text(), location()); - } - return query; - } - / whitespace:[\ \t\r\n]* { - return addMeta(nodeTypes.function.buildNode('is', '*', '*'), text(), location()); + = Space* query:OrQuery? Space* { + if (query !== null) return query; + return nodeTypes.function.buildNode('is', '*', '*'); } OrQuery - = left:AndQuery space 'or'i space right:OrQuery { - return addMeta(nodeTypes.function.buildNode('or', [left, right], 'operator'), text(), location()); + = left:AndQuery Or right:OrQuery { + return buildFunctionNode('or', [left, right]); } / AndQuery AndQuery - = left:NegatedClause space 'and'i space right:AndQuery { - return addMeta(nodeTypes.function.buildNode('and', [left, right], 'operator'), text(), location()); + = left:NotQuery And right:AndQuery{ + return buildFunctionNode('and', [left, right]); } - / left:NegatedClause space !'or'i right:AndQuery { - return addMeta(nodeTypes.function.buildNode('and', [left, right], 'implicit'), text(), location()); + / NotQuery + +NotQuery + = Not query:SubQuery { + return buildFunctionNode('not', [query]); } - / NegatedClause + / SubQuery -NegatedClause - = [!] clause:Clause { - return addMeta(nodeTypes.function.buildNode('not', clause, 'operator'), text(), location()); +SubQuery + = '(' Space* query:OrQuery Space* ')' { return query; } + / Expression + +Expression + = FieldRangeExpression + / FieldValueExpression + / ValueExpression + +FieldRangeExpression + = field:Literal Space* operator:RangeOperator Space* value:(QuotedString / UnquotedLiteral) { + const range = buildNamedArgNode(operator, value); + return buildFunctionNode('range', [field, range]); } - / Clause -Clause - = '(' subQuery:start ')' { - return subQuery; +FieldValueExpression + = field:Literal Space* ':' Space* partial:ListOfValues { + return partial(field); } - / Term -Term - = field:literal_arg_type ':' value:literal_arg_type { - return addMeta(nodeTypes.function.buildNodeWithArgumentNodes('is', [field, value], 'operator'), text(), location()); +ValueExpression + = partial:Value { + const field = buildLiteralNode(null); + return partial(field); } - / field:literal_arg_type ':[' space? gt:literal_arg_type space 'to'i space lt:literal_arg_type space? ']' { - return addMeta(nodeTypes.function.buildNodeWithArgumentNodes('range', [field, gt, lt], 'operator'), text(), location()); + +ListOfValues + = '(' Space* partial:OrListOfValues Space* ')' { return partial; } + / Value + +OrListOfValues + = partialLeft:AndListOfValues Or partialRight:OrListOfValues { + return (field) => buildFunctionNode('or', [partialLeft(field), partialRight(field)]); } - / function - / !Keywords literal:literal_arg_type { return literal; } + / AndListOfValues -function_name - = first:[a-zA-Z]+ rest:[.a-zA-Z0-9_-]* { return first.join('') + rest.join('') } +AndListOfValues + = partialLeft:NotListOfValues And partialRight:AndListOfValues { + return (field) => buildFunctionNode('and', [partialLeft(field), partialRight(field)]); + } + / NotListOfValues -function "function" - = name:function_name space? '(' space? arg_list:arg_list? space? ')' { - return addMeta(nodeTypes.function.buildNodeWithArgumentNodes(name, arg_list || [], 'function'), text(), location()); - } +NotListOfValues + = Not partial:ListOfValues { + return (field) => buildFunctionNode('not', [partial(field)]); + } + / ListOfValues -arg_list - = first:argument rest:(space? ',' space? arg:argument {return arg})* space? ','? { - return [first].concat(rest); - } +Value + = value:QuotedString { + const isPhrase = buildLiteralNode(true); + return (field) => buildFunctionNode('is', [field, value, isPhrase]); + } + / value:WildcardString { + const isPhrase = buildLiteralNode(false); + return (field) => buildFunctionNode('is', [field, value, isPhrase]); + } + / value:UnquotedLiteral { + const isPhrase = buildLiteralNode(false); + return (field) => buildFunctionNode('is', [field, value, isPhrase]); + } -argument - = name:function_name space? '=' space? value:arg_type { - return addMeta(nodeTypes.namedArg.buildNode(name, value), text(), location()); - } - / element:arg_type {return element} +Or + = Space+ 'or'i Space+ -arg_type - = OrQuery - / literal_arg_type +And + = Space+ 'and'i Space+ -literal_arg_type - = literal:literal { - var result = addMeta(nodeTypes.literal.buildNode(literal), text(), location()); - return result; - } +Not + = 'not'i Space+ -Keywords - = 'and'i / 'or'i +Literal + = QuotedString / WildcardString / UnquotedLiteral + +QuotedString + = '"' chars:(EscapedDoubleQuote / [^"])* '"' { + return buildLiteralNode(chars.join('')); + } - /* ----- Core types ----- */ +WildcardString + = sequences:WildcardSequence+ { + const compactedSequences = sequences.reduce((acc, arr, i) => { + const compacted = arr.filter(value => value !== ''); + return [...acc, ...compacted]; + }, []); + if (typeof compactedSequences[0] === 'string') { + compactedSequences[0] = trimLeft(compactedSequences[0]); + } + const lastIndex = compactedSequences.length - 1; + if (typeof compactedSequences[lastIndex] === 'string') { + compactedSequences[lastIndex] = trimRight(compactedSequences[lastIndex]); + } + return buildWildcardNode(compactedSequences); + } -literal "literal" - = '"' chars:dq_char* '"' { return chars.join(''); } // double quoted string - / "'" chars:sq_char* "'" { return chars.join(''); } // single quoted string - / 'true' { return true; } // unquoted literals from here down - / 'false' { return false; } - / 'null' { return null; } - / string:[^\[\]()"',:=\ \t]+ { // this also matches numbers via Number() - var result = string.join(''); - // Sort of hacky, but PEG doesn't have backtracking so - // a number rule is hard to read, and performs worse - if (isNaN(Number(result))) return result; - return Number(result) +WildcardSequence + = left:UnquotedCharacter* '*' right:UnquotedCharacter* { + return [left.join(''), nodeTypes.wildcard.wildcardSymbol, right.join('')]; } -space - = [\ \t\r\n]+ +UnquotedLiteral + = chars:UnquotedCharacter+ { + const sequence = chars.join('').trim(); + if (sequence === 'null') return buildLiteralNode(null); + if (sequence === 'true') return buildLiteralNode(true); + if (sequence === 'false') return buildLiteralNode(false); + const number = Number(sequence); + const value = isNaN(number) ? sequence : number; + return buildLiteralNode(value); + } + +UnquotedCharacter + = EscapedSpecialCharacter + / !Separator char:. { return char; } + +EscapedSpecialCharacter + = '\\' char:SpecialCharacter { return char; } + +EscapedDoubleQuote + = '\\' char:'"' { return char; } + +Separator + = Keyword / SpecialCharacter + +Keyword + = Or / And / Not -dq_char - = "\\" sequence:('"' / "\\") { return sequence; } - / [^"] // everything except " +SpecialCharacter + = [\\():<>"*] -sq_char - = "\\" sequence:("'" / "\\") { return sequence; } - / [^'] // everything except ' +RangeOperator + = '<=' { return 'lte'; } + / '>=' { return 'gte'; } + / '<' { return 'lt'; } + / '>' { return 'gt'; } -integer - = digits:[0-9]+ {return parseInt(digits.join(''))} +Space + = [\ \t\r\n] diff --git a/src/ui/public/kuery/ast/kql.peg b/src/ui/public/kuery/ast/legacy_kuery.peg similarity index 64% rename from src/ui/public/kuery/ast/kql.peg rename to src/ui/public/kuery/ast/legacy_kuery.peg index eb1c6034a366d..08f7a592f43fd 100644 --- a/src/ui/public/kuery/ast/kql.peg +++ b/src/ui/public/kuery/ast/legacy_kuery.peg @@ -40,18 +40,15 @@ } start - = Query - / space* { - return addMeta(nodeTypes.function.buildNode('and', []), text(), location()); - } - -Query = space? query:OrQuery space? { if (query.type === 'literal') { return addMeta(nodeTypes.function.buildNode('and', [query]), text(), location()); } return query; } + / whitespace:[\ \t\r\n]* { + return addMeta(nodeTypes.function.buildNode('is', '*', '*', false), text(), location()); + } OrQuery = left:AndQuery space 'or'i space right:OrQuery { @@ -60,40 +57,67 @@ OrQuery / AndQuery AndQuery - = left:NotQuery space 'and'i space right:AndQuery { + = left:NegatedClause space 'and'i space right:AndQuery { return addMeta(nodeTypes.function.buildNode('and', [left, right]), text(), location()); } - / NotQuery + / left:NegatedClause space !'or'i right:AndQuery { + return addMeta(nodeTypes.function.buildNode('and', [left, right]), text(), location()); + } + / NegatedClause -NotQuery - = 'not'i space clause:Clause { +NegatedClause + = [!] clause:Clause { return addMeta(nodeTypes.function.buildNode('not', clause), text(), location()); } / Clause Clause - = '(' subQuery:Query ')' { + = '(' subQuery:start ')' { return subQuery; } / Term Term - = field:literal_arg_type space? ':' space? value:literal_arg_type { - return addMeta(nodeTypes.function.buildNodeWithArgumentNodes('is', [field, value]), text(), location()); + = field:literal_arg_type ':' value:literal_arg_type { + return addMeta(nodeTypes.function.buildNodeWithArgumentNodes('is', [field, value, nodeTypes.literal.buildNode(true)]), text(), location()); } - / field:literal_arg_type space? ':' space? '[' space? gt:literal_arg_type space 'to'i space lt:literal_arg_type space? ']' { + / field:literal_arg_type ':[' space? gt:literal_arg_type space 'to'i space lt:literal_arg_type space? ']' { return addMeta(nodeTypes.function.buildNodeWithArgumentNodes('range', [field, gt, lt]), text(), location()); } + / function / !Keywords literal:literal_arg_type { return literal; } +function_name + = first:[a-zA-Z]+ rest:[.a-zA-Z0-9_-]* { return first.join('') + rest.join('') } + +function "function" + = name:function_name space? '(' space? arg_list:arg_list? space? ')' { + return addMeta(nodeTypes.function.buildNodeWithArgumentNodes(name, arg_list || []), text(), location()); + } + +arg_list + = first:argument rest:(space? ',' space? arg:argument {return arg})* space? ','? { + return [first].concat(rest); + } + +argument + = name:function_name space? '=' space? value:arg_type { + return addMeta(nodeTypes.namedArg.buildNode(name, value), text(), location()); + } + / element:arg_type {return element} + +arg_type + = OrQuery + / literal_arg_type + literal_arg_type = literal:literal { - var result = addMeta(nodeTypes.literal.buildNode(literal), text(), location()); - return result; + var result = addMeta(nodeTypes.literal.buildNode(literal), text(), location()); + return result; } Keywords - = 'or'i / 'and'i / 'not'i + = 'and'i / 'or'i /* ----- Core types ----- */ @@ -121,3 +145,6 @@ dq_char sq_char = "\\" sequence:("'" / "\\") { return sequence; } / [^'] // everything except ' + +integer + = digits:[0-9]+ {return parseInt(digits.join(''))} diff --git a/src/ui/public/kuery/functions/__tests__/and.js b/src/ui/public/kuery/functions/__tests__/and.js index ad4af11c2bde2..4417da2a2edc6 100644 --- a/src/ui/public/kuery/functions/__tests__/and.js +++ b/src/ui/public/kuery/functions/__tests__/and.js @@ -4,11 +4,10 @@ import { nodeTypes } from '../../node_types'; import * as ast from '../../ast'; import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; import ngMock from 'ng_mock'; -import { expectDeepEqual } from '../../../../../test_utils/expect_deep_equal'; let indexPattern; -const childNode1 = nodeTypes.function.buildNode('is', 'response', 200); +const childNode1 = nodeTypes.function.buildNode('is', 'machine.os', 'osx'); const childNode2 = nodeTypes.function.buildNode('is', 'extension', 'jpg'); describe('kuery functions', function () { @@ -22,11 +21,6 @@ describe('kuery functions', function () { describe('buildNodeParams', function () { - it('should return "arguments" and "serializeStyle" params', function () { - const result = and.buildNodeParams([childNode1, childNode2]); - expect(result).to.only.have.keys('arguments', 'serializeStyle'); - }); - it('arguments should contain the unmodified child nodes', function () { const result = and.buildNodeParams([childNode1, childNode2]); const { arguments: [ actualChildNode1, actualChildNode2 ] } = result; @@ -34,11 +28,6 @@ describe('kuery functions', function () { expect(actualChildNode2).to.be(childNode2); }); - it('serializeStyle should default to "operator"', function () { - const { serializeStyle } = and.buildNodeParams([childNode1, childNode2]); - expect(serializeStyle).to.be('operator'); - }); - }); describe('toElasticsearchQuery', function () { @@ -53,46 +42,7 @@ describe('kuery functions', function () { ); }); - it('should wrap a literal argument with an "is" function targeting the default_field', function () { - const literalFoo = nodeTypes.literal.buildNode('foo'); - const expectedChild = ast.toElasticsearchQuery(nodeTypes.function.buildNode('is', null, 'foo'), indexPattern); - const node = nodeTypes.function.buildNode('and', [literalFoo]); - const result = and.toElasticsearchQuery(node, indexPattern); - const resultChild = result.bool.filter[0]; - expectDeepEqual(resultChild, expectedChild); - }); - }); - describe('toKueryExpression', function () { - - it('should serialize "and" nodes with an implicit syntax when requested', function () { - const node = nodeTypes.function.buildNode('and', [childNode1, childNode2], 'implicit'); - const result = and.toKueryExpression(node); - expect(result).to.be('"response":200 "extension":"jpg"'); - }); - - it('should serialize "and" nodes with an operator syntax when requested', function () { - const node = nodeTypes.function.buildNode('and', [childNode1, childNode2], 'operator'); - const result = and.toKueryExpression(node); - expect(result).to.be('"response":200 and "extension":"jpg"'); - }); - - it('should wrap "or" sub-queries in parenthesis', function () { - const orNode = nodeTypes.function.buildNode('or', [childNode1, childNode2], 'operator'); - const fooBarNode = nodeTypes.function.buildNode('is', 'foo', 'bar'); - const andNode = nodeTypes.function.buildNode('and', [orNode, fooBarNode], 'implicit'); - - const result = and.toKueryExpression(andNode); - expect(result).to.be('("response":200 or "extension":"jpg") "foo":"bar"'); - }); - - it('should throw an error for nodes with unknown or undefined serialize styles', function () { - const node = nodeTypes.function.buildNode('and', [childNode1, childNode2], 'notValid'); - expect(and.toKueryExpression) - .withArgs(node).to.throwException(/Cannot serialize "and" function as "notValid"/); - }); - - }); }); }); diff --git a/src/ui/public/kuery/functions/__tests__/is.js b/src/ui/public/kuery/functions/__tests__/is.js index 11684bedb951b..88fc58ee92ae2 100644 --- a/src/ui/public/kuery/functions/__tests__/is.js +++ b/src/ui/public/kuery/functions/__tests__/is.js @@ -23,13 +23,8 @@ describe('kuery functions', function () { expect(is.buildNodeParams).withArgs('foo').to.throwException(/value is a required argument/); }); - it('should return "arguments" and "serializeStyle" params', function () { - const result = is.buildNodeParams('response', 200); - expect(result).to.only.have.keys('arguments', 'serializeStyle'); - }); - it('arguments should contain the provided fieldName and value as literals', function () { - const { arguments: [ fieldName, value ] } = is.buildNodeParams('response', 200); + const { arguments: [fieldName, value] } = is.buildNodeParams('response', 200); expect(fieldName).to.have.property('type', 'literal'); expect(fieldName).to.have.property('value', 'response'); @@ -38,11 +33,22 @@ describe('kuery functions', function () { expect(value).to.have.property('value', 200); }); - it('serializeStyle should default to "operator"', function () { - const { serializeStyle } = is.buildNodeParams('response', 200); - expect(serializeStyle).to.be('operator'); + it('should detect wildcards in the provided arguments', function () { + const { arguments: [fieldName, value] } = is.buildNodeParams('machine*', 'win*'); + + expect(fieldName).to.have.property('type', 'wildcard'); + expect(value).to.have.property('type', 'wildcard'); + }); + + it('should default to a non-phrase query', function () { + const { arguments: [, , isPhrase] } = is.buildNodeParams('response', 200); + expect(isPhrase.value).to.be(false); }); + it('should allow specification of a phrase query', function () { + const { arguments: [, , isPhrase] } = is.buildNodeParams('response', 200, true); + expect(isPhrase.value).to.be(true); + }); }); describe('toElasticsearchQuery', function () { @@ -61,7 +67,7 @@ describe('kuery functions', function () { const expected = { multi_match: { query: 200, - type: 'phrase', + type: 'best_fields', lenient: true, } }; @@ -71,63 +77,82 @@ describe('kuery functions', function () { expectDeepEqual(result, expected); }); - it('should return an ES multi_match query when fieldName is "*"', function () { - const expected = { - multi_match: { - query: 200, - fields: ['*'], - type: 'phrase', - lenient: true, - } - }; - + it('should return an ES bool query with a sub-query for each field when fieldName is "*"', function () { const node = nodeTypes.function.buildNode('is', '*', 200); const result = is.toElasticsearchQuery(node, indexPattern); - expectDeepEqual(result, expected); + expect(result).to.have.property('bool'); + expect(result.bool.should).to.have.length(indexPattern.fields.length); }); it('should return an ES exists query when value is "*"', function () { const expected = { - exists: { field: 'response' } + bool: { + should: [ + { exists: { field: 'extension' } }, + ], + minimum_should_match: 1 + } }; - const node = nodeTypes.function.buildNode('is', 'response', '*'); + const node = nodeTypes.function.buildNode('is', 'extension', '*'); const result = is.toElasticsearchQuery(node, indexPattern); expectDeepEqual(result, expected); }); - it('should return an ES match_phrase query when a concrete fieldName and value are provided', function () { + it('should return an ES match query when a concrete fieldName and value are provided', function () { const expected = { - match_phrase: { - response: 200 + bool: { + should: [ + { match: { extension: 'jpg' } }, + ], + minimum_should_match: 1 } }; - const node = nodeTypes.function.buildNode('is', 'response', 200); + const node = nodeTypes.function.buildNode('is', 'extension', 'jpg'); const result = is.toElasticsearchQuery(node, indexPattern); expectDeepEqual(result, expected); }); - it('should support scripted fields', function () { - const node = nodeTypes.function.buildNode('is', 'script string', 'foo'); + it('should support creation of phrase queries', function () { + const expected = { + bool: { + should: [ + { match_phrase: { extension: 'jpg' } }, + ], + minimum_should_match: 1 + } + }; + + const node = nodeTypes.function.buildNode('is', 'extension', 'jpg', true); const result = is.toElasticsearchQuery(node, indexPattern); - expect(result).to.have.key('script'); + expectDeepEqual(result, expected); }); - }); - - describe('toKueryExpression', function () { + it('should create a query_string query for wildcard values', function () { + const expected = { + bool: { + should: [ + { + query_string: { + fields: ['extension'], + query: 'jpg*' + } + }, + ], + minimum_should_match: 1 + } + }; - it('should serialize "is" nodes with an operator syntax', function () { - const node = nodeTypes.function.buildNode('is', 'response', 200, 'operator'); - const result = is.toKueryExpression(node); - expect(result).to.be('"response":200'); + const node = nodeTypes.function.buildNode('is', 'extension', 'jpg*'); + const result = is.toElasticsearchQuery(node, indexPattern); + expectDeepEqual(result, expected); }); - it('should throw an error for nodes with unknown or undefined serialize styles', function () { - const node = nodeTypes.function.buildNode('is', 'response', 200, 'notValid'); - expect(is.toKueryExpression) - .withArgs(node).to.throwException(/Cannot serialize "is" function as "notValid"/); + it('should support scripted fields', function () { + const node = nodeTypes.function.buildNode('is', 'script string', 'foo'); + const result = is.toElasticsearchQuery(node, indexPattern); + expect(result.bool.should[0]).to.have.key('script'); }); }); diff --git a/src/ui/public/kuery/functions/__tests__/not.js b/src/ui/public/kuery/functions/__tests__/not.js index 2a7e8838e9813..35cf0940d432d 100644 --- a/src/ui/public/kuery/functions/__tests__/not.js +++ b/src/ui/public/kuery/functions/__tests__/not.js @@ -4,11 +4,10 @@ import { nodeTypes } from '../../node_types'; import * as ast from '../../ast'; import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; import ngMock from 'ng_mock'; -import { expectDeepEqual } from '../../../../../test_utils/expect_deep_equal'; let indexPattern; -const childNode = nodeTypes.function.buildNode('is', 'response', 200); +const childNode = nodeTypes.function.buildNode('is', 'extension', 'jpg'); describe('kuery functions', function () { @@ -21,20 +20,11 @@ describe('kuery functions', function () { describe('buildNodeParams', function () { - it('should return "arguments" and "serializeStyle" params', function () { - const result = not.buildNodeParams(childNode); - expect(result).to.only.have.keys('arguments', 'serializeStyle'); - }); - it('arguments should contain the unmodified child node', function () { const { arguments: [ actualChild ] } = not.buildNodeParams(childNode); expect(actualChild).to.be(childNode); }); - it('serializeStyle should default to "operator"', function () { - const { serializeStyle } = not.buildNodeParams(childNode); - expect(serializeStyle).to.be('operator'); - }); }); @@ -47,52 +37,6 @@ describe('kuery functions', function () { expect(result.bool).to.only.have.keys('must_not'); expect(result.bool.must_not).to.eql(ast.toElasticsearchQuery(childNode, indexPattern)); }); - - it('should wrap a literal argument with an "is" function targeting the default_field', function () { - const literalFoo = nodeTypes.literal.buildNode('foo'); - const expectedChild = ast.toElasticsearchQuery(nodeTypes.function.buildNode('is', null, 'foo'), indexPattern); - const node = nodeTypes.function.buildNode('not', literalFoo); - const result = not.toElasticsearchQuery(node, indexPattern); - const resultChild = result.bool.must_not; - expectDeepEqual(resultChild, expectedChild); - }); - - }); - - describe('toKueryExpression', function () { - - it('should serialize "not" nodes with an operator syntax', function () { - const node = nodeTypes.function.buildNode('not', childNode, 'operator'); - const result = not.toKueryExpression(node); - expect(result).to.be('!"response":200'); - }); - - it('should wrap "and" and "or" sub-queries in parenthesis', function () { - const andNode = nodeTypes.function.buildNode('and', [childNode, childNode], 'operator'); - const notAndNode = nodeTypes.function.buildNode('not', andNode, 'operator'); - expect(not.toKueryExpression(notAndNode)).to.be('!("response":200 and "response":200)'); - - const orNode = nodeTypes.function.buildNode('or', [childNode, childNode], 'operator'); - const notOrNode = nodeTypes.function.buildNode('not', orNode, 'operator'); - expect(not.toKueryExpression(notOrNode)).to.be('!("response":200 or "response":200)'); - }); - - it('should not wrap "and" and "or" sub-queries that use the function syntax', function () { - const andNode = nodeTypes.function.buildNode('and', [childNode, childNode], 'function'); - const notAndNode = nodeTypes.function.buildNode('not', andNode, 'operator'); - expect(not.toKueryExpression(notAndNode)).to.be('!and("response":200, "response":200)'); - - const orNode = nodeTypes.function.buildNode('or', [childNode, childNode], 'function'); - const notOrNode = nodeTypes.function.buildNode('not', orNode, 'operator'); - expect(not.toKueryExpression(notOrNode)).to.be('!or("response":200, "response":200)'); - }); - - it('should throw an error for nodes with unknown or undefined serialize styles', function () { - const node = nodeTypes.function.buildNode('not', childNode, 'notValid'); - expect(not.toKueryExpression) - .withArgs(node).to.throwException(/Cannot serialize "not" function as "notValid"/); - }); - }); }); }); diff --git a/src/ui/public/kuery/functions/__tests__/or.js b/src/ui/public/kuery/functions/__tests__/or.js index 60125e984b1a0..5e71e358a367e 100644 --- a/src/ui/public/kuery/functions/__tests__/or.js +++ b/src/ui/public/kuery/functions/__tests__/or.js @@ -4,11 +4,10 @@ import { nodeTypes } from '../../node_types'; import * as ast from '../../ast'; import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; import ngMock from 'ng_mock'; -import { expectDeepEqual } from '../../../../../test_utils/expect_deep_equal'; let indexPattern; -const childNode1 = nodeTypes.function.buildNode('is', 'response', 200); +const childNode1 = nodeTypes.function.buildNode('is', 'machine.os', 'osx'); const childNode2 = nodeTypes.function.buildNode('is', 'extension', 'jpg'); describe('kuery functions', function () { @@ -22,11 +21,6 @@ describe('kuery functions', function () { describe('buildNodeParams', function () { - it('should return "arguments" and "serializeStyle" params', function () { - const result = or.buildNodeParams([childNode1, childNode2]); - expect(result).to.only.have.keys('arguments', 'serializeStyle'); - }); - it('arguments should contain the unmodified child nodes', function () { const result = or.buildNodeParams([childNode1, childNode2]); const { arguments: [ actualChildNode1, actualChildNode2 ] } = result; @@ -34,11 +28,6 @@ describe('kuery functions', function () { expect(actualChildNode2).to.be(childNode2); }); - it('serializeStyle should default to "operator"', function () { - const { serializeStyle } = or.buildNodeParams([childNode1, childNode2]); - expect(serializeStyle).to.be('operator'); - }); - }); describe('toElasticsearchQuery', function () { @@ -53,15 +42,6 @@ describe('kuery functions', function () { ); }); - it('should wrap a literal argument with an "is" function targeting the default_field', function () { - const literalFoo = nodeTypes.literal.buildNode('foo'); - const expectedChild = ast.toElasticsearchQuery(nodeTypes.function.buildNode('is', null, 'foo'), indexPattern); - const node = nodeTypes.function.buildNode('or', [literalFoo]); - const result = or.toElasticsearchQuery(node, indexPattern); - const resultChild = result.bool.should[0]; - expectDeepEqual(resultChild, expectedChild); - }); - it('should require one of the clauses to match', function () { const node = nodeTypes.function.buildNode('or', [childNode1, childNode2]); const result = or.toElasticsearchQuery(node, indexPattern); @@ -70,20 +50,5 @@ describe('kuery functions', function () { }); - describe('toKueryExpression', function () { - - it('should serialize "or" nodes with an operator syntax', function () { - const node = nodeTypes.function.buildNode('or', [childNode1, childNode2]); - const result = or.toKueryExpression(node); - expect(result).to.be('"response":200 or "extension":"jpg"'); - }); - - it('should throw an error for nodes with unknown or undefined serialize styles', function () { - const node = nodeTypes.function.buildNode('or', [childNode1, childNode2], 'notValid'); - expect(or.toKueryExpression) - .withArgs(node).to.throwException(/Cannot serialize "or" function as "notValid"/); - }); - - }); }); }); diff --git a/src/ui/public/kuery/functions/__tests__/range.js b/src/ui/public/kuery/functions/__tests__/range.js index ef6afc1062384..dad6557208cd9 100644 --- a/src/ui/public/kuery/functions/__tests__/range.js +++ b/src/ui/public/kuery/functions/__tests__/range.js @@ -1,7 +1,7 @@ import expect from 'expect.js'; +import { expectDeepEqual } from '../../../../../test_utils/expect_deep_equal'; import * as range from '../range'; import { nodeTypes } from '../../node_types'; -import _ from 'lodash'; import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; import ngMock from 'ng_mock'; @@ -18,14 +18,9 @@ describe('kuery functions', function () { describe('buildNodeParams', function () { - it('should return "arguments" and "serializeStyle" params', function () { - const result = range.buildNodeParams('bytes', { gt: 1000, lt: 8000 }); - expect(result).to.only.have.keys('arguments', 'serializeStyle'); - }); - it('arguments should contain the provided fieldName as a literal', function () { const result = range.buildNodeParams('bytes', { gt: 1000, lt: 8000 }); - const { arguments: [ fieldName ] } = result; + const { arguments: [fieldName] } = result; expect(fieldName).to.have.property('type', 'literal'); expect(fieldName).to.have.property('value', 'bytes'); @@ -34,7 +29,7 @@ describe('kuery functions', function () { it('arguments should contain the provided params as named arguments', function () { const givenParams = { gt: 1000, lt: 8000, format: 'epoch_millis' }; const result = range.buildNodeParams('bytes', givenParams); - const { arguments: [ , ...params ] } = result; + const { arguments: [, ...params] } = result; expect(params).to.be.an('array'); expect(params).to.not.be.empty(); @@ -47,64 +42,58 @@ describe('kuery functions', function () { }); }); - it('serializeStyle should default to "operator"', function () { - const result = range.buildNodeParams('bytes', { gte: 1000, lte: 8000 }); - const { serializeStyle } = result; - expect(serializeStyle).to.be('operator'); - }); - - it('serializeStyle should be "function" if either end of the range is exclusive', function () { - const result = range.buildNodeParams('bytes', { gt: 1000, lt: 8000 }); - const { serializeStyle } = result; - expect(serializeStyle).to.be('function'); - }); - }); describe('toElasticsearchQuery', function () { it('should return an ES range query for the node\'s field and params', function () { const expected = { - range: { - bytes: { - gt: 1000, - lt: 8000 - } + bool: { + should: [ + { + range: { + bytes: { + gt: 1000, + lt: 8000 + } + } + } + ], + minimum_should_match: 1 } }; const node = nodeTypes.function.buildNode('range', 'bytes', { gt: 1000, lt: 8000 }); const result = range.toElasticsearchQuery(node, indexPattern); - expect(_.isEqual(expected, result)).to.be(true); - }); - - it('should support scripted fields', function () { - const node = nodeTypes.function.buildNode('range', 'script number', { gt: 1000, lt: 8000 }); - const result = range.toElasticsearchQuery(node, indexPattern); - expect(result).to.have.key('script'); + expectDeepEqual(result, expected); }); - }); - - describe('toKueryExpression', function () { - - it('should serialize "range" nodes with an operator syntax', function () { - const node = nodeTypes.function.buildNode('range', 'bytes', { gte: 1000, lte: 8000 }, 'operator'); - const result = range.toKueryExpression(node); - expect(result).to.be('"bytes":[1000 to 8000]'); - }); + it('should support wildcard field names', function () { + const expected = { + bool: { + should: [ + { + range: { + bytes: { + gt: 1000, + lt: 8000 + } + } + } + ], + minimum_should_match: 1 + } + }; - it('should throw an error for nodes with unknown or undefined serialize styles', function () { - const node = nodeTypes.function.buildNode('range', 'bytes', { gte: 1000, lte: 8000 }, 'notValid'); - expect(range.toKueryExpression) - .withArgs(node).to.throwException(/Cannot serialize "range" function as "notValid"/); + const node = nodeTypes.function.buildNode('range', 'byt*', { gt: 1000, lt: 8000 }); + const result = range.toElasticsearchQuery(node, indexPattern); + expectDeepEqual(result, expected); }); - it('should not support exclusive ranges in the operator syntax', function () { - const node = nodeTypes.function.buildNode('range', 'bytes', { gt: 1000, lt: 8000 }); - node.serializeStyle = 'operator'; - expect(range.toKueryExpression) - .withArgs(node).to.throwException(/Operator syntax only supports inclusive ranges/); + it('should support scripted fields', function () { + const node = nodeTypes.function.buildNode('range', 'script number', { gt: 1000, lt: 8000 }); + const result = range.toElasticsearchQuery(node, indexPattern); + expect(result.bool.should[0]).to.have.key('script'); }); }); diff --git a/src/ui/public/kuery/functions/__tests__/utils/get_fields.js b/src/ui/public/kuery/functions/__tests__/utils/get_fields.js new file mode 100644 index 0000000000000..e5179ff79a6da --- /dev/null +++ b/src/ui/public/kuery/functions/__tests__/utils/get_fields.js @@ -0,0 +1,81 @@ +import { getFields } from '../../utils/get_fields'; +import expect from 'expect.js'; +import StubbedLogstashIndexPatternProvider from 'fixtures/stubbed_logstash_index_pattern'; +import ngMock from 'ng_mock'; +import { nodeTypes } from 'ui/kuery'; + +let indexPattern; + +describe('getFields', function () { + + beforeEach(ngMock.module('kibana')); + beforeEach(ngMock.inject(function (Private) { + indexPattern = Private(StubbedLogstashIndexPatternProvider); + })); + + describe('field names without a wildcard', function () { + + it('should thrown an error if the field does not exist in the index pattern', function () { + const fieldNameNode = nodeTypes.literal.buildNode('nonExistentField'); + expect(getFields).withArgs(fieldNameNode, indexPattern).to.throwException( + /Field nonExistentField does not exist in index pattern logstash-\*/ + ); + }); + + it('should return the single matching field in an array', function () { + const fieldNameNode = nodeTypes.literal.buildNode('extension'); + const results = getFields(fieldNameNode, indexPattern); + expect(results).to.be.an('array'); + expect(results).to.have.length(1); + expect(results[0].name).to.be('extension'); + }); + + it('should not match a wildcard in a literal node', function () { + const indexPatternWithWildField = { + title: 'wildIndex', + fields: { + byName: { + 'foo*': { + name: 'foo*' + } + } + } + }; + + const fieldNameNode = nodeTypes.literal.buildNode('foo*'); + const results = getFields(fieldNameNode, indexPatternWithWildField); + expect(results).to.be.an('array'); + expect(results).to.have.length(1); + expect(results[0].name).to.be('foo*'); + + // ensure the wildcard is not actually being parsed + expect(getFields).withArgs(nodeTypes.literal.buildNode('fo*'), indexPatternWithWildField).to.throwException( + /Field fo\* does not exist in index pattern wildIndex/ + ); + + }); + }); + + describe('field name patterns with a wildcard', function () { + + it('should thrown an error if the pattern does not match any fields in the index pattern', function () { + const fieldNameNode = nodeTypes.wildcard.buildNode('nonExistent*'); + expect(getFields).withArgs(fieldNameNode, indexPattern).to.throwException( + /No fields match the pattern nonExistent\* in index pattern logstash-\*/ + ); + }); + + it('should return all fields that match the pattern in an array', function () { + const fieldNameNode = nodeTypes.wildcard.buildNode('machine*'); + const results = getFields(fieldNameNode, indexPattern); + expect(results).to.be.an('array'); + expect(results).to.have.length(2); + expect(results.find((field) => { + return field.name === 'machine.os'; + })).to.be.ok(); + expect(results.find((field) => { + return field.name === 'machine.os.raw'; + })).to.be.ok(); + }); + }); +}); diff --git a/src/ui/public/kuery/functions/and.js b/src/ui/public/kuery/functions/and.js index 2b6efcc4e4c05..9da645d092ca6 100644 --- a/src/ui/public/kuery/functions/and.js +++ b/src/ui/public/kuery/functions/and.js @@ -1,10 +1,8 @@ import * as ast from '../ast'; -import { nodeTypes } from '../node_types'; -export function buildNodeParams(children, serializeStyle = 'operator') { +export function buildNodeParams(children) { return { arguments: children, - serializeStyle }; } @@ -14,33 +12,9 @@ export function toElasticsearchQuery(node, indexPattern) { return { bool: { filter: children.map((child) => { - if (child.type === 'literal') { - child = nodeTypes.function.buildNode('is', null, child.value); - } - return ast.toElasticsearchQuery(child, indexPattern); }) } }; } -export function toKueryExpression(node) { - if (!['operator', 'implicit'].includes(node.serializeStyle)) { - throw new Error(`Cannot serialize "and" function as "${node.serializeStyle}"`); - } - - const queryStrings = (node.arguments || []).map((arg) => { - const query = ast.toKueryExpression(arg); - if (arg.type === 'function' && arg.function === 'or') { - return `(${query})`; - } - return query; - }); - - if (node.serializeStyle === 'implicit') { - return queryStrings.join(' '); - } - if (node.serializeStyle === 'operator') { - return queryStrings.join(' and '); - } -} diff --git a/src/ui/public/kuery/functions/is.js b/src/ui/public/kuery/functions/is.js index 32ef1d6ac2d1d..3f46b0af42808 100644 --- a/src/ui/public/kuery/functions/is.js +++ b/src/ui/public/kuery/functions/is.js @@ -1,8 +1,11 @@ import _ from 'lodash'; +import * as ast from '../ast'; import * as literal from '../node_types/literal'; +import * as wildcard from '../node_types/wildcard'; import { getPhraseScript } from 'ui/filter_manager/lib/phrase'; +import { getFields } from './utils/get_fields'; -export function buildNodeParams(fieldName, value, serializeStyle = 'operator') { +export function buildNodeParams(fieldName, value, isPhrase = false) { if (_.isUndefined(fieldName)) { throw new Error('fieldName is a required argument'); } @@ -10,69 +13,80 @@ export function buildNodeParams(fieldName, value, serializeStyle = 'operator') { throw new Error('value is a required argument'); } + const fieldNode = typeof fieldName === 'string' ? ast.fromLiteralExpression(fieldName) : literal.buildNode(fieldName); + const valueNode = typeof value === 'string' ? ast.fromLiteralExpression(value) : literal.buildNode(value); + const isPhraseNode = literal.buildNode(isPhrase); + return { - arguments: [literal.buildNode(fieldName), literal.buildNode(value)], - serializeStyle + arguments: [fieldNode, valueNode, isPhraseNode], }; } export function toElasticsearchQuery(node, indexPattern) { - const { arguments: [ fieldNameArg, valueArg ] } = node; - const fieldName = literal.toElasticsearchQuery(fieldNameArg); - const field = indexPattern.fields.byName[fieldName]; - const value = !_.isUndefined(valueArg) ? literal.toElasticsearchQuery(valueArg) : valueArg; + const { arguments: [ fieldNameArg, valueArg, isPhraseArg ] } = node; - if (field && field.scripted) { - return { - script: { - ...getPhraseScript(field, value) - } - }; - } - else if (fieldName === null) { - return { - multi_match: { - query: value, - type: 'phrase', - lenient: true, - } - }; - } - else if (fieldName === '*' && value === '*') { - return { match_all: {} }; - } - else if (fieldName === '*' && value !== '*') { + const value = !_.isUndefined(valueArg) ? ast.toElasticsearchQuery(valueArg) : valueArg; + const type = isPhraseArg.value ? 'phrase' : 'best_fields'; + + if (fieldNameArg.value === null) { return { multi_match: { + type, query: value, - fields: ['*'], - type: 'phrase', lenient: true, } }; } - else if (fieldName !== '*' && value === '*') { - return { - exists: { field: fieldName } - }; - } - else { - return { - match_phrase: { - [fieldName]: value - } - }; - } -} -export function toKueryExpression(node) { - if (node.serializeStyle !== 'operator') { - throw new Error(`Cannot serialize "is" function as "${node.serializeStyle}"`); + const fields = getFields(fieldNameArg, indexPattern); + const isExistsQuery = valueArg.type === 'wildcard' && value === '*'; + const isMatchAllQuery = isExistsQuery && fields && fields.length === indexPattern.fields.length; + + if (isMatchAllQuery) { + return { match_all: {} }; } - const { arguments: [ fieldNameArg, valueArg ] } = node; - const fieldName = literal.toKueryExpression(fieldNameArg); - const value = !_.isUndefined(valueArg) ? literal.toKueryExpression(valueArg) : valueArg; + const queries = fields.reduce((accumulator, field) => { + if (field.scripted) { + // Exists queries don't make sense for scripted fields + if (!isExistsQuery) { + return [...accumulator, { + script: { + ...getPhraseScript(field, value) + } + }]; + } + } + else if (isExistsQuery) { + return [...accumulator, { + exists: { + field: field.name + } + }]; + } + else if (valueArg.type === 'wildcard') { + return [...accumulator, { + query_string: { + fields: [field.name], + query: wildcard.toQueryStringQuery(valueArg), + } + }]; + } + else { + const queryType = type === 'phrase' ? 'match_phrase' : 'match'; + return [...accumulator, { + [queryType]: { + [field.name]: value + } + }]; + } + }, []); - return `${fieldName}:${value}`; + return { + bool: { + should: queries, + minimum_should_match: 1 + } + }; } + diff --git a/src/ui/public/kuery/functions/not.js b/src/ui/public/kuery/functions/not.js index 3448ed6441a67..2819d7d4b8e37 100644 --- a/src/ui/public/kuery/functions/not.js +++ b/src/ui/public/kuery/functions/not.js @@ -1,18 +1,13 @@ import * as ast from '../ast'; -import { nodeTypes } from '../node_types'; -export function buildNodeParams(child, serializeStyle = 'operator') { +export function buildNodeParams(child) { return { arguments: [child], - serializeStyle }; } export function toElasticsearchQuery(node, indexPattern) { - let [ argument ] = node.arguments; - if (argument.type === 'literal') { - argument = nodeTypes.function.buildNode('is', null, argument.value); - } + const [ argument ] = node.arguments; return { bool: { @@ -21,22 +16,3 @@ export function toElasticsearchQuery(node, indexPattern) { }; } -export function toKueryExpression(node) { - if (node.serializeStyle !== 'operator') { - throw new Error(`Cannot serialize "not" function as "${node.serializeStyle}"`); - } - - const [ argument ] = node.arguments; - const queryString = ast.toKueryExpression(argument); - - if ( - argument.function && - (argument.function === 'and' || argument.function === 'or') && - argument.serializeStyle !== 'function' - ) { - return `!(${queryString})`; - } - else { - return `!${queryString}`; - } -} diff --git a/src/ui/public/kuery/functions/or.js b/src/ui/public/kuery/functions/or.js index b00ff3831409c..c27e163a46f52 100644 --- a/src/ui/public/kuery/functions/or.js +++ b/src/ui/public/kuery/functions/or.js @@ -1,10 +1,8 @@ import * as ast from '../ast'; -import { nodeTypes } from '../node_types'; -export function buildNodeParams(children, serializeStyle = 'operator') { +export function buildNodeParams(children) { return { arguments: children, - serializeStyle, }; } @@ -14,26 +12,9 @@ export function toElasticsearchQuery(node, indexPattern) { return { bool: { should: children.map((child) => { - if (child.type === 'literal') { - child = nodeTypes.function.buildNode('is', null, child.value); - } - return ast.toElasticsearchQuery(child, indexPattern); }), minimum_should_match: 1, }, }; } - -export function toKueryExpression(node) { - if (node.serializeStyle !== 'operator') { - throw new Error(`Cannot serialize "or" function as "${node.serializeStyle}"`); - } - - const queryStrings = (node.arguments || []).map((arg) => { - return ast.toKueryExpression(arg); - }); - - return queryStrings.join(' or '); -} - diff --git a/src/ui/public/kuery/functions/range.js b/src/ui/public/kuery/functions/range.js index d673f6d8fe8de..a6e5a160b13cf 100644 --- a/src/ui/public/kuery/functions/range.js +++ b/src/ui/public/kuery/functions/range.js @@ -2,62 +2,50 @@ import _ from 'lodash'; import { nodeTypes } from '../node_types'; import * as ast from '../ast'; import { getRangeScript } from 'ui/filter_manager/lib/range'; +import { getFields } from './utils/get_fields'; -export function buildNodeParams(fieldName, params, serializeStyle = 'operator') { +export function buildNodeParams(fieldName, params) { params = _.pick(params, 'gt', 'lt', 'gte', 'lte', 'format'); - const fieldNameArg = nodeTypes.literal.buildNode(fieldName); + const fieldNameArg = typeof fieldName === 'string' ? ast.fromLiteralExpression(fieldName) : nodeTypes.literal.buildNode(fieldName); const args = _.map(params, (value, key) => { return nodeTypes.namedArg.buildNode(key, value); }); - // we only support inclusive ranges in the operator syntax currently - if (_.has(params, 'gt') || _.has(params, 'lt')) { - serializeStyle = 'function'; - } - return { arguments: [fieldNameArg, ...args], - serializeStyle, }; } export function toElasticsearchQuery(node, indexPattern) { const [ fieldNameArg, ...args ] = node.arguments; - const fieldName = nodeTypes.literal.toElasticsearchQuery(fieldNameArg); - const field = indexPattern.fields.byName[fieldName]; + const fields = getFields(fieldNameArg, indexPattern); const namedArgs = extractArguments(args); const queryParams = _.mapValues(namedArgs, ast.toElasticsearchQuery); - if (field && field.scripted) { + const queries = fields.map((field) => { + if (field.scripted) { + return { + script: { + ...getRangeScript(field, queryParams) + } + }; + } + return { - script: { - ...getRangeScript(field, queryParams) + range: { + [field.name]: queryParams } }; - } + }); return { - range: { - [fieldName]: queryParams + bool: { + should: queries, + minimum_should_match: 1 } }; } -export function toKueryExpression(node) { - if (node.serializeStyle !== 'operator') { - throw new Error(`Cannot serialize "range" function as "${node.serializeStyle}"`); - } - const [ fieldNameArg, ...args ] = node.arguments; - const fieldName = ast.toKueryExpression(fieldNameArg); - const { gte, lte } = extractArguments(args); - - if (_.isUndefined(gte) || _.isUndefined(lte)) { - throw new Error(`Operator syntax only supports inclusive ranges`); - } - - return `${fieldName}:[${ast.toKueryExpression(gte)} to ${ast.toKueryExpression(lte)}]`; -} - function extractArguments(args) { if ((args.gt && args.gte) || (args.lt && args.lte)) { throw new Error('range ends cannot be both inclusive and exclusive'); diff --git a/src/ui/public/kuery/functions/utils/get_fields.js b/src/ui/public/kuery/functions/utils/get_fields.js new file mode 100644 index 0000000000000..9a8ff16315f26 --- /dev/null +++ b/src/ui/public/kuery/functions/utils/get_fields.js @@ -0,0 +1,19 @@ +import * as literal from '../../node_types/literal'; +import * as wildcard from '../../node_types/wildcard'; + +export function getFields(node, indexPattern) { + if (node.type === 'literal') { + const fieldName = literal.toElasticsearchQuery(node); + const field = indexPattern.fields.byName[fieldName]; + if (!field) { + throw new Error(`Field ${fieldName} does not exist in index pattern ${indexPattern.title}`); + } + return [field]; + } else if (node.type === 'wildcard') { + const fields = indexPattern.fields.filter(field => wildcard.test(node, field.name)); + if (fields.length === 0) { + throw new Error(`No fields match the pattern ${wildcard.toElasticsearchQuery(node)} in index pattern ${indexPattern.title}`); + } + return fields; + } +} diff --git a/src/ui/public/kuery/node_types/__tests__/function.js b/src/ui/public/kuery/node_types/__tests__/function.js index dc69dd67b91b5..fafb8b4ffa9e0 100644 --- a/src/ui/public/kuery/node_types/__tests__/function.js +++ b/src/ui/public/kuery/node_types/__tests__/function.js @@ -21,7 +21,7 @@ describe('kuery node types', function () { describe('buildNode', function () { it('should return a node representing the given kuery function', function () { - const result = functionType.buildNode('is', 'response', 200); + const result = functionType.buildNode('is', 'extension', 'jpg'); expect(result).to.have.property('type', 'function'); expect(result).to.have.property('function', 'is'); expect(result).to.have.property('arguments'); @@ -32,8 +32,8 @@ describe('kuery node types', function () { describe('buildNodeWithArgumentNodes', function () { it('should return a function node with the given argument list untouched', function () { - const fieldNameLiteral = nodeTypes.literal.buildNode('response'); - const valueLiteral = nodeTypes.literal.buildNode(200); + const fieldNameLiteral = nodeTypes.literal.buildNode('extension'); + const valueLiteral = nodeTypes.literal.buildNode('jpg'); const argumentNodes = [fieldNameLiteral, valueLiteral]; const result = functionType.buildNodeWithArgumentNodes('is', argumentNodes); @@ -49,7 +49,7 @@ describe('kuery node types', function () { describe('toElasticsearchQuery', function () { it('should return the given function type\'s ES query representation', function () { - const node = functionType.buildNode('is', 'response', 200); + const node = functionType.buildNode('is', 'extension', 'jpg'); const expected = isFunction.toElasticsearchQuery(node, indexPattern); const result = functionType.toElasticsearchQuery(node, indexPattern); expect(_.isEqual(expected, result)).to.be(true); @@ -57,32 +57,6 @@ describe('kuery node types', function () { }); - describe('toKueryExpression', function () { - - it('should return the function syntax representation of the given node by default', function () { - const node = functionType.buildNode('exists', 'foo'); - expect(functionType.toKueryExpression(node)).to.be('exists("foo")'); - }); - - it('should return the function syntax representation of the given node if serializeStyle is "function"', function () { - const node = functionType.buildNode('exists', 'foo'); - node.serializeStyle = 'function'; - expect(functionType.toKueryExpression(node)).to.be('exists("foo")'); - }); - - it('should defer to the function\'s serializer if another serializeStyle is specified', function () { - const node = functionType.buildNode('is', 'response', 200); - expect(node.serializeStyle).to.be('operator'); - expect(functionType.toKueryExpression(node)).to.be('"response":200'); - }); - - it('should simply return the node\'s "text" property if one exists', function () { - const node = functionType.buildNode('exists', 'foo'); - node.text = 'bar'; - expect(functionType.toKueryExpression(node)).to.be('bar'); - }); - - }); }); diff --git a/src/ui/public/kuery/node_types/__tests__/literal.js b/src/ui/public/kuery/node_types/__tests__/literal.js index 8e288f4e1ce9d..19ea086fd6f7b 100644 --- a/src/ui/public/kuery/node_types/__tests__/literal.js +++ b/src/ui/public/kuery/node_types/__tests__/literal.js @@ -25,19 +25,6 @@ describe('kuery node types', function () { }); - describe('toKueryExpression', function () { - - it('should return the literal value represented by the given node', function () { - const numberNode = literal.buildNode(200); - expect(literal.toKueryExpression(numberNode)).to.be(200); - }); - - it('should wrap string values in double quotes', function () { - const stringNode = literal.buildNode('foo'); - expect(literal.toKueryExpression(stringNode)).to.be('"foo"'); - }); - - }); }); diff --git a/src/ui/public/kuery/node_types/__tests__/named_arg.js b/src/ui/public/kuery/node_types/__tests__/named_arg.js index fb7a7cab19044..8102faf6136c1 100644 --- a/src/ui/public/kuery/node_types/__tests__/named_arg.js +++ b/src/ui/public/kuery/node_types/__tests__/named_arg.js @@ -39,15 +39,6 @@ describe('kuery node types', function () { }); - describe('toKueryExpression', function () { - - it('should return the argument name and value represented by the given node', function () { - const node = namedArg.buildNode('fieldName', 'foo'); - expect(namedArg.toKueryExpression(node)).to.be('fieldName="foo"'); - }); - - }); - }); }); diff --git a/src/ui/public/kuery/node_types/__tests__/wildcard.js b/src/ui/public/kuery/node_types/__tests__/wildcard.js new file mode 100644 index 0000000000000..bca02976851d1 --- /dev/null +++ b/src/ui/public/kuery/node_types/__tests__/wildcard.js @@ -0,0 +1,78 @@ +import expect from 'expect.js'; +import * as wildcard from '../wildcard'; + +describe('kuery node types', function () { + + describe('wildcard', function () { + + describe('buildNode', function () { + + it('should accept an array argument representing a wildcard string', function () { + const wildcardValue = [ + 'foo', + Symbol('*'), + 'bar', + ]; + + const result = wildcard.buildNode(wildcardValue); + expect(result).to.have.property('type', 'wildcard'); + expect(result).to.have.property('value', wildcardValue); + }); + + it('should accept and parse a wildcard string', function () { + const result = wildcard.buildNode('foo*bar'); + expect(result).to.have.property('type', 'wildcard'); + + expect(result.value[0]).to.be('foo'); + + expect(result.value[1]).to.be.a('symbol'); + expect(result.value[1].toString()).to.be('Symbol(*)'); + + expect(result.value[2]).to.be('bar'); + }); + + }); + + describe('toElasticsearchQuery', function () { + + it('should return the string representation of the wildcard literal', function () { + const node = wildcard.buildNode('foo*bar'); + const result = wildcard.toElasticsearchQuery(node); + expect(result).to.be('foo*bar'); + }); + + }); + + describe('toQueryStringQuery', function () { + + it('should return the string representation of the wildcard literal', function () { + const node = wildcard.buildNode('foo*bar'); + const result = wildcard.toQueryStringQuery(node); + expect(result).to.be('foo*bar'); + }); + + it('should escape query_string query special characters other than wildcard', function () { + const node = wildcard.buildNode('+foo*bar'); + const result = wildcard.toQueryStringQuery(node); + expect(result).to.be('\\+foo*bar'); + }); + + }); + + describe('test', function () { + + it('should return a boolean indicating whether the string matches the given wildcard node', function () { + const node = wildcard.buildNode('foo*bar'); + expect(wildcard.test(node, 'foobar')).to.be(true); + expect(wildcard.test(node, 'foobazbar')).to.be(true); + expect(wildcard.test(node, 'foobar')).to.be(true); + + expect(wildcard.test(node, 'fooqux')).to.be(false); + expect(wildcard.test(node, 'bazbar')).to.be(false); + }); + + }); + + }); + +}); diff --git a/src/ui/public/kuery/node_types/function.js b/src/ui/public/kuery/node_types/function.js index afa403f57b3e0..7f8b498006884 100644 --- a/src/ui/public/kuery/node_types/function.js +++ b/src/ui/public/kuery/node_types/function.js @@ -1,6 +1,5 @@ import _ from 'lodash'; import { functions } from '../functions'; -import { nodeTypes } from '../node_types'; export function buildNode(functionName, ...functionArgs) { const kueryFunction = functions[functionName]; @@ -17,7 +16,7 @@ export function buildNode(functionName, ...functionArgs) { } // Mainly only useful in the grammar where we'll already have real argument nodes in hand -export function buildNodeWithArgumentNodes(functionName, argumentNodes, serializeStyle = 'function') { +export function buildNodeWithArgumentNodes(functionName, argumentNodes) { if (_.isUndefined(functions[functionName])) { throw new Error(`Unknown function "${functionName}"`); } @@ -26,7 +25,6 @@ export function buildNodeWithArgumentNodes(functionName, argumentNodes, serializ type: 'function', function: functionName, arguments: argumentNodes, - serializeStyle }; } @@ -35,20 +33,3 @@ export function toElasticsearchQuery(node, indexPattern) { return kueryFunction.toElasticsearchQuery(node, indexPattern); } -export function toKueryExpression(node) { - const kueryFunction = functions[node.function]; - - if (!_.isUndefined(node.text)) { - return node.text; - } - - if (node.serializeStyle && node.serializeStyle !== 'function') { - return kueryFunction.toKueryExpression(node); - } - - const functionArguments = (node.arguments || []).map((argument) => { - return nodeTypes[argument.type].toKueryExpression(argument); - }); - - return `${node.function}(${functionArguments.join(', ')})`; -} diff --git a/src/ui/public/kuery/node_types/index.js b/src/ui/public/kuery/node_types/index.js index 8a9e00a3bd8a5..26249b0ed8e16 100644 --- a/src/ui/public/kuery/node_types/index.js +++ b/src/ui/public/kuery/node_types/index.js @@ -1,10 +1,11 @@ import * as functionType from './function'; import * as literal from './literal'; import * as namedArg from './named_arg'; +import * as wildcard from './wildcard'; export const nodeTypes = { function: functionType, literal, namedArg, + wildcard, }; - diff --git a/src/ui/public/kuery/node_types/literal.js b/src/ui/public/kuery/node_types/literal.js index b8c2c47f76c61..22af4ff6791be 100644 --- a/src/ui/public/kuery/node_types/literal.js +++ b/src/ui/public/kuery/node_types/literal.js @@ -1,5 +1,3 @@ -import _ from 'lodash'; - export function buildNode(value) { return { type: 'literal', @@ -11,11 +9,3 @@ export function toElasticsearchQuery(node) { return node.value; } -export function toKueryExpression(node) { - if (_.isString(node.value)) { - const escapedValue = node.value.replace(/"/g, '\\"'); - return `"${escapedValue}"`; - } - - return node.value; -} diff --git a/src/ui/public/kuery/node_types/named_arg.js b/src/ui/public/kuery/node_types/named_arg.js index 4db7f6c2c4cef..52fb2e209b8c9 100644 --- a/src/ui/public/kuery/node_types/named_arg.js +++ b/src/ui/public/kuery/node_types/named_arg.js @@ -15,6 +15,3 @@ export function toElasticsearchQuery(node) { return ast.toElasticsearchQuery(node.value); } -export function toKueryExpression(node) { - return `${node.name}=${ast.toKueryExpression(node.value)}`; -} diff --git a/src/ui/public/kuery/node_types/wildcard.js b/src/ui/public/kuery/node_types/wildcard.js new file mode 100644 index 0000000000000..dcacb3e69426b --- /dev/null +++ b/src/ui/public/kuery/node_types/wildcard.js @@ -0,0 +1,59 @@ +import { fromLiteralExpression } from '../ast/ast'; + +export const wildcardSymbol = Symbol('*'); + +// Copied from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions +function escapeRegExp(string) { + return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string +} + +// See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_reserved_characters +function escapeQueryString(string) { + return string.replace(/[+-=&|> { + if (typeof sequence === 'symbol') { + return '.*'; + } else { + return escapeRegExp(sequence); + } + }).join(''); + const regexp = new RegExp(`^${regex}$`); + return regexp.test(string); +} + +export function toElasticsearchQuery(node) { + const { value } = node; + return value.map(sequence => { + if (typeof sequence === 'symbol') { + return '*'; + } else { + return sequence; + } + }).join(''); +} + +export function toQueryStringQuery(node) { + const { value } = node; + return value.map(sequence => { + if (typeof sequence === 'symbol') { + return '*'; + } else { + return escapeQueryString(sequence); + } + }).join(''); +} diff --git a/src/ui/public/query_bar/directive/query_bar.html b/src/ui/public/query_bar/directive/query_bar.html index a76dc586c2063..641a36fba7a3e 100644 --- a/src/ui/public/query_bar/directive/query_bar.html +++ b/src/ui/public/query_bar/directive/query_bar.html @@ -79,22 +79,6 @@
- -
- -