Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
0fb37b5
Implement top_metrics agg
nik9000 Jan 17, 2020
a105a63
Moar tests
nik9000 Jan 17, 2020
4d2557d
Better
nik9000 Jan 17, 2020
5146563
Checkstyle
nik9000 Jan 17, 2020
12abffd
Fixup docs compile
nik9000 Jan 17, 2020
a685e29
Explain
nik9000 Jan 17, 2020
0f130d4
Sneaky sneaky
nik9000 Jan 17, 2020
eacddb5
Merge branch 'master' into top_metrics
nik9000 Jan 17, 2020
2b7a62e
Throw
nik9000 Jan 17, 2020
caf1004
Merge branch 'master' into top_metrics
nik9000 Jan 21, 2020
9f12a5e
Factory time
nik9000 Jan 21, 2020
845a8e7
Move
nik9000 Jan 21, 2020
390c4c5
Use fancy test methods
nik9000 Jan 21, 2020
21a2387
Bucketed sort
nik9000 Jan 22, 2020
feeca63
Squish nocommits
nik9000 Jan 23, 2020
228ccc9
Merge branch 'master' into top_metrics
nik9000 Jan 23, 2020
5f71b84
Merge branch 'master' into top_metrics
nik9000 Jan 27, 2020
aa33abb
Merge branch 'master' into top_metrics
nik9000 Jan 27, 2020
a2569de
Move SortValue to core
nik9000 Jan 27, 2020
e2925e3
moar sort
nik9000 Jan 28, 2020
98091fa
Rename
nik9000 Jan 28, 2020
b9d8085
Cleanup
nik9000 Jan 28, 2020
935c755
Drop method from SortValue
nik9000 Jan 28, 2020
fb7f122
last test
nik9000 Jan 28, 2020
a6c69d8
Smaller collector?
nik9000 Jan 28, 2020
9544c15
Move to InternalAggregationTestCase
nik9000 Jan 28, 2020
6efdd14
high level client support
nik9000 Jan 28, 2020
0a44b09
A little better docs
nik9000 Jan 29, 2020
ef8afc3
Merge branch 'master' into top_metrics
nik9000 Jan 29, 2020
0b191f8
Test
nik9000 Jan 29, 2020
49ee3c7
Javadoc
nik9000 Jan 30, 2020
6d83a2a
yaml
nik9000 Jan 30, 2020
80bf931
Force define
nik9000 Jan 30, 2020
7f46898
Script
nik9000 Jan 31, 2020
c36f28e
geo_distance
nik9000 Jan 31, 2020
e846862
Fix indentation
nik9000 Jan 31, 2020
af3fe9d
Merge branch 'master' into top_metrics
nik9000 Jan 31, 2020
05c6464
WIP
nik9000 Jan 31, 2020
0a9e3cc
Merge branch 'master' into top_metrics
nik9000 Jan 31, 2020
36eb495
Fixup casting
nik9000 Jan 31, 2020
290cc27
Cleanup
nik9000 Jan 31, 2020
6dbc02b
Drop out of date comment
nik9000 Feb 3, 2020
5902e6c
Merge branch 'master' into top_metrics
nik9000 Feb 3, 2020
ca3acf2
Update docs
nik9000 Feb 3, 2020
1fc6a2a
numeric_type examples
nik9000 Feb 3, 2020
a06ca3c
Merge branch 'master' into top_metrics
nik9000 Feb 4, 2020
7762c4f
Ooops
nik9000 Feb 4, 2020
0b78715
Implement sorting by top_metrics
nik9000 Feb 4, 2020
ea2e997
Better shuffle
nik9000 Feb 5, 2020
bdd0676
Checkstyle
nik9000 Feb 5, 2020
09d969b
Merge branch 'master' into top_metrics
nik9000 Feb 5, 2020
d7429b6
Merge branch 'master' into top_metrics
nik9000 Feb 5, 2020
d1b90e2
Merge branch 'master' into top_metrics
nik9000 Feb 5, 2020
c7b6317
Docs and test
nik9000 Feb 5, 2020
ba6e21f
Tests
nik9000 Feb 5, 2020
fdada3c
Merge branch 'master' into top_metrics
nik9000 Feb 5, 2020
e47d6c6
Merge branch 'master' into top_metrics
nik9000 Feb 10, 2020
abfc012
Fix javadoc
nik9000 Feb 10, 2020
03ff2b5
Javadoc
nik9000 Feb 10, 2020
04c2fcd
Merge!
nik9000 Feb 11, 2020
9752139
Merge branch 'master' into top_metrics
nik9000 Feb 11, 2020
4240a53
Merge branch 'master' into top_metrics
nik9000 Feb 11, 2020
ca6e52d
Test
nik9000 Feb 11, 2020
792da85
Merge branch 'master' into top_metrics
nik9000 Feb 13, 2020
6577077
Merge branch 'master' into top_metrics
nik9000 Feb 13, 2020
43c705b
Merge tests
nik9000 Feb 13, 2020
5f46829
Flip example
nik9000 Feb 13, 2020
2b5dca1
No tie breaking
nik9000 Feb 13, 2020
e6fd601
Error message
nik9000 Feb 13, 2020
134d99b
Word
nik9000 Feb 13, 2020
896e08a
Another word
nik9000 Feb 14, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.analytics.ParsedStringStats;
import org.elasticsearch.client.analytics.ParsedTopMetrics;
import org.elasticsearch.client.analytics.StringStatsAggregationBuilder;
import org.elasticsearch.client.analytics.TopMetricsAggregationBuilder;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.client.core.CountResponse;
import org.elasticsearch.client.core.GetSourceRequest;
Expand Down Expand Up @@ -1929,6 +1931,7 @@ static List<NamedXContentRegistry.Entry> getDefaultNamedXContents() {
map.put(TopHitsAggregationBuilder.NAME, (p, c) -> ParsedTopHits.fromXContent(p, (String) c));
map.put(CompositeAggregationBuilder.NAME, (p, c) -> ParsedComposite.fromXContent(p, (String) c));
map.put(StringStatsAggregationBuilder.NAME, (p, c) -> ParsedStringStats.PARSER.parse(p, (String) c));
map.put(TopMetricsAggregationBuilder.NAME, (p, c) -> ParsedTopMetrics.PARSER.parse(p, (String) c));
List<NamedXContentRegistry.Entry> entries = map.entrySet().stream()
.map(entry -> new NamedXContentRegistry.Entry(Aggregation.class, new ParseField(entry.getKey()), entry.getValue()))
.collect(Collectors.toList());
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.elasticsearch.client.analytics;

import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.xcontent.ConstructingObjectParser;
import org.elasticsearch.common.xcontent.ObjectParser;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.XContentParserUtils;
import org.elasticsearch.search.aggregations.ParsedAggregation;

import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import static org.elasticsearch.common.xcontent.ConstructingObjectParser.constructorArg;

/**
* Results of the {@code top_metrics} aggregation.
*/
public class ParsedTopMetrics extends ParsedAggregation {
private static final ParseField TOP_FIELD = new ParseField("top");

private final List<TopMetrics> topMetrics;

private ParsedTopMetrics(String name, List<TopMetrics> topMetrics) {
setName(name);
this.topMetrics = topMetrics;
}

/**
* The list of top metrics, in sorted order.
*/
public List<TopMetrics> getTopMetrics() {
return topMetrics;
}

@Override
public String getType() {
return TopMetricsAggregationBuilder.NAME;
}

@Override
protected XContentBuilder doXContentBody(XContentBuilder builder, Params params) throws IOException {
builder.startArray(TOP_FIELD.getPreferredName());
for (TopMetrics top : topMetrics) {
top.toXContent(builder, params);
}
return builder.endArray();
}

public static final ConstructingObjectParser<ParsedTopMetrics, String> PARSER = new ConstructingObjectParser<>(
TopMetricsAggregationBuilder.NAME, true, (args, name) -> {
@SuppressWarnings("unchecked")
List<TopMetrics> topMetrics = (List<TopMetrics>) args[0];
return new ParsedTopMetrics(name, topMetrics);
});
static {
PARSER.declareObjectArray(constructorArg(), (p, c) -> TopMetrics.PARSER.parse(p, null), TOP_FIELD);
ParsedAggregation.declareAggregationFields(PARSER);
}

/**
* The metrics belonging to the document with the "top" sort key.
*/
public static class TopMetrics implements ToXContent {
private static final ParseField SORT_FIELD = new ParseField("sort");
private static final ParseField METRICS_FIELD = new ParseField("metrics");

private final List<Object> sort;
private final Map<String, Double> metrics;

private TopMetrics(List<Object> sort, Map<String, Double> metrics) {
this.sort = sort;
this.metrics = metrics;
}

/**
* The sort key for these top metrics.
*/
public List<Object> getSort() {
return sort;
}

/**
* The top metric values returned by the aggregation.
*/
public Map<String, Double> getMetrics() {
return metrics;
}

private static final ConstructingObjectParser<TopMetrics, Void> PARSER = new ConstructingObjectParser<>("top", true,
(args, name) -> {
@SuppressWarnings("unchecked")
List<Object> sort = (List<Object>) args[0];
@SuppressWarnings("unchecked")
Map<String, Double> metrics = (Map<String, Double>) args[1];
return new TopMetrics(sort, metrics);
});
static {
PARSER.declareFieldArray(constructorArg(), (p, c) -> XContentParserUtils.parseFieldsValue(p),
SORT_FIELD, ObjectParser.ValueType.VALUE_ARRAY);
PARSER.declareObject(constructorArg(), (p, c) -> p.map(HashMap::new, XContentParser::doubleValue), METRICS_FIELD);
}

public XContentBuilder toXContent(XContentBuilder builder, ToXContent.Params params) throws IOException {
builder.startObject();
builder.field(SORT_FIELD.getPreferredName(), sort);
builder.field(METRICS_FIELD.getPreferredName(), metrics);
builder.endObject();
return builder;
};
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.elasticsearch.client.analytics;

import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Writeable;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.index.query.QueryRewriteContext;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AbstractAggregationBuilder;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortBuilder;

import java.io.IOException;
import java.util.Map;

/**
* Builds the Top Metrics aggregation request.
* <p>
* NOTE: This extends {@linkplain AbstractAggregationBuilder} for compatibility
* with {@link SearchSourceBuilder#aggregation(AggregationBuilder)} but it
* doesn't support any "server" side things like
* {@linkplain Writeable#writeTo(StreamOutput)},
* {@linkplain AggregationBuilder#rewrite(QueryRewriteContext)}, or
* {@linkplain AbstractAggregationBuilder#build(QueryShardContext, AggregatorFactory)}.
*/
public class TopMetricsAggregationBuilder extends AbstractAggregationBuilder<TopMetricsAggregationBuilder> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was confused by this, since usually things that descend from AbstractAggregationBuilder are the entry points for that aggregation. In this case, it looks like we're just using the XContentBuilder aspect of this, presumably for the HLRC to build out requests? If that's the case, I think it would help to make clear in the javadoc that this is not the main builder for this aggregation, and be explicit about its intended use.

I don't think I'm alone in thinking our aggregation builders do too much, and this seems symptomatic of that. Maybe at some point, we can talk about breaking up those roles.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have to descend from AbstractAggregationBuilder to make the client happy. I can check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also pretty sure you have to. All I'm suggesting is adding a note to the javadoc that explains what an aggregation builder is doing here, and that it's not the "main" builder for Top Metrics. It makes sense in context of this PR, but I am imagining a future where someone is looking at all classes that implement AbstractAggregationBuilder and scratching their head.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added javadoc explaining this.

public static final String NAME = "top_metrics";

private final SortBuilder<?> sort;
private final String metric;

/**
* Build the request.
* @param name the name of the metric
* @param sort the sort key used to select the top metrics
* @param metric the name of the field to select
*/
public TopMetricsAggregationBuilder(String name, SortBuilder<?> sort, String metric) {
super(name);
this.sort = sort;
this.metric = metric;
}

@Override
public String getType() {
return NAME;
}

@Override
protected XContentBuilder internalXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
{
builder.startArray("sort");
sort.toXContent(builder, params);
builder.endArray();
builder.startObject("metric").field("field", metric).endObject();
}
return builder.endObject();
}

@Override
protected void doWriteTo(StreamOutput out) throws IOException {
throw new UnsupportedOperationException();
}

@Override
protected AggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, Builder subfactoriesBuilder)
throws IOException {
throw new UnsupportedOperationException();
}

@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metaData) {
throw new UnsupportedOperationException();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -676,7 +676,11 @@ public void testDefaultNamedXContents() {
List<NamedXContentRegistry.Entry> namedXContents = RestHighLevelClient.getDefaultNamedXContents();
int expectedInternalAggregations = InternalAggregationTestCase.getDefaultNamedXContents().size();
int expectedSuggestions = 3;

// Explicitly check for metrics from the analytics module because they aren't in InternalAggregationTestCase
assertTrue(namedXContents.removeIf(e -> e.name.getPreferredName().equals("string_stats")));
assertTrue(namedXContents.removeIf(e -> e.name.getPreferredName().equals("top_metrics")));

assertEquals(expectedInternalAggregations + expectedSuggestions, namedXContents.size());
Map<Class<?>, Integer> categories = new HashMap<>();
for (NamedXContentRegistry.Entry namedXContent : namedXContents) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,21 @@
import org.elasticsearch.client.ESRestHighLevelClientTestCase;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.search.sort.FieldSortBuilder;
import org.elasticsearch.search.sort.SortOrder;

import java.io.IOException;

import static java.util.Collections.singletonList;
import static java.util.Collections.singletonMap;
import static org.hamcrest.Matchers.aMapWithSize;
import static org.hamcrest.Matchers.closeTo;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasEntry;
import static org.hamcrest.Matchers.hasSize;

public class AnalyticsAggsIT extends ESRestHighLevelClientTestCase {
public void testBasic() throws IOException {
public void testStringStats() throws IOException {
BulkRequest bulk = new BulkRequest("test").setRefreshPolicy(RefreshPolicy.IMMEDIATE);
bulk.add(new IndexRequest().source(XContentType.JSON, "message", "trying out elasticsearch"));
bulk.add(new IndexRequest().source(XContentType.JSON, "message", "more words"));
Expand All @@ -55,4 +60,20 @@ public void testBasic() throws IOException {
assertThat(stats.getDistribution(), hasEntry(equalTo("r"), closeTo(.12, .005)));
assertThat(stats.getDistribution(), hasEntry(equalTo("t"), closeTo(.09, .005)));
}

public void testBasic() throws IOException {
BulkRequest bulk = new BulkRequest("test").setRefreshPolicy(RefreshPolicy.IMMEDIATE);
bulk.add(new IndexRequest().source(XContentType.JSON, "s", 1, "v", 2));
bulk.add(new IndexRequest().source(XContentType.JSON, "s", 2, "v", 3));
highLevelClient().bulk(bulk, RequestOptions.DEFAULT);
SearchRequest search = new SearchRequest("test");
search.source().aggregation(new TopMetricsAggregationBuilder(
"test", new FieldSortBuilder("s").order(SortOrder.DESC), "v"));
SearchResponse response = highLevelClient().search(search, RequestOptions.DEFAULT);
ParsedTopMetrics top = response.getAggregations().get("test");
assertThat(top.getTopMetrics(), hasSize(1));
ParsedTopMetrics.TopMetrics metric = top.getTopMetrics().get(0);
assertThat(metric.getSort(), equalTo(singletonList(2)));
assertThat(metric.getMetrics(), equalTo(singletonMap("v", 3.0)));
}
}
1 change: 1 addition & 0 deletions docs/java-rest/high-level/aggs-builders.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This page lists all the available aggregations with their corresponding `Aggrega
| {ref}/search-aggregations-metrics-stats-aggregation.html[Stats] | {agg-ref}/metrics/stats/StatsAggregationBuilder.html[StatsAggregationBuilder] | {agg-ref}/AggregationBuilders.html#stats-java.lang.String-[AggregationBuilders.stats()]
| {ref}/search-aggregations-metrics-sum-aggregation.html[Sum] | {agg-ref}/metrics/sum/SumAggregationBuilder.html[SumAggregationBuilder] | {agg-ref}/AggregationBuilders.html#sum-java.lang.String-[AggregationBuilders.sum()]
| {ref}/search-aggregations-metrics-top-hits-aggregation.html[Top hits] | {agg-ref}/metrics/tophits/TopHitsAggregationBuilder.html[TopHitsAggregationBuilder] | {agg-ref}/AggregationBuilders.html#topHits-java.lang.String-[AggregationBuilders.topHits()]
| {ref}/search-aggregations-metrics-top-metrics.html[Top Metrics] | {javadoc-client}/analytics/TopMetricsAggregationBuilder.html[TopMetricsAggregationBuilder] | None
| {ref}/search-aggregations-metrics-valuecount-aggregation.html[Value Count] | {agg-ref}/metrics/valuecount/ValueCountAggregationBuilder.html[ValueCountAggregationBuilder] | {agg-ref}/AggregationBuilders.html#count-java.lang.String-[AggregationBuilders.count()]
| {ref}/search-aggregations-metrics-string-stats-aggregation.html[String Stats] | {javadoc-client}/analytics/StringStatsAggregationBuilder.html[StringStatsAggregationBuilder] | None
|======
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/aggregations/metrics.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ include::metrics/sum-aggregation.asciidoc[]

include::metrics/tophits-aggregation.asciidoc[]

include::metrics/top-metrics-aggregation.asciidoc[]

include::metrics/valuecount-aggregation.asciidoc[]

include::metrics/median-absolute-deviation-aggregation.asciidoc[]
Expand Down
Loading