-
Notifications
You must be signed in to change notification settings - Fork 25.7k
Split regular histograms from date histograms. #19551
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is a bit strange to throw an AssertionError when assertions might not be enabled? Maybe just throw and IllegalArgumentException or a RuntimeException or ElasticsearchException or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We seem to be using this exception in a number of places already. I like that this is an error and not an exception as it means there is an error in the code rather than on the user end, and should not be caught?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feel free to tell me I'm totally wrong - but do we feel confident enough that a mistake here (in code) is a good enough reason to cause the node to restart ? I'm worried about endless restarts, where a single mistake on a specific API can cause problems on an entire node.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a good point, I am not sure. On the one hand, this is indeed a specific API and restarting is unlikely to fix anything, but on the other hand this is the same for stack overflows are memory leaks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for stack overflows are memory leaks?
I'm not sure what you mean? can you unpack those examples?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if you throw a subclass of RuntimeException here it'll get bubbled back to the user and we'll get a bug report if it shows up. AssertionError's problem is that catching it and forwarding it around properly gets us dangerously close to catching OOMs. We probably could make an exception for catching AssertionError but I'm not sure it is worth it when we can just throw a RuntimeException of some sort instead.
Also it just feels weird to throw an assertion error from outside of assertions! Its icky 😨
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the record, the JVM code itself seems to be using AssertionError for impossible branches.
for stack overflows are memory leaks?
I'm not sure what you mean? can you unpack those examples?
I mean that a specific API could trigger eg. a StackOverflowError, which will cause the node to shut down even though it does not prevent normal operations of the node otherwise.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the record, the JVM code itself seems to be using AssertionError for impossible branches.
Weird. I can live with it but it is weird.
|
I left a bunch of minor stuff. I love that InternalHistogram.Factory is going. And the new factory is well documented. |
|
Thanks @nik9000 for having a look to this huge PR! Why don't you like AssertionError? I actually like it for branches that should never get visited, as a way to mean that an invariant has been violated and that the code is buggy. Also the fact that it is an error and not an exception makes it less likely to be swallowed since errors are not supposed to be caught. Finally we seem to already have a number of places where we already use assertion errors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, but since its abstract can we still document what its doing? Does it really require the insane generic typing?
|
I pushed more commits to address comments. The only one I did not address is the naming convention for setters since we use it in all aggregations and it would probably need to be a bigger discussion. |
|
LGTM. |
Currently both aggregations really share the same implementation. This commit splits the implementations so that regular histograms can support decimal intervals/offsets and compute correct buckets for negative decimal values. However the response API is still the same. So for intance both regular histograms and date histograms will produce an `org.elasticsearch.search.aggregations.bucket.histogram.Histogram` aggregation. The optimization to compute an identifier of the rounded value and the rounded value itself has been removed since it was only used by regular histograms, which now do the rounding themselves instead of relying on the Rounding abstraction. Closes elastic#8082 Closes elastic#4847
56040c5 to
a0818d3
Compare
|
Thanks @nik9000 ! |
* master: Fix REST test documentation [Test] move methods from bwc test to test package for use in plugins (elastic#19738) package-info.java should be in src/main only. Split regular histograms from date histograms. elastic#19551 Tighten up concurrent store metadata listing and engine writes (elastic#19684) Plugins: Make NamedWriteableRegistry immutable and add extenion point for named writeables Add documentation for the 'elasticsearch-translog' tool [TEST] Increase time waiting for all shards to move off/on to a node Fixes the active shard count check in the case of (elastic#19760) Fixes cat tasks operation in detailed mode ignore some docker craziness in scccomp environment checks
Currently both aggregations really share the same implementation. This commit
splits the implementations so that regular histograms can support decimal
intervals/offsets and compute correct buckets for negative decimal values.
However the response API is still the same. So for intance both regular
histograms and date histograms will produce an
org.elasticsearch.search.aggregations.bucket.histogram.Histogramaggregation.
The optimization to compute an identifier of the rounded value and the
rounded value itself has been removed since it was only used by regular
histograms, which now do the rounding themselves instead of relying on the
Rounding abstraction.
Closes #8082
Closes #4847