Right now aggregations like terms will blindly call buildAggregation for all buckets that might be collected whether or not there were any documents in that aggregation. That forces all sub-aggregations to add guards like
if (valuesSources == null || bucket >= counts.size()) {
But the parent aggregations already know if they've sent any documents to the sub-aggregations for a particular bucket. And they could just call buildEmptyAggregation for buckets that have never seen any docs. That'd be so much kinder.