Doc-value-only fields

[jpountz]

Users who index time series typically care a lot about indexing rate and space efficiency. Disabling inverted structures like the inverted index and points would help on both fronts. Queries could still work using doc values, but more slowly, which is a trade-off that these users are often happy to make.

Default mappings would still create inverted structures, so users would have to opt-in to trade search efficiency for disk space / indexing rate.

I'd like to make this change depend on some feedback mechanism as outlined in #48058, so that users having slow queries because inverted structures have been disabled would never come as a surprise.

[elasticmachine]

Pinging @elastic/es-search (:Search/Mapping)

[mayya-sharipova]

I would like to clarify something. Users currently can disable inverted data structures for fields by using "index": false

"mappings": {
      "properties": {
          "location": {
              "type": "geo_point",
              "doc_values": true, 
              "index": false
          }
      }
  }

By this will disable some queries. Is your proposal to make all queries work on docValues instead of inverted data structures? I am wondering if this even possible for all field types?

[jpountz]

Is your proposal to make all queries work on docValues instead of inverted data structures?

Yes. For instance all numeric fields create queries on doc values today, but only as a way to speed up query execution when there is another filter in the query that is more selective than this one (using IndexOrDocValuesQuery).

I am wondering if this even possible for all field types?

I think this would work for all fields that support doc values, but I think that this feature would be mostly useful for number fields, so I was thinking of focusing on those first.

[jtibshirani]

The issue #48665 could relate to this idea -- if an index is sorted on a numeric field, we could perform fast range queries that rely only on the field's doc values.

[javanna]

This is being worked on as part of #82409 for numeric fields. @jpountz you mentioned above that you would like this type of slower queries to be linked to some warning mechanism. Do you think that that is a hard requirement or can we add support queries on doc_values although we haven't yet figured out the details around the warning mechanism?

[jpountz]

I think that the direction we're taking involves doing more and more costly stuff (think of runtime fields or identifying sequences of events with EQL) and we will want to reconsider whether we really want to warn on slow operations. So I wouldn't make this a hard requirement, and I wonder if this should still be a requirement at all.