Post-scripts for Continuous Dataframes #43061
Description
TL;DR It would be useful to be able to specify a painless script that can transform docs before indexing when using dataframes
Continuous data frames are really powerful for a number of non-ML use cases. They're great
for turning time series data into entity centric data. We plan on using this feature to build a snapshot of what things look like 'now' for uptime.
One limitation of data frames is that the documents that are created reflect the JSON structure of the aggregations used. This is pretty ugly in our use case. What we've wound up doing is writing everything in a big scripted metric aggregation so we can control the exact naming of fields etc. We want these field names to be friendly for all users of the index, and to be ECS compatible as well.
If dataframes supported a post-script option, we could potentially use more traditional aggregations then simply clean up the output with a simpler painless script.
Additionally, even with a scripted_metric aggregation, one may only manipulate results appearing under the scripted metric's namespace. This post-script option would let us put fields exactly where we want them.