Allow user to customize query cost estimate

[betodealmeida]

CATEGORY

Choose one

• Bug Fix
• Enhancement (new features, refinement)
• Refactor
• Add tests
• Build / Development Environment
• Documentation

SUMMARY

#8172 introduced the ability to estimate query costs before running them. Currently, this is supported by Presto, and it returns a relative CPU and network cost.

Ideally, we want to show the cost as dollars, time or percentile, instead of showing relative costs. Since the relationship between the relative cost and the actual cost varies from infrastructure to infrastructure, we need to provide users a way of customizing the query cost estimate.

Here's a hypothetical example where we use a coefficient to compute the query cost in dollars based on CPU and network cost:

def presto_query_cost_formatter(result):
    cpu_coefficient = 1e-12
    network_coefficient = 1e-12

    cost = 0
    for row in result:
        cost += row.get("cpuCost", 0) * cpu_coefficient
        cost += row.get("networkCost", 0) * network_coefficient

    return [{"Cost": f"US$ {cost:.2f}"}]


DEFAULT_FEATURE_FLAGS = {
    "ESTIMATE_QUERY_COST": True,
    "QUERY_COST_FORMATTER": {"presto": presto_query_cost_formatter},
}

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Requires DB Migration.
• Confirm DB Migration upgrade and downgrade tested.
• Introduces new feature or API
• Removes existing feature or API

REVIEWERS

@khtruong

[etr2460]

one documentation request!

[etr2460]

could we add a type to this so that people know what valid params there are? Or add that in a comment somewhere?

[betodealmeida]

Done! Sorry, I'm still not fully used to types in Python, and I keep forgetting them. :-P

[codecov-io]

Codecov Report

Merging #8470 into master will increase coverage by 0.09%.
The diff coverage is 23.52%.

@@            Coverage Diff             @@
##           master    #8470      +/-   ##
==========================================
+ Coverage   66.41%   66.51%   +0.09%     
==========================================
  Files         449      449              
  Lines       22567    22607      +40     
  Branches     2367     2367              
==========================================
+ Hits        14987    15036      +49     
+ Misses       7442     7433       -9     
  Partials      138      138

Impacted Files	Coverage Δ
superset/views/core.py	71.94% <0%> (ø)	⬆️
superset/db_engine_specs/presto.py	22.64% <20%> (+0.2%)	⬆️
superset/db_engine_specs/base.py	80.59% <66.66%> (-0.14%)	⬇️
superset/db_engine_specs/oracle.py	66.66% <0%> (-25%)	⬇️
superset/db_engine_specs/mssql.py	48.83% <0%> (-9.5%)	⬇️
superset/db_engine_specs/impala.py	59.09% <0%> (-5.91%)	⬇️
superset/db_engine_specs/sqlite.py	66.66% <0%> (-4.31%)	⬇️
superset/db_engine_specs/bigquery.py	42.66% <0%> (-2.41%)	⬇️
superset/db_engine_specs/postgres.py	83.33% <0%> (-0.54%)	⬇️
superset/connectors/sqla/models.py	84.92% <0%> (+0.04%)	⬆️
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1adf742...f5be80f. Read the comment docs.

[etr2460]

lgtm, thanks for the docs!