Add timeout for HBO stats fetch

[feilong-liu]

Fix issue #20355
Depended by https://github.com/facebookexternal/presto-facebook/pull/2394

• Add timeout to HBO optimizers, will not populate stats equivalent nodes field if timeout
• Add queries which timeouts to log

Test plan - (Please fill in how you tested your changes)

Tested locally end to end.

== RELEASE NOTES ==

General Changes
* Add timeout for HBO optimizer, timeout set by session parameter `history_based_optimizer_timeout_limit`

[feilong-liu]

When HBO register function failed to load statistics, add warning and also add to logging

[feilong-liu]

Here we consider empty stats returned as failure of loading history

[feilong-liu]

Remove query ID when we invalidate the cache

[feilong-liu]

Pass timeout limit to stats provider

[feilong-liu]

directly return if load of history failed

[feilong-liu]

Store the query ID which failed to get history stats

[pranjalssh]

Can you check if it catches timeout appropriately for pipeline in D46899410

A lot of cost will come from HistoricalStatisticsEquivalentPlanMarkingOptimizer where we prefetch all hbo stats

[feilong-liu]

In unit tests, we want to test for queries which do not have join/aggregation too.

[feilong-liu]

If we only care about query plans with join/aggregation, return false here, and skip adding stats equivalent node for this query plan.

[feilong-liu]

Added for unit tests, where queries without join/aggregation are tested.

[feilong-liu]

if false returned, we will skip adding stats equivalent node in HBO marking optimizer.

[feilong-liu]

Even if fetch history timeouts or return empty, we still need StatsEquivalentPlanNode as it will be used in populating HBO history.

[feilong-liu]

Can you check if it catches timeout appropriately for pipeline in D46899410

A lot of cost will come from HistoricalStatisticsEquivalentPlanMarkingOptimizer where we prefetch all hbo stats

I debug the pipeline in this diff, the problem is not in fetching stats, but from calculating plan node hashes, which includes inputing table meta data access that is the problem here.

This PR does not solve this problem, as I do not find a way to pass timeout to meta data access here, and solving this problem should be from a separate PR imo.

But still, this PR can solve the following problems: 1) reduce overhead, as I added a new field to track queries which fail to fetch history stats, and skip trying to get HBO stats in this case 2) add new field in logging for timeout queries, so that we can track these queries.

[mlyublena]

do we want to log additional information about the failure?
do we plan to use this for other usecases beyond HBO timeout?

[mlyublena]

perhaps make the function more general and pass in the optimizer name as argument

[feilong-liu]

do we want to log additional information about the failure?

Maybe just this for now, we can add more later if needed.

do we plan to use this for other usecases beyond HBO timeout?

Other optimizers can populate this field as well.

perhaps make the function more general and pass in the optimizer name as argument

This is a private method in HBO optimizer, will not be used by other optimizers

[mlyublena]

why is this is a set? do we expect to have more than 1 item there? or is it so we can quickly check for containment without worrying if it is set or not

[feilong-liu]

The query ID will be removed after the query completes. Yes, there can be more than 1 item, when multiple queries are running concurrently.

The life span of the query ID will be the same as in other Maps in the same class.

[mlyublena]

that's cool!
I think another use case for this field could be tracking bugs in disabled optimizers: when checking if an optimizer is applicable (when verbose_optimizer_info_enabled=true) I added a try/block to prevent the main loop crashing with a buggy (but disabled) optimizer: instead of silently ignoring this, we can record that we found a bug for an applicable optimizer (see PlanOptimizer::isApplicable)

[pranjalssh]

lets call it "optimizer.history-based-optimizer-timeout". And use Duration instead of int. Config can then read values like "1s" or "100ms"