[MIOpen] Cutoff perf for GenericSearch

[cderb]

Motivation

The time it takes to search through tunable solvers can be considerable. It is desirable to have a method of terminating searches early if there is a reasonable expectation that the search will not yield the most performant solution.
We would like to shorten overall search time by adding a cutoff time and skip time for tuning based on previously observed results.

Technical Details

Environment variables added to generic_serach_controls.hpp to affect early termination or skipping portions of search. MIOPEN_SEARCH_CUTOFF, MIOPEN_SEARCH_CUTOFF_MUL, MIOPEN_SEARCH_SKIP_PCT.
Search cutoff/skipping logic added to GenericSearch.
generic_search_best_time - best time of best solution, to set baseline performance before GenericSearch will enter into multi-run sampling, which will skip perf configurations which are expected to be slower than the current best solution.
generic_search_worst_time - worst time of best solver, used to set cutoff time to prematurely end overly slow searches
Message passing for cutoff time added with mutable variables in ExecutionContext. This is how the cutoff time is communicated between solvers.
Re-ordered algorithms in FindCore to have potentially faster tunable solvers examined first.

Test Plan

Compare results of with MIOPEN_SEARCH_CUTOFF ON and OFF. Winning solver should have parity for both methods.
In some cases should observe early termination of search for tunable solvers.
The effectiveness will be determined by the decrease in runtime of the MIOpenDriver command with MIOPEN_SEARCH_CUTOFF ON during tuning.

[BrianHarrisonAMD]

I think we need to figure out a good testing plan for rolling this out, but I like it.

I would prefer we test enough to have confidence that this can just replace the existing SearchDbUpdate, but probably a discussion with @BradPepersAMD, @JonathanLichtnerAMD, and @TristanZerrAMD about that is needed.

[cderb]

I think It will be sufficient to:

1. Collect a find db diff between the results of SearchDbUpdate and SearchCutoffDbUpdate. This would demonstrate correctness, if the winning solver is the same between the 2. Bonus points if the runner ups are present as well. I have observed that for some configurations the optimal solver may also have a few extremely poorly performing configurations which does sometimes lead to it being cutoff.
2. Collect run time information for comparison of the 2 enforce modes. This would demonstrate the performance benefit.

I've collected some examples of these, but we may still desire a greater sample set before we proceed to making this a default.

[cderb]

Brought in changes from #2833
Provides tuning cutoff option during find loop.

Added message passing for the current best time into the evaluation loop of each solver.
Early termination of the evaluation when elapsed time reaches a threshold.
Skip Naive solvers when other solutions are present.

[mattwill-amd]

One variable added to the table. Looks good.

[JonathanLichtnerAMD]

LGTM