[SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster#28412

Closed

baohe-zhang wants to merge 24 commits intoapache:masterfrom

baohe-zhang:SPARK-31608

Commits on Apr 29, 2020

Add HybridKVStore and unit test
Baohe Zhang
committed

Commits on Apr 30, 2020

Resolve a race condition in UT and fix some style issues
Baohe Zhang
committed

Commits on May 6, 2020

Cleanup store when encounters Exception
Baohe Zhang
committed
Fix style issue
Baohe Zhang
committed
Update config doc
Baohe Zhang
committed

Commits on May 12, 2020

Add memory restriction
Baohe Zhang
committed
Merge branch 'master' into SPARK-31608
Baohe Zhang
committed

Commits on Jun 4, 2020

Simplify logic by dumping InMemoryStore data to LevelDB
Baohe Zhang
committed
Fix style check and remove unused imports
Baohe Zhang
committed

Commits on Jun 8, 2020

Move HybridStore class to history pacakge, create HistoryServerMemoryManager class for memory tracking
Baohe Zhang
committed
Fix typo
Baohe Zhang
committed
Set HybridStore to false by default, add doc for maxMemoryUsage
Baohe Zhang
committed
Minor fix
Baohe Zhang
committed
Merge branch 'master' into SPARK-31608
Baohe Zhang
committed
Minor fix
Baohe Zhang
committed

Commits on Jun 9, 2020

Removed special logic for CachedQuantile, added retry logic
Baohe Zhang
committed
Addressed comments
Baohe Zhang
committed

Commits on Jun 10, 2020

Seperate rebuildAppStore and switchToLevelDB logic
Baohe Zhang
committed
Wrap switch code in try block
Baohe Zhang
committed
Update close() of HybridStore
Baohe Zhang
committed
Turn off HybridStore by default
Baohe Zhang
committed

Commits on Jun 15, 2020

Update monitoring.md, won't create memoryManager if hybrid store is disabled, seperate logic of create leveldb store.
Baohe Zhang
committed
Update switch thread name
Baohe Zhang
committed

Commits on Jul 6, 2020

Update memory usage estimation
Baohe Zhang
committed