diff --git a/velox/docs/develop.rst b/velox/docs/develop.rst index 48c4641e71c7..229f6b90a5cf 100644 --- a/velox/docs/develop.rst +++ b/velox/docs/develop.rst @@ -33,3 +33,4 @@ This guide is intended for Velox contributors and developers of Velox-based appl develop/TpchBenchmark develop/window develop/dynamic-loading + develop/cache diff --git a/velox/docs/develop/cache.rst b/velox/docs/develop/cache.rst new file mode 100644 index 000000000000..a82b62a6d8d2 --- /dev/null +++ b/velox/docs/develop/cache.rst @@ -0,0 +1,39 @@ +===================================== +AsyncDataCache (In Memory Data Cache) +===================================== + +Background +---------- +Velox provides a transparent data cache (AsyncDataCache) to accelerate table scans operators through hot data reuse and prefetch algorithms. +This data cache is integrated with the memory system to achieve dynamic memory sharing between the data cache and query memory. +When a query fails to allocate memory, Velox retries the allocation by shrinking the data cache. +Therefore, the data cache size is automatically adjusted in response to the query memory usage change. +See `Memory Management - Velox Documentation `_ +for more information about Velox's data cache. + +Configuration Properties +------------------------ +See `Configuration Properties +`_ +for AsyncDataCache related configuration properties. + +========= +SSD Cache +========= + +Background +---------- +The in-memory data cache (AsyncDataCache) is configured to use SSD when provided. +The SSD serves as an extension for the AsyncDataCache. +This helps mitigate the number of reads from slower storage. + +Configuration Properties +------------------------ +See `Configuration Properties +`_ +for SSD Cache related configuration properties. + +Metrics +------- +There are SSD cache relevant metrics that Velox emits during query execution and runtime. +See `Debugging Metrics <./debugging/metrics.rst>`_ and `Monitoring Metrics <../monitoring/metrics.rst>`_ for more details.