From 7e5288dbe56513f610745cdf4a83725764cab5aa Mon Sep 17 00:00:00 2001 From: Igor Dvorzhak Date: Mon, 31 Oct 2022 13:23:00 -0700 Subject: [PATCH] Add a note about cost implication for enabled `fs.gs.status.parallel.enable` (#895) --- gcs/CONFIGURATION.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/gcs/CONFIGURATION.md b/gcs/CONFIGURATION.md index 04ae6f05a7..0723e88bf2 100644 --- a/gcs/CONFIGURATION.md +++ b/gcs/CONFIGURATION.md @@ -59,6 +59,10 @@ If `true`, executes Cloud Storage object requests in `FileSystem`'s `listStatus` and `getFileStatus` methods in parallel to reduce latency. + Note that enabling this performance optimization can increase workload + execution cost due to the increased number of Cloud Storage `objects.get` + and `objects.list` requests. + * `fs.gs.lazy.init.enable` (default: `false`) Enables lazy initialization of `GoogleHadoopFileSystem` instances.