HDDS-4313. Create thread-local instance of FileSystem in HadoopFsGenerator

[adoroszlai]

What changes were proposed in this pull request?

1. Create a separate instance of FileSystem in HadoopFsGenerator for each test thread.
2. Move directory creation to test setup. Previously mkdirs() resulted in an extra RPC for each file.
3. Provide a hook method in BaseFreonGenerator for thread-specific cleanup.

https://issues.apache.org/jira/browse/HDDS-4313

How was this patch tested?

ozone freon ockg -n1 -t1 -p warmup
ozone freon dfsg -n10000 -t10

/opt/profiler/profiler.sh -e lock -d 60 -o svg -f dfsg-lock.svg $(ps aux | grep 'proc_freo[n]' | awk '{ print $2 }')

[elek]

+1 Thanks for the patch @adoroszlai

Question: Is it reasonable to keep the original behavior (one client, multiple threads) or it's totally invalid?

[adoroszlai]

Thanks @elek for the review.

Question: Is it reasonable to keep the original behavior (one client, multiple threads) or it's totally invalid?

I don't know. If you think it's worth the effort to add a flag to control this behavior, I can do that.

[adoroszlai]

Thanks @elek for the review.

Question: Is it reasonable to keep the original behavior (one client, multiple threads) or it's totally invalid?

I don't know. If you think it's worth the effort to add a flag to control this behavior, I can do that.

... if you create a follow-up Jira. ;)