Prevent buildtest from hanging due to directory traversal

[shahzebsiddiqui]

At the moment, buildtest has support for directory traversal when searching for buildspecs that can lead to serious issues if one specifies a directory path with lots of files. This can lead to a full directory traversal of filesystem which can impact some of the features in buildtest that may include the following options

• buildtest build -b
• buildtest buildspec validate -b
• buildtest buildspec find -d

buildtest uses the walk_tree method using the file_traversal_limit to stop buildtest from searching for files beyond the limit

buildtest/buildtest/utils/file.py

Lines 228 to 229 in 26ed347

	if files_traversed >= file_traverse_limit:
	return [os.path.abspath(fname) for fname in files_list]

.

The default value is set to 999999 however this may need to be configurable such that user can alter behavior for directory traversal.

We have a few options we can try to customize this value

1. Add support in configuration file and/or command line that can be overridden. Perhaps a key file_traversal_limit in buildspec cache and option --file-traversal-limit.

In configuration file it would be under the system definition it could be something like this

system:
  generic:
      file_traversal_limit: 5000

The objective is to prevent user input to lead into long directory traversal where one inputs the following. The following walk_tree methods should be updated with value of file_traversla

1. buildtest build -b /tmp

buildtest/buildtest/cli/build.py

Line 550 in 0804a13

buildspecs = walk_tree(buildspec, ".yml")

2. buildtest buildspec validate -b /tmp and buildtest bc find -d /

buildtest/buildtest/cli/buildspec.py

Line 239 in 0804a13

buildspec = walk_tree(path, ".yml")

For the option we should have it as a global option.

We should not update the walk_tree method with file_traversal_limit in the following invocation

1. This invocation is for buildtest schema

buildtest/buildtest/cli/schema.py

Line 48 in 0804a13

schema_examples = walk_tree(examples, ".yml")

4. This invocation is for buildtest history
   

   buildtest/buildtest/cli/history.py

   Line 113 in 0804a13

   history_files = walk_tree(BUILD_HISTORY_DIR, ".json")