helper: Allow logs isolation per acceptance test

[radeksimko]

Why

We run nightly acceptance tests for and collect debug logs produced during the whole run. As we run tests in parallel for some providers (e.g. AWS), the logs get written to the file in parallel too which isn't very helpful for debugging, especially when you're trying to isolate a problem related to a single test and track down a chain of HTTP requests/responses and you have to skip unrelated messages just b/c another test happens to run at the same time.

This also means we only need to download a single ~0.5M file for the test that is actually failing instead of downloading debug log for the whole run (~650M in case of AWS).

How

This patch allows us to use a new ENV variable:

TF_LOG_PATH_MASK=/log/location/tf-test-%s.log

I have considered the option of using templating (/log/location/test-{{.TestName}}.log) instead of sprintf which would be safer (we'd know if interpolation failed), but I treat this as initial version we can iterate on and potentially break the convention as it's only used during test runs, not by users, so we don't need to be worried as much about deprecation etc.

Example

ls -lh ./*.log
-rw-r--r--  1 radeksimko  staff   592K Oct 14 16:49 tf-test-TestAccAWSBatchComputeEnvironment_createEc2.log
-rw-r--r--  1 radeksimko  staff   593K Oct 14 16:50 tf-test-TestAccAWSBatchComputeEnvironment_createEc2WithTags.log
-rw-r--r--  1 radeksimko  staff   392K Oct 14 16:58 tf-test-TestAccAWSBatchComputeEnvironment_createEc2WithoutComputeResources.log
-rw-r--r--  1 radeksimko  staff   597K Oct 14 16:51 tf-test-TestAccAWSBatchComputeEnvironment_createSpot.log
-rw-r--r--  1 radeksimko  staff   396K Oct 14 17:00 tf-test-TestAccAWSBatchComputeEnvironment_createSpotWithoutBidPercentage.log
-rw-r--r--  1 radeksimko  staff   591K Oct 14 16:52 tf-test-TestAccAWSBatchComputeEnvironment_createUnmanaged.log
-rw-r--r--  1 radeksimko  staff   594K Oct 14 16:59 tf-test-TestAccAWSBatchComputeEnvironment_createUnmanagedWithComputeResources.log
-rw-r--r--  1 radeksimko  staff   964K Oct 14 16:58 tf-test-TestAccAWSBatchComputeEnvironment_updateComputeEnvironmentName.log
-rw-r--r--  1 radeksimko  staff   961K Oct 14 16:56 tf-test-TestAccAWSBatchComputeEnvironment_updateInstanceType.log
-rw-r--r--  1 radeksimko  staff   941K Oct 14 16:54 tf-test-TestAccAWSBatchComputeEnvironment_updateMaxvCpus.log
-rw-r--r--  1 radeksimko  staff   112K Oct 14 17:01 tf-test-TestAccAWSBatchJobDefinition_basic.log
-rw-r--r--  1 radeksimko  staff   199K Oct 14 17:01 tf-test-TestAccAWSBatchJobDefinition_updateForcesNewResource.log
-rw-r--r--  1 radeksimko  staff   560K Oct 14 17:03 tf-test-TestAccAWSBatchJobQueue_basic.log
-rw-r--r--  1 radeksimko  staff   882K Oct 14 17:05 tf-test-TestAccAWSBatchJobQueue_update.log

Future

I reckon we might have more advanced/formalised tracing and logging which would allow the user to send logs with more context somewhere they can process them more efficiently and easily in the future. This is IMO much bigger project and this patch isn't supposed to address that.

[apparentlymart]

Seems like a reasonable idea to me!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.