HDDS-5056. Avoid false-positive error messages during pipeline creations

[elek]

JIRA: https://issues.apache.org/jira/browse/HDDS-5056

What changes were proposed in this pull request?

BackgroundPipelineCreator in SCM tries to pipelines with regular intervals. This check is very simple, if new pipeline can be created, it will create them.

However, if the pipelines couldn't be created (which is very likely) an error message is printed out:

scm_1       | 2021-03-31 15:34:57,004 [RatisPipelineUtilsThread - 0] ERROR pipeline.PipelineManagerV2Impl: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed because nodes are engaged in other pipelines and every node can only be engaged in max 1 pipelines. Required 3. Found 0

Today it's printed out on ERROR level, but this message doesn't provide any new information I think the PipelineManager.createPipeline shouldn't use ERROR level especially as the exception is re-thrown:

    lock.lock();
   try {
     Pipeline pipeline = pipelineFactory.create(type, factor);
     stateManager.addPipeline(pipeline.getProtobufMessage(
         ClientVersions.CURRENT_VERSION));
     recordMetricsForPipeline(pipeline);
     return pipeline;
   } catch (IOException ex) {
     LOG.error("Failed to create pipeline of type {} and factor {}. " +
         "Exception: {}", type, factor, ex.getMessage());
     metrics.incNumPipelineCreationFailed();
     throw ex;
   } finally {
     lock.unlock();
   }

It should be the responsibility of the caller to log the exception or ignore it (in case of BackgroundPipelineCreator)

How was this patch tested?

Full CI tests.

[bharatviswa504]

+1 LGTM.
Thanks, @elek for taking care of this.

[elek]

Thanks the review @bharatviswa504 and @bshashikant. Build become green. I am merging it now..