Support for autoscale for HDInsight clusters

[kosinsky]

Adds autoscale (capacity and scheduled) support for HDInsight clusters. Following resources were updated :

• azurerm_hdinsight_hadoop_cluster
• azurerm_hdinsight_hbase_cluster (scheduled only)
• azurerm_hdinsight_interactive_query_cluster
• azurerm_hdinsight_spark_cluster

Others weren't modified, because:

• Spark doesn't support autoscale
• Storm and ML/RServer cluster types are deprecated and will retired at the end of 2020

Capacity based autoscale:

      autoscale {
        capacity {
          min_instance_count = 2
          max_instance_count = 3
        }
      }

Scheduled autoscale:

      autoscale {
        recurrence {
          timezone  = "Pacific Standard Time"          
          schedule {
            days      = ["Monday"]
            time      = "10:00"
            target_instance_count = 5
          }
          schedule {
            days      = ["Saturday", "Sunday"]
            time      = "10:00"
            target_instance_count = 3
          }
        }
      }

Tests:

make testacc TEST='./azurerm/internal/services/hdinsight/tests/' TESTARGS='-run=autoscale -parallel=3' TESTTIMEOUT='300m'
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
TF_ACC=1 go test ./azurerm/internal/services/hdinsight/tests/ -v -run=autoscale -parallel=3 -timeout 300m -ldflags="-X=github.com/terraform-providers/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccAzureRMHDInsightHadoopCluster_autoscale
=== PAUSE TestAccAzureRMHDInsightHadoopCluster_autoscale
=== RUN   TestAccAzureRMHDInsightHBaseCluster_autoscale
=== PAUSE TestAccAzureRMHDInsightHBaseCluster_autoscale
=== RUN   TestAccAzureRMHDInsightInteractiveQueryCluster_autoscale
=== PAUSE TestAccAzureRMHDInsightInteractiveQueryCluster_autoscale
=== RUN   TestAccAzureRMHDInsightSparkCluster_autoscale
=== PAUSE TestAccAzureRMHDInsightSparkCluster_autoscale
=== CONT  TestAccAzureRMHDInsightHadoopCluster_autoscale
=== CONT  TestAccAzureRMHDInsightSparkCluster_autoscale
=== CONT  TestAccAzureRMHDInsightInteractiveQueryCluster_autoscale
--- PASS: TestAccAzureRMHDInsightInteractiveQueryCluster_autoscale (1480.93s)
=== CONT  TestAccAzureRMHDInsightHBaseCluster_autoscale
--- PASS: TestAccAzureRMHDInsightHadoopCluster_autoscale (1853.24s)
--- PASS: TestAccAzureRMHDInsightSparkCluster_autoscale (2087.36s)
--- PASS: TestAccAzureRMHDInsightHBaseCluster_autoscale (1318.89s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/hdinsight/tests     2800.965s

Fixes #6470

[jackofallops]

Hi @kosinsky

Thanks for this PR. It's looking good. We've done some refactor/tidying work that have moved some of the helper code around, so this will need a rebase (it should do so cleanly I think). There's one change I think will need to be made after that (commented below). Once that's done I'll run the CI tests and I think this should be fine as is. There's an additional question on the bug you mention which would be good to have an answer on.

Thanks again.

[kosinsky]

Merge and post merge refactoring reactions are done. Please, take a look

[tombuildsstuff]

hey @kosinsky

Thanks for pushing those changes :)

I've taken a look through and left some comments inline but this is looking good - if we can fix up the comments then this should otherwise be good to merge 👍

Thanks!

[kosinsky]

@tombuildsstuff @jackofallops I addressed your feedback. Sorry for delay.

[sseppola]

@kosinsky thank you for all this work! It's much appreciated.

In case this was drowned in other notifications and forgotten about I wanted to leave a comment in hope it attracts some attention to a nearly finished PR :)

[kosinsky]

@tombuildsstuff @jackofallops Could you take one more look? I addressed your feedback. Please, let me know if I missed something or you have additional one.

[mbfrahry]

Hey @kosinsky! I got a chance to look over this and it looks good! The only issue is that it's quite a bit behind...through no fault of your own. Once it's back up to speed, we can get it merged quickly.

[katbyte]

@kosinsky - going to close this for #11547 - Thanks for getting it started!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.