Inconsistency of terraform code for jfrog artifactory provider for federated repos.

[railoni]

We are using below version of providers.

terraform {
  required_providers {
    artifactory = {
      source  = "jfrog/artifactory"
      version = "9.7.0"
    }
    project = {
      source  = "jfrog/project"
      version = "1.3.3"
    }
    xray = {
      source  = "jfrog/xray"
      version = "2.0.1"
    }
    random = {
      source = "hashicorp/random"
      version = "3.5.1"
    }
  }
}

We are running terraform code against 2 artifcatory server ha1 and ha2. We have federated repos in each side and they are in sync.

Whenever we apply the terraform code to provision projects, local, remote, virtual and federated repos.

Projects were not getting assigned to federated repos, so we have used local-exec artifactory api call to set project for repositories.

But ever we reapply the terraform code on same. Projects were getting removed for all the repos.

Please help us resolving this inconsistency of terraform code artifactory.

Here is the code for one of our federated repo.

resource "artifactory_federated_maven_repository" "maven-bu-prod-fed_ha1" {
  depends_on = [artifactory_federated_maven_repository.maven-bu-np-fed_ha1]
  count                = var.maven_enable ? 1 : 0
  key = "${var.business_unit}-maven-prod"
  project_key = project.BU_project_HA2.key
  xray_index                      = true
  #cleanup_on_delete = true

  member {
    enabled = true
    url     = "${var.artifactory-ha1_url}artifactory/${var.business_unit}-maven-prod"
  }

  member {
    enabled = true
    url     = "${var.artifactory-ha2_url}artifactory/${var.business_unit}-maven-prod"
  }
  checksum_policy_type            = "client-checksums"
  handle_releases                 = true
  handle_snapshots                = false
  suppress_pom_consistency_checks = false
  project_environments = ["PROD"]
  provider = artifactory.ha1

  provisioner "local-exec" {
    command = <<-EOT
      exec curl --location --request PUT '${var.artifactory-ha2_url}access/api/v1/projects/_/attach/repositories/${project.BU_project_HA2.key}-maven-prod/${project.BU_project_HA2.key}?force=true/false' --header 'Authorization: Bearer ${var.artifactory-ha2_token}'
    EOT
  }
 
  provisioner "local-exec" {
    command = <<-EOT
      exec curl -u 'admin:${var.ha2_password}' -X POST ${var.artifactory-ha2_url}artifactory/api/repositories/${project.BU_project_HA2.key}-maven-prod -H 'Content-Type: application/json' -d '{"environments":["PROD"]}'
    EOT
  }

}

resource "null_resource" "destroy_prod_maven" {

  triggers = {
    artifactory-ha2_url = var.artifactory-ha2_url
    projectKey          = project.BU_project_HA2.key
    ha2_password        = var.ha2_password
  }
  provisioner "local-exec" {
    when     = destroy
    command = <<-EOT
      exec curl -u 'admin:${self.triggers.ha2_password}' -X DELETE ${self.triggers.artifactory-ha2_url}artifactory/api/repositories/${self.triggers.projectKey}-maven-prod
    EOT
  }
}

[alexhung]

@railoni First, I notice this force=true/false which is incorrect. It should have either true or false value in the query param. Not sure if it is related but just want to point out.

[alexhung]

@railoni Can you share the configuration for project resource? I am curious what you have there w.r.t. repos.

[railoni]

Hi @alexhung Please find the attached file for project resource.
project.txt

[alexhung]

@railoni This issue will be eliminated when I add new resource project_repository to allow separate management of projects and repos. This is scheduled in beginning of Q2.

[alexhung]

Related #105

[railoni]

Thank you for the update Alex. currently we are using below lifecyle in project resources as a temporary solution.
lifecycle { ignore_changes = [repos] }

[alexhung]

@railoni Let me know if the new version solves the issue.

[railoni]

Hi @alexhung getting below error while intializing with below provider versions.

Also Could you please provide some examples for us. as we create project first and create repos next and add project_key = project.BU_project_HA2.key in resource "artifactory_federated_nuget_repository"

│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider jfrog/project: no available releases match the given constraints 1.3.3, 1.5.0
╵
╷
│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider jfrog/artifactory: no available releases match the given constraints 9.7.0, 10.3.0

[alexhung]

Error: Failed to query available provider packages

@railoni This error typically happens when there's already a copy of the provider binary in your system path. I get it whenever I try to install a copy of my provider from the registry whilst a locally built binary also exists in my system.

It may be caused by other reasons in your case but this is not a bug of the provider.

[alexhung]

@railoni Here's the barebone Terraform configuration that works on my environment.

terraform {
  required_providers {
    artifactory = {
      source  = "jfrog/artifactory"
      version = "10.3.1"
    }

    project = {
      source  = "jfrog/project"
      version = "1.5.1"
    }
  }
}

provider "artifactory" {
}

provider "project" {
}

resource "project" "my-project" {
  key = "myproj"
  display_name = "My Project"

  admin_privileges {
    manage_members   = true
    manage_resources = true
    index_resources  = true
  }
}

resource "artifactory_federated_nuget_repository" "alexh-nuget-federated-1" {
  key         = "${project.my-project.key}-alexh-nuget-1"
  project_key = project.my-project.key

  member {
    url     = "http://localhost.charlesproxy.com:8082/artifactory/${project.my-project.key}-alexh-nuget-1"
    enabled = true
  }
}

[alexhung]

@railoni Here's an example that use the new project_repository resource instead of project_key:

terraform {
  required_providers {
    artifactory = {
      source  = "jfrog/artifactory"
      version = "10.3.1"
    }

    project = {
      source  = "jfrog/project"
      version = "1.5.1"
    }
  }
}

provider "artifactory" {
}

provider "project" {
}

resource "project" "my-project" {
  key = "myproj"
  display_name = "My Project"

  admin_privileges {
    manage_members   = true
    manage_resources = true
    index_resources  = true
  }
}

resource "artifactory_federated_nuget_repository" "alexh-nuget-federated-1" {
  key = "${project.my-project.key}-alexh-nuget-1"

  member {
    url     = "http://localhost.charlesproxy.com:8082/artifactory/${project.my-project.key}-alexh-nuget-1"
    enabled = true
  }

  lifecycle {
    ignore_changes = [ project_key ]
  }
}

resource "project_repository" "alexh-nuget-federated-1" {
  project_key = project.my-project.key
  key         = artifactory_federated_nuget_repository.alexh-nuget-federated-1.key
}

[railoni]

Hi @alexhung after removing the older provider version about to initilize.

We have 2 artifactory servers one is HA1 and other is HA2. We are creating federated repos and federating each other.
With the example you have provided. i am able to set the project for HA1 federated repo but not able to set project and environment for HA2 fed repo.
Below is the code that we are using to federated repos.

How to set project and environment for member in fed repos :
"${var.artifactory-ha1_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"

resource "artifactory_federated_gradle_repository" "gradle-remote-ha1" {
  depends_on = []
  key                             = "${project.BU_project_HA2.key}-gradle-remote-prod"
  description                     = "gradle remote prod repo"
  project_environments            = ["PROD"]
  max_unique_snapshots = 5
  project_key  = project.BU_project_HA2.key
  provider = artifactory.ha1
  xray_index                      = true
  member {
    enabled = true
    url     = "${var.artifactory-ha1_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"
  }

  member {
    enabled = true
    url     = "${var.artifactory-ha2_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"
  }

  lifecycle {
    ignore_changes = [project_key, project_environments]
  }

}

resource "project_repository" "gradle-remote-ha1" {
  project_key = project.BU_project_HA2.key
  key         = artifactory_federated_gradle_repository.gradle-remote-ha1.key
}

[alexhung]

How to set project and environment for member in fed repos :
"${var.artifactory-ha1_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"

@railoni I don't understand your question.

If you need to setup the other federated repo, you would specific the configuration for that repo with the corresponding project and environment. You would need to apply the second set of configuration on the other instance but other than that, there's nothing different from setting this up using the web UI.

It'd also be more helpful if you can provide details on what's not working for you. Error messages, outputs, etc. would help diagnose the issue.

I can only make general suggestion as I don't have intimate knowledge of your environment and the rest of your TF configurations.

[railoni]

Hi @alexhung I am getting below error.

│ Error:
│ 400 PUT https://artifactory-ha2_url/artifactory/api/repositories/base-gradle
│ {
│   "errors" : [ {
│     "status" : 400,
│     "message" : "Virtual repository must include only repositories from the same project it was created with\n"
│   } ]
│ }
│
│   with module.base.artifactory_virtual_gradle_repository.virtual_gradle_ha2,
│   on artifactory_base/gradle.tf line 142, in resource "artifactory_virtual_gradle_repository" "virtual_gradle_ha2":
│  142: resource "artifactory_virtual_gradle_repository" "virtual_gradle_ha2" {
│

Because one of the member in federated repo is not getting project and environment.

resource "artifactory_federated_gradle_repository" "gradle-remote-ha1" {
  depends_on = []
  key                             = "${project.BU_project_HA2.key}-gradle-remote-prod"
  description                     = "gradle remote prod repo"
  project_environments            = ["PROD"]
  max_unique_snapshots = 5
  project_key  = project.BU_project_HA2.key
  provider = artifactory.ha1
  xray_index                      = true
  member {
    enabled = true
    url     = "${var.artifactory-ha1_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"
  }

  member {
    enabled = true
    url     = "${var.artifactory-ha2_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"
  }

  lifecycle {
    ignore_changes = [project_key, project_environments]
  }

}

Do to set project and environment to it.

 member {
    enabled = true
    url     = "${var.artifactory-ha2_url}artifactory/${project.BU_project_HA2.key}-gradle-remote-prod"
  }

[alexhung]

@railoni IIRC you need another artifactory_federated_gradle_repository for the second member. Meaning, you need one artifactory_federated_gradle_repository configuration for federated repo on HA1, and another configuration for federated repo on HA2.

I haven't seen that from your snippet.

[railoni]

Hi @alexhung If i add another artifactory_federated_gradle_repository resource for the second member for HA2 getting into below errors.

Error:
│ 400 PUT https://artifactory-url//artifactory/api/repositories/base-maven-remote-prod
│ {
│ "errors" : [ {
│ "status" : 400,
│ "message" : "error when validating repository name: Case insensitive repository key already exists"
│ } ]
│ }
│
│ with module.base.artifactory_federated_maven_repository.remote-prod-ha2,
│ on artifactory_base/maven.tf line 206, in resource "artifactory_federated_maven_repository" "remote-prod-ha2":
│ 206: resource "artifactory_federated_maven_repository" "remote-prod-ha2" {
│
╵
╷
│ Error:
│ 400 PUT https://artifactory-url/artifactory/api/repositories/base-maven-shared-dev
│ {
│ "errors" : [ {
│ "status" : 400,
│ "message" : "error when validating repository name: Case insensitive repository key already exists"
│ } ]
│ }
│
│ with module.base.artifactory_federated_maven_repository.shared-dev-ha2,
│ on artifactory_base/maven.tf line 291, in resource "artifactory_federated_maven_repository" "shared-dev-ha2":
│ 291: resource "artifactory_federated_maven_repository" "shared-dev-ha2" {

[alexhung]

Case insensitive repository key already exists

The repo already exists on your Artifactory. Check that you don't have duplicate variables, loop, etc.