Pulling from Artifactory doesn't seem to always work

[grepwood]

I've searched for threads, but there's only #451 and #375 but none of those issues are related to the kind of troubles I'm dealing with.

I'm migrating from rules_docker to rules_oci and I have stumbled upon issues with oci_pull in conjunction with the Artifactory instance we use at work.

We have such code imported into our WORKSPACE and executed, to ensure all microservices have their base images ready before we build them:

load("@rules_oci//oci:pull.bzl", "oci_pull")
load("//nginx:image.bzl", "define_nginx_image")
...
load("//operations/fake-ldap:image.bzl", "define_ldap_image")

def download_docker_base_images():
    oci_pull(
        name = "rhel",
        tag = "9.5",
        image = "artifactory.at.work/internal-docker-images/rhel"
        digest = "sha256:..."
    )
    ...
    define_nginx_image()
    define_ldap_image()
)

The file operations/fake-ldap/image.bzl is as such:

load("@rules_oci//oci:pull.bzl", "oci_pull")

def define_ldap_image():
    oci_pull(
        name = "ldap",
        tag = "1.5.0",
        image = "artifactory.at.work/external-docker-images/osixia/openldap",
        digest = "sha256:..."
    )

Now what I get when I try to build all of my targets all at once, it fails to download the manifest only for this image. The rhel and nginx and all images in between are fine. Here's what's going on:

(time) WARNING: Download from https://artifactory.at.work/v2/external-docker-images/osixia/openldap/manifests/sha256:... failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 400 Bad Request

What's strange is that I download the image just fine with this command:

docker pull artifactory.at.work/external-docker-images/osixia/openldap:1.5.0

What did I do wrong?

[thesayyn]

It's hard to tell what went wrong but the pulling the image works with docker pull and doesn't with rules_oci, then its a bug in rules_oci. It would be great to have a repro of this so we can fix it.

[grepwood]

Alright, I found something.
The version of Artifactory I have at work doesn't seem to fully support v2 Docker registry API for remote registries - only for local registries.
I tried to curl for https://artifactory.at.work/v2/external-docker-images/osixia/openldap/manifests/1.5.0 and got a very peculiar response:

• HTTP return code 400
• declared type: application/json
• actual text content of the response is not a JSON, it's Could not fetch manifests

But when I tried to curl the endpoint https://artifactory.at.work/v2/internal-docker-images/rhel/manifests/9.5, it worked like a charm.

I worked around this issue by pulling the image into my local cache via docker pull, retagged it with docker tag and pushed it into the internal repo we have at the Artifactory with docker push. The Artifactory is still aware that it has all the layers of this image cached somewhere, as each layer produced Layer exists upon docker push.

I think this isn't a problem that rules_oci is responsible for, because Artifactory should produce the manifest regardless if the image lives in our secluded network or on the Moon.

I'll close the issue with this comment but one thing is going to bug me - how different is oci_pull to container_pull, that rules_docker's container_pull managed to pull that image despite the API being broken 🤔