Harbor registry loaded due to digest validation for each image pull request

[gnr1227]

Team,

We submitted few hundred thousands of jobs using job scheduler at same time using singularity - ORASprotocol (these jobs lands on multiple host machines in the compute cluster using container images)

We observed that, several jobs failed with error message "failed with status code [manifests stable]: 401 Unauthorized"

Debugging the issue we found root cause to be due to Dragonfly making HEAD requests to Harbor for each job which caused to Harbor to reach max connection limit and throwing above error.

We need a solution to avoid Dragonfly going to Harbor for each job request (thereby avoiding exhausting max connection limit) and instead fetch the information using the Dragonfly cache.

Regards,
Narendra