Renovate composer dependency updates doesn't respect filtering

[timwsuqld]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab v19.56.3

Please tell us more about your question or problem

Composer supports filtering repositories so that you don't search a repository for dependencies that aren't there. https://getcomposer.org/doc/articles/repository-priorities.md#filtering-packages

We have a repository defined for some internal packages, but can see in the debug logs that renovate is attempting to hit API endpoints for many packages being updated. You can see in the below composer.json snippit, that only packages in the sua/* namespace should be searched for in the gitlab domain. But the debug logs show it searching for all packages in that repository.

    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/suaustralia/otrs-rest-api"
        },
        {
            "type": "vcs",
            "url": "https://github.com/suaustralia/kml-parser"
        },
        {
            "type": "vcs",
            "url": "https://github.com/suaustralia/httpful"
        },
        {
            "type": "composer",
            "url": "https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/packages.json",
            "only": [
                "sua/*"
            ]
        }
    ],

This doesn't prevent things from working for us, but it does mean that we are hitting the server for packages it'll never have. For some people it could break things though, is the only/exclude config options may be important to make sure that a dependency comes from the right repo.

Logs (if relevant)

Logs

DEBUG: Deleted cached dep updates (repository=mysu/suqld_intranet)
 INFO: Dependency extraction complete (repository=mysu/suqld_intranet, baseBranch=develop)
       "stats": {
         "managers": {
           "composer": {"fileCount": 1, "depCount": 121},
           "gitlabci": {"fileCount": 1, "depCount": 16},
           "npm": {"fileCount": 1, "depCount": 91},
           "pip_requirements": {"fileCount": 1, "depCount": 4}
         },
         "total": {"fileCount": 4, "depCount": 232}
       }
DEBUG: Setting npmrc (repository=mysu/suqld_intranet)
DEBUG: hostRules: no authentication for pypi.org (repository=mysu/suqld_intranet)
DEBUG: Using queue: host=pypi.org, concurrency=16 (repository=mysu/suqld_intranet)
DEBUG: Using queue: host=registry.npmjs.org, concurrency=999 (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/api-platform/core~dev.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=472) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/api-platform/core.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=473) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/beberlei/doctrineextensions~dev.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=753) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/beberlei/doctrineextensions.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=754) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/brick/money~dev.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=742) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/brick/money.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=743) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/bjeavons/zxcvbn-php.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=753) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/bjeavons/zxcvbn-php~dev.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=754) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/damienharper/auditor.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=238) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/damienharper/auditor~dev.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=239) (repository=mysu/suqld_intranet)
DEBUG: GET https://gitlab.domain.org.au/api/v4/group/92/-/packages/composer/p2/damienharper/auditor-bundle.json = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=149) (repository=mysu/suqld_intranet)
...

Reproduction repo https://github.com/timwsuqld/renovate-composer-filtering

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team

[rarkins]

Please provide a full composer.json example in a reproduction repo. If need be, the registry with filter can be a dummy address.

[timwsuqld]

Reproduction repo created, run on our local Gitlab runner as that was easiest for me to test.

You can see in the debug that config.composer.deps has registryUrls that doesn't preserve the exclusion filters, so it has no way of knowing to exclude specific registries for different packages.

                 "registryUrls": [
                   "https://version-control.it.su.org.au/api/v4/group/92/-/packages/composer",
                   "https://packagist.org"
                 ],

You can then seen requests like DEBUG: GET https://version-control.it.su.org.au/api/v4/group/92/-/packages/composer/p2/bjeavons/zxcvbn-php~dev.json which should never happen, as the package does not match the filter sua/*, so it should be skipping our custom registry URL.

[rarkins]

Please rewrite the "expected behavior" section of the reproduction as it's currently pointless (repeating the title). Also where are the "sua" packages in the reproduction?

[timwsuqld]

There doesn't need to be any sua/* packages in the reproduction, and that helps highlight the issue. At no point should renovate be sending requests to https://version-control.it.su.org.au/ as there are no packages that match the filter sua/*. If there are packages that match that filter, then only requests for those specific packages should be sent to that repository.

Renovate composer helper essentially needs to reimplement the filtering in composer (https://getcomposer.org/doc/articles/repository-priorities.md#filtering-packages).

[rarkins]

It's not a full reproduction because you could satisfy it by completely ignoring any registry which has a filter. A proper reproduction would have a dependency which matches and one which doesn't.