Investigate upgrade solutions (yet again)

[jku]

As far as I can tell dependendabot cannot be configured to upgrade pinned dependencies (requirements/*.txt) as the dependencies have new releqses: the versioning-strategy configuration only applies to the constrained dependencies (requirements/*.in) and requirements/*.txt will never be upgraded unless the constraints force an upgrade.

#215 is a practical problem right now but in general I don't think we should be running potentially very old software just because nothing explicitly requires newer versions.

Let's consider a workflow that runs uv pip compile --upgrade --generate-hashes and files a PR periodically

[jku]

actually @DK96-OS in the original issue may be right: dependency-type: all maybe does the trick?

[jku]

So I tested dependency-type: all (using the situation before #223 as a test case).

It half works but I am just terrified by how the sausage is made:

• there's loads of internal dependabot errors still in the output, three dependencies still fail to update (including pydantic). Dependabot of course doesn't make the errors visible anywhere except hidden in Insights->Dependency Graph->Dependabot...
• dependabot ends up running pip-compile once for each transitive dependency:
  • this means invoking pip-compile 100 times, altogether this takes 14 minutes and produces 12000 lines of output
  • the whole process makes 3996 http requests to pypi 🤯

That said, when dependencies are roughly up-todate, the process is not this chaotic so maybe we could consider dependency-type: all?