Skip to content

Add multi-resolve functions to resolvers#2072

Merged
supakeen merged 8 commits intoosbuild:mainfrom
achilleas-k:resolve-all
Jan 7, 2026
Merged

Add multi-resolve functions to resolvers#2072
supakeen merged 8 commits intoosbuild:mainfrom
achilleas-k:resolve-all

Conversation

@achilleas-k
Copy link
Member

This PR adds convenience functions to the package, container, and ostree resolvers. This is an old branch I had sitting around that was following up from the original introduction of the manifestgen package. While working on Add proxy support to image-builder-cli (HMS-9645), I remembered that I wanted to move some of the conveniences of manifestgen into the resolver packages themselves and this is part of that.

Some background

The general flow of resolving content to generate a manifest is:

  1. Instantiate manifest based on image type with customizations.
  2. Get unresolved content descriptions (packages, containers, ostree commits) from the manifest, indexed by pipeline name.
  3. Resolve each content type using an appropriate resolver, preserving the pipeline name indexing.
  4. Serialize the manifest using the resolved content.

In (simplified) code, this looks like:

// Instantiate manifest
manifest := imgType.Manifest(blueprint, imageOptions, repos, rngSeed)

// Get unresolved content descriptions
packageSets := manifest.GetPackageSetChains()
containers := manifest.GetContainerSourceSpecs()
commits := manifest.GetOSTreeSourceSpecs()

// Resolve each content type
solver := depsolvednf.NewSolver(...)
depsolvedSets := make(map[string]depsolvednf.DepsolveResult)
for pipeline, packageSet := range packageSets {
	depsolvedSets[pipeline] = solver.Depsolve(packageSet)
}

// ... repeat for containers and ostree commits

serializedManifest := manifest.Serialize(depsolvedSets, resolvedContainers, resolvedCommits)

NOTE: This isn't entirely accurate code, but is simplified to emphasise the important parts (e.g. error return values are dropped).

The manifestgen.Generator.Generate function implements this flow, using the default configurations for each resolver and allows overriding them when needed.

New stuff

This PR only provides the ResolveAll() convenience which performs the iteration over the unresolved content indexed by the pipeline names. This moves some of the convenience provided by manifestgen.Generator.Generate() into the resolvers themselves, making them useful in cases where manifestgen isn't practical (e.g. in osbuild-composer where content is resolved on different workers).

Container resolvers

For a while now, we've had two container resolver implementations, an async one (the original) and a blocking resolver (introduced in #1176). There was a short discussion on that PR about the differences between the two. I summarised the consequences of switching to a blocking resolver in this comment. I'd like to prioritise consistency here over potential performance, but if we think we would benefit from parallel container resolution, we could go back to only having the async resolver but make its interface the same as the other resolvers (with Resolve() and ResolveAll()).

@achilleas-k achilleas-k requested a review from a team as a code owner December 5, 2025 13:51
@achilleas-k achilleas-k force-pushed the resolve-all branch 2 times, most recently from 8150679 to 26d0115 Compare December 5, 2025 14:00
@achilleas-k
Copy link
Member Author

Note that full (resolved) manifests have changed because the order of containers in the sources changes when using a different resolver. This isn't a functional change and doesn't affect the manifest ID.

@achilleas-k

This comment was marked as outdated.

@achilleas-k achilleas-k marked this pull request as draft December 5, 2025 17:26
@achilleas-k

This comment was marked as outdated.

A convenience function that runs Depsolve() on each package set slice
and returns a map of results with the same keys.

This is a follow-up to 5abe1d5 which
introduced the DefaultDepsolver() function in manifestgen.  It
essentially moves the core of that function into the dnfjson package.
@achilleas-k achilleas-k marked this pull request as ready for review December 11, 2025 09:34
bcl
bcl previously approved these changes Dec 12, 2025
Copy link
Contributor

@bcl bcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, agree with @lzap's comments.

thozza
thozza previously approved these changes Dec 17, 2025
Copy link
Member

@thozza thozza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I would not spend too much time optimizing or discussing SBOMs in the Solver, because at the end of the day, Solver won't be the place that will handle SBOMs in the future. All the SBOM code will be eventually removed from the Solver.

Comment on lines 337 to 338
solver := NewSolver("platform:el9", "9", "x86_64", "rhel9.0", tmpdir)
solver.SBOMType(tc.sbomType)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love us to move away from this pattern of constructing an instance just to modify it in the next step. Ideally moving towards the functional options pattern.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the functional options you showed me the other day looks really nice.

Add a new property to the solver for the SBOM type.  This simplifies the
signatures of DepsolveAll().  Ideally we would make the same change for
Depsolve(), but that would be a breaking API change and we don't want to
be too disruptive now.
Move manifestgen.DefaultCommitResolver() to ostree.ResolveAll().

The convenience function in manifestgen only calls the resolve function
in a nested loop, so let's make it part of the ostree package where the
original resolver and all ostree related types and functions are
defined.
Extend the Resolver interface to implement Resolve() and ResolveAll().
This is consistent with the API of the other resolvers.

The blockingResolver implements Resolve() as its main resolver function.
The Add() function now wraps Resolve() and collects the resolved specs
in the results slice.  This is the inverse of the asyncResolver where
Resolve() wraps Add().
ResolveAll() calls Resolve() iteratively on all container source specs.
Add a 1 minute timeout to each resolve call.
@achilleas-k achilleas-k dismissed stale reviews from thozza and bcl via d115985 January 5, 2026 16:30
@achilleas-k achilleas-k requested a review from lzap January 6, 2026 09:47
Copy link
Contributor

@lzap lzap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really like us utilizing context more, while it is fine adding more fire-grained timeouts you could have an overall deadline as well. Also, contexts are great for logging contexts, granted, this is more relevant for the service (per request, per org) than CLI but it has its own use there as well.

Anyways, all is fine.

Copy link
Contributor

@bcl bcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@supakeen supakeen added this pull request to the merge queue Jan 7, 2026
Merged via the queue into osbuild:main with commit 5b097d7 Jan 7, 2026
25 checks passed
@achilleas-k achilleas-k deleted the resolve-all branch January 8, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants