proposal: "junction" nodes to repackage multiple pieces of Freight as a single piece of Freight

[aayushsrivastava]

Checklist

• I've searched the issue queue to verify this is not a duplicate feature request.
• I've pasted the output of kargo version, if applicable.
  • kargo version: v1.1.2
• I've pasted logs, if applicable.

Proposed Feature

I want to select multiple freight from different freight timelines when creating a promotion to a stage. Currently, a promotion can only accommodate one freight.

Motivation

I am trying to create a CD pipeline for a Team to deploy their microservices. Each of the microservices is a Kargo Warehouse ¹.

Changes to multiple microservices are accumulated in a uat stage and released together to the prod stage in a regular cadence.

During the manual promotion to prod, it is cumbersome to do promotions freight-by-freight. Since we want to also review the changes before updating the live GitOps branch, we will use git-open-pr and git-wait-for-pr in the promotion steps for the prod stage. This makes it worse since Kargo takes up to 5 minutes to confirm and move from a git-wait-for-pr step. With multiple freight, it takes up to (5 x N) minutes to complete the deployment of N microservices because of the delay in git-wait-for-pr step. There would also be N PRs to review.

If multiple Freight could be promoted together in one Promotion, we would only raise one PR and the delay in git-wait-for-pr step will also only take 5 minutes instead of (5 x N) minutes.

[1] One may suggest we could use a single Freight with all images to work around this. The reason I am modeling microservice to Warehouse 1:1 is that it is easy to manually create a freight for "hotfixes". If I had instead created a single Warehouse that subscribes to all the images, a hotfix would require specifying the expected versions of every image during manual Freight Assembly which is cumbersome and mistake-prone.

Suggested Implementation

The Promotion CR could accept a list of freight instead of a single freight. That would be a breaking change for the CRD though.

There might be good reasons for only allowing a single freight update per promotion, curious to know the why if that's the case.

[aayushsrivastava]

I thought more about the problem mentioned in the "Motivation" section, and how I could best solve it with the present feature set of Kargo.

I could create multiple "parallel" DAGs in one project. One DAG for each microservice. This would help avoid the "forced serialization" of microservice promotions which happens due to shared DAG.

I would still like to raise one PR, if possible. Wondering if it's safe for the same logical stages in different DAGs to push to a single shared branch (eg. a branch called promotion-request/prod) so that I can raise a single PR to be merged into the stage-specific branch. I don't see any problems since Kargo performs a git pull --rebase during the git-push promotion step and the git-open-pr step also seems to adopt existing PRs on the same branch on gitlab. The combination of the two makes me feel it is safe to do multiple promotions in parallel as long as they are modifying different paths on the git repository.

There are a few UX changes that would still be needed to make the multiple DAGs pattern suitable for general-use.

1. The image history grid should only show the stages from the DAG that deals with the image. Currently, it shows every single stage in a Kargo project which is a problem because the space is limited for that grid. I will probably create another issue for this.
2. The slow scroll speed makes it challenging to perform operations on "parallel" DAGs. Issue: Kargo UI always centers and has slow scroll speed #3161

[krancour]

My typical advice is that if you have artifacts that must (for one reason or another) move through a pipeline as a unit, then those artifacts belong in a single piece of Freight. i.e. Should be created by the same Warehouse, and then the problem of needing to do n > 1 Promotions to effect a single logical state change goes away...

Changes to multiple microservices are accumulated in a uat stage and released together to the prod stage in a regular cadence.

The way you've worded this, I'm inferring that you actually don't care about the artifacts for various microservices progressing through the pipeline as a unit all the way up to and including UAT, and then for the "last mile" to Production, you do want to Progress all those artifacts as a single unit.

Have I got that right?

Although we do not seem to have an open issue for it, this exact use case has been on our radar for a long time, and you can see that reflected in places like stage.spec.requestedFreight where the kind of Freight you're looking for is expressed with an "origin." At present, a Warehouse is the only place from which Freight can originate, but we built this way because we anticipated more types of origins in the future -- and in particular, we've anticipated a "junction" whose purpose would be to "repackage" the contents of two or more pieces of Freight as a new, single piece of Freight so that all the artifacts referenced therein progress from that point forward as a unit.

The way I see something like this working: A junction would be as if a Warehouse and a Stage "had a baby," in that it would behave partly like one and partly like the other. Like a Stage, it could request Freight from multiple origins (Warehouses), and acceptable sources for those (i.e. upstream Stages) and you could "promote" Freight from any of those Warehouses to it. Unlike a Stage, it wouldn't deploy anything. Like a Warehouse, it would produce new Freight (in this case, by combining all its current Freight) either automatically or upon request.

As I said, this has been in the back of our heads for a long time, and your use case is the exact one for which we'd conceived this. Do you agree this would solve your problem?

This is, unfortunately, not trivial or we'd have done it already. In terms of size and complexity, this is the sort of feature that would eat up one entire development cycle all by itself.

We'll have to see how @jessesuen thinks this ought to be prioritized.

Edit: I'm tentatively changing the title to match this line of thinking, but can adjust it further if the thread turns in a different direction.

[Brightside56]

[1] One may suggest we could use a single Freight with all images to work around this. The reason I am modeling microservice to Warehouse 1:1 is that it is easy to manually create a freight for "hotfixes". If I had instead created a single Warehouse that subscribes to all the images, a hotfix would require specifying the expected versions of every image during manual Freight Assembly which is cumbersome and mistake-prone.

I know it's not so flexible as spaceship implementation with repackaging, but #2988

[krancour]

I don't think loading up your Freight with a large number of artifacts is the answer. There have been a few other threads about this recently. I see a lot of people doing this and it's finally setting in that this isn't necessarily what people want, rather it's what people are getting stuck with when the patterns to solve their problems aren't clear. That's a gap we're going to be working very hard to address now.