Allow building only desired chunks

[kylos101]

Is your feature request related to a problem? Please describe

Inspecting how a new test is running requires building all chunks and running all tests, which makes for a long dev loop.

Describe the behaviour you'd like

Provide a means to build and run a subset of chunks (which could also be a single chunk)

Describe alternatives you've considered

Create a new branch to do testing from main, just with the chunks I want to build and test.
Once the tests are working, copy them into yet another new branch created from main.

Additional context

This should help folks more easily test, which will hopefully encourage more authoring of tests.

[princerachit]

Dazzle already has an ignore flag at project level that can be used to ignore chunks. This issue is requesting same feature through addition instead of negation.

e.g. command

dazzle project ignore dep-cacert-update

results in the following section being added to dazzle.yml file.

combiner:
  combinations: []
ignore:
- dep-cacert-update

While building, the chunks mentioned here are ignored.

https://github.com/gitpod-io/dazzle/blob/87d48f997df4ce37c97cc4169b4908e4ad2939a7/pkg/dazzle/project.go#L200-L202

I think it is better to reuse this flag than adding another one. what are your thoughts @kylos101 @Furisto ?

[Furisto]

@princerachit I was not aware of that flag. That's great!. We could write a simple script like
for file in $(ls chunks) ; do dazzle project ignore "$file"; done
to add every chunk. Just have to remove the one that you are currently working on.

When I do that and run dazzle-up.sh the build process seems to be stuck in some kind of endless loop where it builds the same thing over and over again.

Edit: Actually this might have just been different variants.

[kylos101]

That could work, @princerachit , nice find! As @Furisto suggested, we could update the script (as needed) to remove chunks we want to work with.

[kylos101]

That could work, @princerachit , nice find! As @Furisto suggested, we could update the script (as needed) to remove chunks we want to work with.

Ignore works as advertised. 🙂

We could change dazzle-up.sh so that:

1. It takes an array of chunks as input that we intend to build and combine
2. It builds an ignore list for all chunks, excluding the ones supplied as input
3. Build "as is" and Combine (not using --all, but instead using the input chunks). @Furisto this might explain why you saw the long combine time?
4. Remove the ignore list on error or complete (using yq, will need to be added to the docker image)

Finally, when reviewing PRs, we'd just have to make sure we don't accidently include the ignore.

WDYT? @Furisto and @princerachit ?

[Furisto]

@kylos101 Makes sense, will adapt the script

[kylos101]

Thanks, @Furisto ! Adding a brief mention in our contribution guide might help too. For example, if the user wants to alter tool-postgresql, etc.

[kylos101]

@Furisto given this:

Build "as is" and Combine (not using --all, but instead using the input chunks). @Furisto this might explain why you saw the long combine time?

For the input args, we might want two things:

1. an array of chunks (to be used for dazzle build)
2. a combination name (to be used for dazzle combine)

Why? Combination definitions in the dazzle.yaml sometimes include reference combination(s), as well as chunks. Assuming we just want to combine the chunks is invalid, best to use the definition in the dazzle.yaml file.

I didn't realize bash has a getopts function! Take a look at what @aledbf did here. Might make sense to use for dazzle-up.sh, too.

[kylos101]

@Furisto I am going to transfer this to our workspace-images repo

[kylos101]

@princerachit , if you could take on this issue, that would help us get very close to completing this related epic. Let us know?

@Furisto I'm not sure if you have any script prepared for this already? If you do, can you share?