Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gadget,tests: add support for volume-assignments in gadget #14563

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

Meulengracht
Copy link
Member

This will allow the gadget yaml to specify specific disk mappings when either installing or updating the system.

The specification relevant to this feature is here: docs

JIRA: SNAPDENG-32507

@github-actions github-actions bot added the Run Nested -auto- Label automatically added in case nested tests need to be executed label Oct 1, 2024
Copy link

codecov bot commented Oct 2, 2024

Codecov Report

Attention: Patch coverage is 77.61194% with 60 lines in your changes missing coverage. Please review.

Project coverage is 78.96%. Comparing base (96ea7b0) to head (22fc496).
Report is 29 commits behind head on master.

Files with missing lines Patch % Lines
osutil/disks/mockdisk.go 37.14% 16 Missing and 6 partials ⚠️
gadget/gadget.go 87.60% 8 Missing and 7 partials ⚠️
gadget/install/install.go 64.28% 8 Missing and 7 partials ⚠️
gadget/update.go 88.57% 4 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14563      +/-   ##
==========================================
+ Coverage   78.95%   78.96%   +0.01%     
==========================================
  Files        1084     1085       +1     
  Lines      146638   147263     +625     
==========================================
+ Hits       115773   116283     +510     
- Misses      23667    23739      +72     
- Partials     7198     7241      +43     
Flag Coverage Δ
unittests 78.96% <77.61%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Meulengracht Meulengracht force-pushed the feature/gadget-volume-assignment branch 2 times, most recently from 70df3d1 to a5c5e3f Compare October 3, 2024 13:48
@Meulengracht Meulengracht marked this pull request as ready for review October 4, 2024 07:16
Copy link
Contributor

@bboozzoo bboozzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some high level comments. The PR is a tad too long for a quick review.

gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
Copy link
Member

@alfonsosanchezbeato alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice! I made a first pass now.

gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
@bboozzoo
Copy link
Contributor

needs a rebase

@Meulengracht Meulengracht force-pushed the feature/gadget-volume-assignment branch from 078585f to a61b6a3 Compare October 30, 2024 11:27
Copy link
Contributor

@bboozzoo bboozzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks quite good, some minor comments

gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
gadget/install/install.go Outdated Show resolved Hide resolved
Comment on lines +1192 to +1204
switch {
case len(common) != len(newVolumes) && len(common) != len(oldVolumes):
// there are both volumes removed from old and volumes added to new
return fmt.Errorf("cannot update gadget assets: volumes were both added and removed")
case len(common) != len(newVolumes):
// then there are volumes in old that are not in new, i.e. a volume
// was removed
return fmt.Errorf("cannot update gadget assets: volumes were removed")
case len(common) != len(oldVolumes):
// then there are volumes in new that are not in old, i.e. a volume
// was added
return fmt.Errorf("cannot update gadget assets: volumes were added")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could use it as an opportunity to simplify, but since it's just moving preexeisting code it's ok to leave it as it is:

Suggested change
switch {
case len(common) != len(newVolumes) && len(common) != len(oldVolumes):
// there are both volumes removed from old and volumes added to new
return fmt.Errorf("cannot update gadget assets: volumes were both added and removed")
case len(common) != len(newVolumes):
// then there are volumes in old that are not in new, i.e. a volume
// was removed
return fmt.Errorf("cannot update gadget assets: volumes were removed")
case len(common) != len(oldVolumes):
// then there are volumes in new that are not in old, i.e. a volume
// was added
return fmt.Errorf("cannot update gadget assets: volumes were added")
}
if len(common) != len(newVolumes) || len(common) != len(oldVolumes) {
// there are both volumes removed from old and volumes added to new
return fmt.Errorf("cannot update gadget assets: differing volumes in new and old gadget")
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i left this as an exercise for a separate PR

tests/lib/nested.sh Show resolved Hide resolved
Copy link
Contributor

@bboozzoo bboozzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some nitpicks

gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/install/install.go Show resolved Hide resolved
@@ -343,10 +376,15 @@ func Run(model gadget.Model, gadgetRoot string, kernelSnapInfo *KernelSnapInfo,
return nil, err
}

volumes, err := determineDeviceVolumes(info)
if err != nil {
return nil, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or here

gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
@@ -324,6 +339,24 @@ func onDiskStructsSortedIdx(vss map[int]*gadget.OnDiskStructure) []int {
return yamlIdxSl
}

func determineDeviceVolumes(gi *gadget.Info) (map[string]*gadget.Volume, error) {
if len(gi.VolumeAssignments) != 0 {
devVols, err := gadget.VolumesForCurrentDeviceAssignment(gi)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if we changed the signature of the function a little such that we could have:

Suggested change
devVols, err := gadget.VolumesForCurrentDeviceAssignment(gi)
vols, explicitlyAssigned, err := gadget.VolumesForCurrentDevice(gi)
if explicitlyAssigned {
// log
}

were (vols, explicitlyAssigned) is (assigned volumes, true) if there was a match, otherwise it's simply (gi.Volumes, false) ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did something similar to this

gadget/update.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
Copy link
Member

@alfonsosanchezbeato alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Some comments, but seems to be almost there

tests/nested/manual/install-volume-assignment/task.yaml Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
gadget/gadget.go Show resolved Hide resolved
gadget/gadget.go Outdated Show resolved Hide resolved
Copy link
Member

@alfonsosanchezbeato alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes! I have one objection though, please see online comment.

Comment on lines +655 to +656
default:
return nil, fmt.Errorf("multiple matching volume-assignment for current device")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not error in this case, again, according to the spec we return the first match, but it does not say to error out if there is more than one match. I can see cases where allowing multiple matches would be useful, for instance a first assignment A that uses by-id and an assignment B that uses by-path, because the user has an special assignment A for a concrete unit but the rest use B, and in that case the special unit would match both assignments, so the order matters. And if we add more fields to decide on assignments this will start to make much more sense. In summary, I foresee vendors wanting to use this flexibility in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Run Nested -auto- Label automatically added in case nested tests need to be executed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants