-
Notifications
You must be signed in to change notification settings - Fork 79
Support uploading to OpenStack #1921
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| package openstack | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "io" | ||
| "os" | ||
|
|
||
| "github.com/osbuild/images/pkg/cloud" | ||
|
|
||
| "github.com/gophercloud/gophercloud/v2" | ||
| ostack "github.com/gophercloud/gophercloud/v2/openstack" | ||
| "github.com/gophercloud/gophercloud/v2/openstack/image/v2/imagedata" | ||
| "github.com/gophercloud/gophercloud/v2/openstack/image/v2/images" | ||
| ) | ||
|
|
||
| var _ = cloud.Uploader(&openstackUploader{}) | ||
|
|
||
| type openstackUploader struct { | ||
| image string | ||
| diskFormat string | ||
| containerFormat string | ||
| } | ||
|
|
||
| type UploaderOptions struct { | ||
| DiskFormat string | ||
| ContainerFormat string | ||
| } | ||
|
|
||
| func NewUploader(image string, opts *UploaderOptions) (cloud.Uploader, error) { | ||
| return &openstackUploader{ | ||
| image: image, | ||
| diskFormat: opts.DiskFormat, | ||
| containerFormat: opts.ContainerFormat, | ||
| }, nil | ||
| } | ||
|
|
||
| func (ou *openstackUploader) Check(status io.Writer) error { | ||
| return nil | ||
| } | ||
|
|
||
| func (ou *openstackUploader) UploadAndRegister(r io.Reader, uploadSize uint64, status io.Writer) (err error) { | ||
| fmt.Fprintf(status, "Uploading to OpenStack...\n") | ||
|
|
||
| opts, err := ostack.AuthOptionsFromEnv() | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to read OpenStack ENV variables. Please source the OpenStack RC file: %w", err) | ||
| } | ||
|
|
||
| // This is needed otherwise we get the following error when authenticating: | ||
| // You must provide exactly one of DomainID or DomainName to | ||
| // authenticate by Username | ||
| // Even with an RC file that works perfectly fine with `openstack token issue` | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems to be related to this issue: gophercloud/gophercloud#3440
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I linked the issue from within the code comment. |
||
| // See https://github.com/gophercloud/gophercloud/issues/3440 | ||
| // See https://github.com/gophercloud/gophercloud/issues/3240 | ||
| if opts.DomainName == "" { | ||
| opts.DomainName = os.Getenv("OS_USER_DOMAIN_NAME") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we double check that
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we should. If it is which is not ideal but it is better than failing because it is empty even though the upload might maybe work without that? I am not sure what happens on other OpenStack instances. But if you prefer to have the check, I can definitely add it. |
||
| } | ||
|
|
||
| ctx := context.Background() | ||
| provider, err := ostack.AuthenticatedClient(ctx, opts) | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to authenticate to OpenStack: %w", err) | ||
| } | ||
|
|
||
| client, err := ostack.NewImageV2(provider, gophercloud.EndpointOpts{ | ||
| Region: os.Getenv("OS_REGION_NAME"), | ||
| }) | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to initialize the client: %w", err) | ||
| } | ||
|
|
||
| createOpts := images.CreateOpts{ | ||
| Name: ou.image, | ||
| DiskFormat: ou.diskFormat, | ||
| ContainerFormat: ou.containerFormat, | ||
| } | ||
| img, err := images.Create(ctx, client, createOpts).Extract() | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to create the image metadata: %w", err) | ||
| } | ||
|
|
||
| err = imagedata.Upload(ctx, client, img.ID, r).ExtractErr() | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to upload the image: %w", err) | ||
| } | ||
|
|
||
| // This would glitch the progressbar, but once it gets fixed, we would | ||
| // like to print this message | ||
| // fmt.Printf("Created image: %s (ID: %s)\n", img.Name, img.ID) | ||
|
|
||
| return nil | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be nice to do some of the checks we do in UploadAndRegister here already. Not a blocker though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be honest, I am confused about the purpose of the
Checkmethod. My expectation was that if the check fails, we don't even try to upload. Which doesn't seem to be true. I return errors from myCheckandUploadAndRegistergets called anyway.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, interesting. Then I must misremember how this works, its fine for now, we can always refactor it. The idea was that we would run "Check()" before we do an expensive upload and then discover that some permissions to move the image into the right place are missing (iirc that can happen on AWS). But if you say its not working for you we will investigate. Its easy enough to move some of the checks here in a followup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, adding the checks later should be easy. Check this out.
The upload starts happening
And it's not a glitch because the "Uploading to OpenStack..." is there also. So we are definitely inside of the
UploadAndRegisterfunction.