Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(provisioning): extra format options (mkfs) added #335

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

mhkarimi1383
Copy link
Contributor

@mhkarimi1383 mhkarimi1383 commented Sep 19, 2024

Pull Request template

Please, go through these steps before you submit a PR.

Why is this PR required? What issue does it fix?:

adding extra format (mkfs) options to newly created Volumes

also switched from deprecated utils/monut to mount-utils to be able to add this feature and moving from an old package to a maintained one

What this PR does?:

Refactor to new mount utils package and change format and mount function

Does this PR require any upgrade changes?:

No, But requires document updates for StorageClasses

If the changes in this PR are manually verified, list down the scenarios covered::

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

  • Fixes Extra mkfs options #296
  • PR Title follows the convention of <type>(<scope>): <subject>
  • Has the change log section been updated?
  • Commit has unit tests
  • Commit has integration tests
  • (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
  • (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them:

@dsharma-dc
Copy link
Contributor

How is the new formatOptions going to be set by user? via storageclass? There is no mkfsOptions unlike mountOptions as provided by storageclass.
Please write a test as well to show how this is used.

@tiagolobocastro
Copy link

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

@mhkarimi1383
Copy link
Contributor Author

mhkarimi1383 commented Dec 5, 2024

Hi

formatOptions is added to MountInfo just like mountOptions

So, It will work if you add it to storageclass

@mhkarimi1383
Copy link
Contributor Author

Also there is no test for the mounter itself

@dsharma-dc
Copy link
Contributor

dsharma-dc commented Dec 6, 2024

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

@dsharma-dc
Copy link
Contributor

Also there is no test for the mounter itself

You can run this manually and document here the steps to set formatOptions.

@mhkarimi1383
Copy link
Contributor Author

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

It's my first time that I'm contributing to this project so I just looked for where mountOptions is handled to handle formatOptions too

@mhkarimi1383
Copy link
Contributor Author

Hello again

I'm using NixOS, and I can't run make, waiting on #351

@mhkarimi1383
Copy link
Contributor Author

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

I checked out more and I saw that we have to put this on where we are mounting volume for the first time, So there is no need to add this into GetVolAndMountInfo, we have to handle that just like mountOptions

@mhkarimi1383
Copy link
Contributor Author

Not able to test due to #353 and #352

Signed-off-by: Muhammed Hussein Karimi <[email protected]>
…ction

I had to upgrade some packages and switched to newer mount utils (currently used lib is depricated)

Signed-off-by: Muhammed Hussein Karimi <[email protected]>
Signed-off-by: Muhammed Hussein Karimi <[email protected]>
Signed-off-by: Muhammed Hussein Karimi <[email protected]>
Signed-off-by: Muhammed Hussein Karimi <[email protected]>
go.mod Outdated
@@ -1,6 +1,8 @@
module github.com/openebs/lvm-localpv

go 1.19
go 1.22.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is causing github actions to fail. Let's not update this until we uniformly upgrade to golang 1.22 throughout.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revert that

My go version was the cause of this, That's why I have contributed for Nix Shell :)

@dsharma-dc
Copy link
Contributor

@mhkarimi1383
Also, please refer to the change https://github.com/openebs/zfs-localpv/pull/542/files . Something similar is likely required here also for these formatOptions to be usable.
cc: @abhilashshetty04 @Abhinandan-Purkait

@mhkarimi1383
Copy link
Contributor Author

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

Signed-off-by: Muhammed Hussein Karimi <[email protected]>
@dsharma-dc
Copy link
Contributor

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

Would you like to do that as part of this PR? or instead try using a mount-utils that might work with go 1.19.

@mhkarimi1383
Copy link
Contributor Author

I can do both, whatever is Ok :)

@dsharma-dc
Copy link
Contributor

I can do both, whatever is Ok :)

Thanks, I'd say then try using a compatible mount-utils version(I hope no other issues pop up), and Go upgrade can be taken up as a new PR.

@mhkarimi1383
Copy link
Contributor Author

@dsharma-dc

Ok, I will revert my changes to go.mod, etc. And will change the package again

@mhkarimi1383
Copy link
Contributor Author

mhkarimi1383 commented Dec 22, 2024

I have fixed package versions and switched to a correct version of mount utils

And I was able to build project successfully and tests passed

@mhkarimi1383
Copy link
Contributor Author

Also why in Makefile we are using sudo to build images?
It will break systems with podman

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.22%. Comparing base (8feeaa0) to head (12087cf).
Report is 60 commits behind head on develop.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #335      +/-   ##
===========================================
- Coverage    98.66%   97.22%   -1.44%     
===========================================
  Files            2        2              
  Lines          673      902     +229     
===========================================
+ Hits           664      877     +213     
- Misses           5       18      +13     
- Partials         4        7       +3     
Flag Coverage Δ
bddtests 97.22% <ø> (-1.44%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mhkarimi1383
Copy link
Contributor Author

mhkarimi1383 commented Dec 22, 2024

Also getting

/usr/local/bin/lvm-driver: No such file or directory

after building and running built image

Both with sudo and without sudo while building...

@mhkarimi1383
Copy link
Contributor Author

mhkarimi1383 commented Dec 22, 2024

I think this is due to some shared libs
We could make go build to do everything statistically

As I understood that's because of my Operating System (NixOS), since it has different type of packaging/paths

@dsharma-dc
Copy link
Contributor

Also why in Makefile we are using sudo to build images? It will break systems with podman

Perhaps because Podman is pretty new compared to Docker, when this Makefile was written. We may want to change that at some point.

@dsharma-dc
Copy link
Contributor

@mhkarimi1383 Also, please refer to the change https://github.com/openebs/zfs-localpv/pull/542/files . Something similar is likely required here also for these formatOptions to be usable. cc: @abhilashshetty04 @Abhinandan-Purkait

@mhkarimi1383 Did you see this comment yet?

@mhkarimi1383
Copy link
Contributor Author

@mhkarimi1383 Also, please refer to the change https://github.com/openebs/zfs-localpv/pull/542/files . Something similar is likely required here also for these formatOptions to be usable. cc: @abhilashshetty04 @Abhinandan-Purkait

@mhkarimi1383 Did you see this comment yet?

Yes,

But since I'm a bit new to this project I need to run the project locally on my minikube setup
Due to shared libs problem I can't test it locally

@mhkarimi1383
Copy link
Contributor Author

It is OK to remove sudo and make builds static?
Or I have to do it in another PR?

@tiagolobocastro
Copy link

It is OK to remove sudo and make builds static? Or I have to do it in another PR?

Why is static builds required? Isn't having all required inputs in the nix-shell enough?

@mhkarimi1383
Copy link
Contributor Author

It is OK to remove sudo and make builds static? Or I have to do it in another PR?

Why is static builds required? Isn't having all required inputs in the nix-shell enough?

Paths are different there and I think because of that
I'm getting

/usr/local/bin/lvm-driver: No such file or directory

While running the built image

I think making builds static will make the binary file work everywhere :)

Just guessing :)
I think I have to give static build a try

@mhkarimi1383
Copy link
Contributor Author

After trying to do static link some libs are missing again, getting

ldd /usr/local/bin/lvm-driver
	/nix/store/9v5d40jyvmwgnq1nj8f19ji2rcc5dksd-glibc-2.37-45/lib/ld-linux-x86-64.so.2 (0x7f2f6b592000)
	libpthread.so.0 => /nix/store/9v5d40jyvmwgnq1nj8f19ji2rcc5dksd-glibc-2.37-45/lib/ld-linux-x86-64.so.2 (0x7f2f6b592000)
	libc.so.6 => /nix/store/9v5d40jyvmwgnq1nj8f19ji2rcc5dksd-glibc-2.37-45/lib/ld-linux-x86-64.so.2 (0x7f2f6b592000)
Error relocating /usr/local/bin/lvm-driver: __vfprintf_chk: symbol not found
Error relocating /usr/local/bin/lvm-driver: __fprintf_chk: symbol not found

@mhkarimi1383
Copy link
Contributor Author

I can't test this project when using NixOS, Using docker for build is a good option too

@mhkarimi1383
Copy link
Contributor Author

mhkarimi1383 commented Dec 29, 2024

For me problem solved by these changes

If it's OK, I will create a new PR to apply them (I don't know if you are Ok with using musl)

diff --git a/Makefile b/Makefile
index dda7432..27501c3 100644
--- a/Makefile
+++ b/Makefile
@@ -222,7 +222,7 @@ lvm-driver-image: lvm-driver
    @echo "+ Generating ${CSI_DRIVER} image"
    @echo "--------------------------------"
    @cp bin/${CSI_DRIVER}/${CSI_DRIVER} buildscripts/${CSI_DRIVER}/
-   cd buildscripts/${CSI_DRIVER} && sudo docker build -t ${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG} ${DBUILD_ARGS} . && sudo docker tag ${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG} quay.io/${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG}
+   cd buildscripts/${CSI_DRIVER} && docker build -t ${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG} ${DBUILD_ARGS} . && docker tag ${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG} quay.io/${IMAGE_ORG}/${CSI_DRIVER}:${IMAGE_TAG}
    @rm buildscripts/${CSI_DRIVER}/${CSI_DRIVER}
 
 .PHONY: ansible-runner-image
@@ -230,7 +230,7 @@ ansible-runner-image:
    @echo "------------------"
    @echo "--> Build ansible-runner image for lvm-localpv e2e-tests" 
    @echo "------------------"
-   sudo docker build . -f e2e-tests/Dockerfile -t ${IMAGE_ORG}/lvm-localpv-e2e:ci
+   docker build . -f e2e-tests/Dockerfile -t ${IMAGE_ORG}/lvm-localpv-e2e:ci
 
 .PHONY: ci
 ci:
diff --git a/buildscripts/build.sh b/buildscripts/build.sh
index a6e7be5..9b3374e 100755
--- a/buildscripts/build.sh
+++ b/buildscripts/build.sh
@@ -100,8 +100,9 @@ env GOOS=$GOOS GOARCH=$GOARCH go build -ldflags \
     "-X github.com/openebs/lvm-localpv/pkg/version.GitCommit=${GIT_COMMIT} \
     -X main.CtlName='${CTLNAME}' \
     -X github.com/openebs/lvm-localpv/pkg/version.Version=${VERSION} \
-    -X github.com/openebs/lvm-localpv/pkg/version.VersionMeta=${VERSION_META}"\
-    -o $output_name\
+    -X github.com/openebs/lvm-localpv/pkg/version.VersionMeta=${VERSION_META} \
+    -linkmode external -w -extldflags '-static'"\
+    -a -o $output_name\
     ./cmd
 
 echo ""
diff --git a/shell.nix b/shell.nix
index d070f65..58acd8d 100644
--- a/shell.nix
+++ b/shell.nix
@@ -4,7 +4,9 @@ let
 in
 pkgs.mkShell {
   name = "scripts-shell";
+
   buildInputs = with pkgs; [
+    nix-ld                # Enable nix-ld linker
     chart-testing
     ginkgo
     git
@@ -16,7 +18,14 @@ pkgs.mkShell {
     minikube
     semver-tool
     yq-go
+    musl                  # Musl for static linking
+    gcc
+    binutils
+    gdb
+    coreutils
+    stdenv.cc.cc
   ];
+
   shellHook = ''
     export HOME=${builtins.getEnv "HOME"}
     export GOPATH=$(pwd)/nix/.go

@dsharma-dc
Copy link
Contributor

I would say if above changes to build let you test things locally, you can go ahead and make only functional changes in this PR and let the build remain as is for now.
cc: @tiagolobocastro , @abhilashshetty04 , @abhilashshetty04

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extra mkfs options
4 participants