Mounted Blobfuse2 volume / container owned by root #809
Comments
|
there is |
|
with |
|
Hi @andyzhangx. This is the PV yaml used with Blobfuse v1 using UID/GID: apiVersion: v1 |
|
@mortenjoenby I think your blobfuse2 mount does not work, could you ssh to your application, and run following command to check blobfuse2 mount? |
|
Hi @andyzhangx. And it actually shows UID/GID, so why can't I set it? |
|
@vibhansa-msft @souravgupta-msft do you know whether |
|
From what I can tell there's at least a difference. Seems that UID/GID are actually 0 in both cases though. Only difference here is the "defaul_permissions" set on v1 mount. |
|
This is the container log from the csi-blob-node: |
@andyzhangx we do accept uid and gid as input. But we have observed some issues in libfuse layer with that. So, it does not work as expected. |
|
@souravgupta-msft / @andyzhangx , currently I don't think uid/gid is even accepted. It failed for us at least. |
|
@mortenjoenby : Are you saying these options were accepted in preview.4 and not in the GA build? That's not the case they are accepted in GA version as well. However, we have observed that passing them down to libfuse does not work as intended and hence we started ignoring those options at our end. User expectation here was that mount will be accessible to only the user whose uid/gid is given but mounting through /etc/fstab was going via root user and setting these values was resulting into some access issues. |
|
@mortenjoenby : I looked at the code and it does seem to have issue with parsing uid and gid. We can fix that parsing logic from our end but that will still not serve the purpose. I tried setting it up on the mount and here is my observation:
|
|
Hi @vibhansa-msft (not sure I can tag you - don't know why). When UID/GID is not supported, we simply can't see the files, so in a troubleshooting scenario on the application side, we won't be able to see that the files are there. If the files were at least visible that might be ok, but they are not. I can only view them using Azure Portal and viewing the Blob container. |
|
When you say it was working with blobfuse-1.4.5, is that working with "allow_other"?
|
|
if things are working with allow_other then there is no point of giving uid.gid at all, because allow_other just means mount is accessible to all users. |
|
|
|
if allow_other is there then it does not make sense to pass down uid and gid. I agree to the point that we need to fix our parsing logic but want to understand what role uid/gid are playing in this workflow. If allow_other is there in mount option then any user is free to access to mount point. Its just that file ownership will be shown with the set uid nothing else. |
|
Hi @andyzhangx / @vibhansa-msft. In AKS, if using allow_other=true (default), won't it then be possible to exploit that and e.g. see the files from another pod? |
@mortenjoenby with blob-csi-driver/deploy/example/storageclass-blobfuse.yaml Lines 12 to 14 in c989521 it's 0777 permission for files and folders with |
|
Hi @andyzhangx . Only difference from Blobfuse v1 is that the ownership is root and not the application owner. And one other question. Where do I find all the default values used in the CSI Driver for Blobfuse v2? |
|
@mortenjoenby just remove blob-csi-driver/deploy/example/storageclass-blobfuse.yaml Lines 12 to 14 in c989521 You could find all mount options in blob csi driver logs in around |
|
@mortenjoenby: user and owner of files are derived with uid/gid value submitted in mount command. In v1 we were accepting those values and passing it down to libfuse to show the ownership correctly. But we in v2 we decided to ignore that because it was not serving the purpose correctly. As I mentioned in my above example even if user1 mounts blobfuse with uid/gid set for 'user2' and no 'allow_other' user1 can still access the files while user2 cannot. Kindly validate this behavior with both v1 and v2 with AKS, that can help us understand what the correct defaults shall be and whether we shall still support uid/gid or not. |
|
As for the original issue of mount failing when uid/gid are supplied I have corrected the code in this |
@vibhansa-msft , we will try this, but I am not sure we can squeeze it in today. |
|
Sure, I tried it in my seutp and even in v1 I see this issue. As per observation giving uid/gid does not make any difference (other then showing a different owner to a file). allow_other is what controls who can access the files. If you confirm the same, we need to think on whether to set this uid/gid at all in case of AKS. |
@vibhansa-msft so |
|
For backward compatibility Blobfuse2 accepts uid and gid parameters as input. However, in past we have observed these parameters do not work as expected. Even when uid is set and mount is done from another user, user with the uid provided is not able to access the files and folder unless allow_other is provided in mount options. As flags were creating a confusion for the customers, we have stopped passing them down to libfuse layer. They will be accepted as valid inputs but later discarded at blobfuse layer. |
|
thanks @vibhansa-msft will close this issue. |
|
seems there is issue here, currently on AKS, it's |
|
can we reopen this issue? |
|
Of course you can. Do I need to do something? |
|
Hi we are currently facing the same issue, is there any temporary workaround ? |
@1CuriousPenguin you could ssh to the agent node, and then run |
|
Thanks @andyzhangx. Since we are using Helm, I'm trying to update to v1.21.0 and then apply patch to install blobfuse v2. Will that fix the issue or do we need to wait for a new release ? |
@1CuriousPenguin pls wait for v1.21.1 release, it would be ready this week. |
We are testing blobfuse2 as we want to compare performance to blobfuse (v1.4.5).
We managed to get our pod running with a mounted volume (only after updating Ubuntu image from august to december build - due to "exec: "blobfuse2": executable file not found in $PATH" error").
What happened:
We successfully mount a blobfuse2 volume on an existing blobcontainer which we have previously used for testing with blobfuse v1.4.5 - CSI driver v1.17.0).
Problem: Container mounts as root and not application owner
What you expected to happen:
We want the container to mount as the application user that runs our application.
When using azure-storage-fuse standalone on a VM I can mount the blobfuse2 volume as the OS user who should own the content.
I would like to know how I can do this within AKS?
How to reproduce it:
$ ls -l
drwxrwxrwx 2 root root 4096 Jan 3 15:11 background-processarea
Environment:
kubectl version): 1.24.3uname -a): 5.4.0-1098-azure doc: fix doc typo #104~18.04.2-UbuntuThe text was updated successfully, but these errors were encountered: