Ceph refactor #1538
Draft
Ceph refactor #1538
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously during Ceph storage mounting Incus would try and parse the `ceph.conf` files itself. While this approach works in the majority of cases it failed to cover more then a few common setups and was fragile. This commit changes that to instead makes calls to the Ceph admin tool `ceph` which provides easy to parse JSON output and is by definition "correct". I also introduce a `CephBuildMount` function that essentially takes the output of `CephMonitors`, `CephFsid`, `CephKeyring`, and the CephFS path to produce a source path and options list suitable for passing to `mount`. This also uses the new mount syntax stlye enabling support for (sub)volumes. With this the `cephfs.getConfig` function is made redundant and `cephfs` driver is tweaked to use the modified discovery functions as well as `CephBuildMount`. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
QEMU interfaces with librados for providing Ceph RBD support and as such already has quite robust cluster discovery abilities. Instead of telling QEMU everything it could possibly want to know and risking telling it something wrong, tell it what it needs to know (user, pool, image). Then QEMU can almost certainly do a better and faster job of collecting the necessary information for initiating a cluster connection. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
MadnessASAP
force-pushed
the
ceph-refactor
branch
2 times, most recently
from
0496169 to
ee4955e
Compare
Add a `INCUS_CEPH_CLIENT` variable for testing with a restricted caps client. Moved cephfs testing to a nonstandard fs name for testing mount syntax. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
MadnessASAP
force-pushed
the
ceph-refactor
branch
from
ee4955e to
947aa95
Compare
for testing different capabilities combos it's easiest to delete and recreate the client, obviously this doesn't work if the client is admin. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In continuation of #1473 it is now approaching completion. It passes the full test suite, in addition incusd now can use ceph on my own cluster where previously it could not.
Also initial tests seem to indicate it fixes #1023:
Remaining is to write a few tests to capture some of the additional Ceph superpowers and to squash the whole mess of commits down.
With that in mind, comments and concerns are now very much welcome