Ceph refactor #1538
Merged
Ceph refactor #1538
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MadnessASAP
force-pushed
the
ceph-refactor
branch
3 times, most recently
from
ee4955e to
947aa95
Compare
|
The good news is, I can apply the entire diff of this PR onto the LTS I'm on (6.0) and it will still compile just fine. |
MadnessASAP
force-pushed
the
ceph-refactor
branch
from
34cf29e to
ebb72ef
Compare
|
Good news! Not dead! Just slow coming back from the holidays. Happy to hear it compiles on LTS @benaryorg. The Ceph logic is good to go, I'm just not yet satisfied with the testing yet, hopefully get that taken care of in the next few days and this will be ready to be (hopefully) merged. |
MadnessASAP
force-pushed
the
ceph-refactor
branch
from
507e776 to
b3aeeca
Compare
|
Upon further reflection, I've come to the conclusion that I don't have the necessary time or knowledge to be rooting around within the Incus/GitHub testing framework. The PR as it stands passes the existing tests and works within my homelab for the past couple months. As such I'm confident in saying it does not cause a regression and it does cure a previous pain point. I'm happy to address specific points of concern but otherwise I'm going to say this ready for merge. |
stgraber
reviewed
Feb 23, 2025
MadnessASAP
force-pushed
the
ceph-refactor
branch
2 times, most recently
from
8264285 to
8dfa099
Compare
stgraber
force-pushed
the
ceph-refactor
branch
2 times, most recently
from
19c2440 to
6bb91ea
Compare
stgraber
approved these changes
Feb 26, 2025
Previously during Ceph storage mounting Incus would try and parse the `ceph.conf` files itself. While this approach works in the majority of cases it failed to cover more then a few common setups and was fragile. This commit changes that to instead makes calls to the Ceph admin tool `ceph` which provides easy to parse JSON output and is by definition "correct". It also introduces a `CephBuildMount` function that essentially takes the output of `CephMonitors`, `CephFsid`, `CephKeyring`, and the CephFS path to produce a source path and options list suitable for passing to `mount`. This also uses the new mount syntax stlye enabling support for (sub)volumes. With this the `cephfs.getConfig` function is made redundant and `cephfs` driver is tweaked to use the modified discovery functions as well as `CephBuildMount`. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
QEMU interfaces with librados for providing Ceph RBD support and as such already has quite robust cluster discovery abilities. Instead of telling QEMU everything it could possibly want to know and risking telling it something wrong, tell it what it needs to know (user, pool, image). Then QEMU can almost certainly do a better and faster job of collecting the necessary information for initiating a cluster connection. Signed-off-by: Michael 'ASAP' Weinrich <[email protected]>
stgraber
force-pushed
the
ceph-refactor
branch
from
6bb91ea to
0e8e519
Compare
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Mar 2, 2025
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lxc/incus](https://github.com/lxc/incus) | minor | `v6.9.0` -> `v6.10.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lxc/incus (lxc/incus)</summary> ### [`v6.10.0`](https://github.com/lxc/incus/releases/tag/v6.10.0): Incus 6.10 [Compare Source](lxc/incus@v6.9.0...v6.10.0) #### What's Changed - incusd/instance/drivers/qmp: Handle missing log directory by [@​stgraber](https://github.com/stgraber) in lxc/incus#1604 - incus-user: keep track of socket path used to connect to the server by [@​bboozzoo](https://github.com/bboozzoo) in lxc/incus#1607 - incus-user: unify logging, support --verbose and --debug by [@​bboozzoo](https://github.com/bboozzoo) in lxc/incus#1606 - Add project support to profiles in preseed init by [@​megheaiulian](https://github.com/megheaiulian) in lxc/incus#1608 - incusd/network/ovn: Fix bad route check by [@​stgraber](https://github.com/stgraber) in lxc/incus#1616 - incus/file/pull: Ensure we have a leading / in all paths by [@​stgraber](https://github.com/stgraber) in lxc/incus#1617 - incus/file/pull: Read files in chunks by [@​stgraber](https://github.com/stgraber) in lxc/incus#1623 - doc/installing: mention incus group on NixOS by [@​dawidd6](https://github.com/dawidd6) in lxc/incus#1622 - incus/file/pull: Actually make read buffer 1MiB by [@​stgraber](https://github.com/stgraber) in lxc/incus#1624 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1639 - incusd/device/disk: Allow virtiofsd on non-x86 by [@​stgraber](https://github.com/stgraber) in lxc/incus#1638 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1640 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1642 - incusd/instance/drivers/qemu: Add IOMMU device by [@​stgraber](https://github.com/stgraber) in lxc/incus#1644 - incus/file: Remove unused function by [@​stgraber](https://github.com/stgraber) in lxc/incus#1645 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1646 - incus/network/info (ovn): Fix object not found. by [@​rxtom](https://github.com/rxtom) in lxc/incus#1628 - incusd/instance/drivers: Improve NUMA balancing by [@​lnutimura](https://github.com/lnutimura) in lxc/incus#1626 - incusd/network/bridge: Fix deletion of tunnels and dummy devices by [@​montag451](https://github.com/montag451) in lxc/incus#1627 - incus/file: Move from path to filepath by [@​stgraber](https://github.com/stgraber) in lxc/incus#1647 - Added LZ4 support for incus import by [@​Spitfireap](https://github.com/Spitfireap) in lxc/incus#1611 - Add `vrf` parameter for routed-nic devices by [@​ibot3](https://github.com/ibot3) in lxc/incus#1615 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1648 - Translations update from Hosted Weblate by [@​weblate](https://github.com/weblate) in lxc/incus#1651 - Move generators to the cmd package by [@​stgraber](https://github.com/stgraber) in lxc/incus#1652 - Fix incorrect volume group naming when `vg_name` is not specified by [@​presztak](https://github.com/presztak) in lxc/incus#1653 - Rename incus-generate and incus-doc by [@​breml](https://github.com/breml) in lxc/incus#1654 - Implement `smbios11` config keys by [@​stgraber](https://github.com/stgraber) in lxc/incus#1655 - Fix instance copy error when using '--refresh' flag by [@​presztak](https://github.com/presztak) in lxc/incus#1658 - Fix docs for load balancer create backend by [@​gwenya](https://github.com/gwenya) in lxc/incus#1661 - incusd/instance/utils: Only check uid/gid for containers by [@​stgraber](https://github.com/stgraber) in lxc/incus#1662 - incusd/main_nsexec: Fix change_namespaces fallback to handle multiple… by [@​stgraber](https://github.com/stgraber) in lxc/incus#1664 - Check if disk is remote when migrating with an extra disk by [@​presztak](https://github.com/presztak) in lxc/incus#1669 - incusd/instance/edk2: Look for bios.bin in /usr/share/seabios by [@​stgraber](https://github.com/stgraber) in lxc/incus#1672 - Replace ast.Package with types.Package by [@​breml](https://github.com/breml) in lxc/incus#1665 - list/format: provide more information on error by [@​rxtom](https://github.com/rxtom) in lxc/incus#1666 - Add additional validation when joining a new cluster member by [@​presztak](https://github.com/presztak) in lxc/incus#1680 - Upgrade flosch/pongo2 to v6 by [@​nanjj](https://github.com/nanjj) in lxc/incus#1677 - incusd/resources: Prevent concurrent runs and cache data for 10s by [@​stgraber](https://github.com/stgraber) in lxc/incus#1681 - Fix importing from older backups by [@​stgraber](https://github.com/stgraber) in lxc/incus#1683 - fix: Don't attempt to download signatures for oci by [@​m2Giles](https://github.com/m2Giles) in lxc/incus#1685 - Ensure directories have 755 permissions in `incus file push -p` command by [@​presztak](https://github.com/presztak) in lxc/incus#1687 - devcontainer: Update Go to 1.23 by [@​breml](https://github.com/breml) in lxc/incus#1689 - Make "Code generated" comments for generate-database Go conformant by [@​breml](https://github.com/breml) in lxc/incus#1690 - Disclaimer internal tool for generate-database and generate-config by [@​breml](https://github.com/breml) in lxc/incus#1694 - Truncate the block file during custom volume migration by [@​presztak](https://github.com/presztak) in lxc/incus#1696 - Rework virtiofsd uid/gid map handling by [@​stgraber](https://github.com/stgraber) in lxc/incus#1692 - Remove unused arguments and parameters by [@​presztak](https://github.com/presztak) in lxc/incus#1699 - generate-database: Use deferred func to map errors & make generated code self-sufficient by [@​breml](https://github.com/breml) in lxc/incus#1695 - incus/top: Fix handling of all-projects by [@​stgraber](https://github.com/stgraber) in lxc/incus#1701 - Ceph refactor by [@​MadnessASAP](https://github.com/MadnessASAP) in lxc/incus#1538 - incus/file: Port remaining functions to SFTP by [@​HassanAlsamahi](https://github.com/HassanAlsamahi) in lxc/incus#1649 - Add filtering to all API collections by [@​gwenya](https://github.com/gwenya) in lxc/incus#1679 - Add provider for DNS-01 ACME challenge by [@​accuser](https://github.com/accuser) in lxc/incus#1668 #### New Contributors - [@​bboozzoo](https://github.com/bboozzoo) made their first contribution in lxc/incus#1607 - [@​dawidd6](https://github.com/dawidd6) made their first contribution in lxc/incus#1622 - [@​rxtom](https://github.com/rxtom) made their first contribution in lxc/incus#1628 - [@​lnutimura](https://github.com/lnutimura) made their first contribution in lxc/incus#1626 - [@​ibot3](https://github.com/ibot3) made their first contribution in lxc/incus#1615 - [@​gwenya](https://github.com/gwenya) made their first contribution in lxc/incus#1661 - [@​accuser](https://github.com/accuser) made their first contribution in lxc/incus#1668 **Full Changelog**: lxc/incus@v6.9.0...v6.10.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xODIuMyIsInVwZGF0ZWRJblZlciI6IjM5LjE4Mi4zIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In continuation of #1473 it is now approaching completion. It passes the full test suite, in addition incusd now can use ceph on my own cluster where previously it could not.
Also initial tests seem to indicate it fixes #1023:
Remaining is to write a few tests to capture some of the additional Ceph superpowers and to squash the whole mess of commits down.
With that in mind, comments and concerns are now very much welcome