diff --git a/doc/auto_storage.md b/doc/auto_storage.md deleted file mode 100644 index 4d0831e54e..0000000000 --- a/doc/auto_storage.md +++ /dev/null @@ -1,1161 +0,0 @@ -# Storage Section of the Agama Profile - -This document describes Agama's approach to configure storage using a profile for unattended -installation. - -## Agama and AutoYaST - -The Agama profile has a special `legacyAutoyastStorage` section which is a 1:1 representation of the -XML AutoYaST profile. This section supports everything offered by the *partitioning* AutoYaST -section. Note that Agama does not validate this special section, so be careful to provide valid -AutoYaST options. - -~~~json -{ - "legacyAutoyastStorage": [ - { - "use": "all", - "partitions": [] - } - ] -} -~~~ - -Although that special section is offered for backwards compatibility and to ease gradual migration -from AutoYaST to Agama, there are no plans to introduce any improvement or new feature in the legacy -mode due to the [intrinsic limitations](./autoyast_storage.md) of the original AutoYaST schema. - -### Implementation Considerations for AutoYaST Specification - -In principle, implementing the legacy AutoYaST module is as simple as converting the corresponding -section of the profile into a `Y2Storage::PartitioningSection` object and use -`Y2Storage::AutoInstProposal` to calculate the result. - -But there are some special cases in which AutoYaST fallbacks to read some settings from the YaST -settings or to use some YaST mechanisms. Those cases should be taken into account during the -implementation. - -For example, AutoYaST relies on the traditional YaST proposal settings when "auto" is used to -specify the size of a partition or to determine the default list of subvolumes when Btrfs is used. -See also the sections "Automatic Partitioning" and "Guided Partitioning" at the AutoYaST -documentation for situations in which AutoYaST uses the standard YaST `GuidedProposal` as fallback. - -## The New Storage Schema - -Agama offers its own storage schema which is more semantic, comprehensive and flexible than the -AutoYaST one. - -The new schema allows: - -* To clearly distinguish between different types of devices and their properties. -* To perform more advanced searches for disks, partitions, etc. -* To indicate deleting and resizing on demand. - -The Agama schema is used by a new Agama specific proposal. This decouples the algorithm from the -AutoYaST one, making much easier to support new use cases and avoiding backward compatibility with -fringe AutoYaST scenarios. It also supports some features that are not available in the AutoYaST -proposal like deleting or resizing partitions on demand. - -### Basic Structure of the Storage Section - -A formal specification of the outer level of the `storage` section would look like this. - -``` -Storage - drives - volumeGroups - mdRaids - btrfsRaids - nfsMounts - boot [BootSettings] - encryption [EncryptionSettings] -``` - -Thus, a `storage` section can contain several entries describing how to configure the corresponding -storage devices and a couple of extra entries to setup some general aspects that influence the final -layout. - -Each volume group, RAID, bcache device or NFS share can represent a new logical device to be created -or an existing device from the system to be processed. Entries below `drives` represent devices -that can be used as regular disks. That includes removable and fixed disks, SD cards, DASD or zFCP -devices, iSCSI disks, multipath devices, etc. Those entries always correspond to devices that can be -found at the system, since Agama cannot create that kind of devices. - -In fact, a single entry can represent several devices from the system. That is explained in depth at -the section "searching existing devices" of this document. - -On the first versions of Agama, an alternative syntax will be accepted including only one `guided` -entry. - -``` -Storage - guided -``` - -That allows to rely on the YaST component known as `GuidedProposal`. That alternative will be -removed as soon as all the capabilities of that `GuidedProposal` could be expressed in terms of a -regular storage configuration like the one explained above. - -### Entries for Describing the Devices - -The formal specification of the previous section can be extended as we dive into the structure. - -``` -Drive - search [] - alias [] - encryption [] - filesystem [] - ptableType [] - partitions [] - -VolumeGroup - search [] - alias [] - name [] - peSize [] - physicalVolumes [[]>] - logicalVolumes [] - delete [] - -MdRaid - search [] - alias [] - name - level [] - chunkSize [] - devices [<[]>] - size [] - encryption [] - filesystem [Filesystem] - ptableType [] - partitions [] - delete [] - -BtrfsRaid - search [] - alias [] - dataRaidLevel - metadataRaidLevel - devices [<[]>] - label [] - mkfsOptions [] - [Btrfs] - delete [] - -NFS - alias [] - path [] - mount [] - -Partition - search [] - alias [] - id [] - size [] - encryption [Encryption] - filesystem [] - delete [] - deleteIfNeeded [] - -LogicalVolume - search [] - alias [] - name [] - size [] - pool [] - usedPool [] - stripes [] - stripSize [] - encryption [Encryption] - filesystem [] - delete [] - deleteIfNeeded [] -Encryption - reuse - type - -EncryptionType - -EncryptionLUKS1 - password - keySize [] - cipher [] - -EncryptionLUKS2 - password - keySize [] - cipher [] - pdkdf [] - label [] - -EncryptionPervasiveLUKS2 - password - -Filesystem - reuse - type - label [] - mkfsOptions [] - path - mountOptions [] - mountBy [] - -Btrfs - subvolumePrefix [] - subvolumes [] - snapshots [] - quotas [] - -Size - -SizeRange - min - max - -BootSettings - configure - device - -EncryptionSettings - method - key [] - pdkdf [] - cipher [] - keySize [] -``` - -To illustrate how all that fits together, let's see the following example in which the first disk of -the system is partitioned and a volume group is created on top of that partition (after encrypting -it) to allocate two file systems. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "alias": "pv", - "id": "lvm", - "size": { "min": "12 GiB" }, - "encryption": { - "luks2": { "password": "my secret passphrase" } - } - } - ] - } - ], - "volumeGroups": [ - { - "name": "system", - "physicalVolumes": [ "pv" ], - "logicalVolumes": [ - { - "size": { "min": "10 GiB" }, - "filesystem": { "path": "/", "type": "btrfs" } - }, - { - "size": "2 GiB", - "filesystem": { "path": "swap", "type": "swap" } - } - ] - } - ] -} -``` - -### Specifying the Size of a Device - -When creating some kinds of devices or resizing existing ones (if possible) it may be necessary to -specify the desired size. As seen in the specification above, that can be done in several ways. - -The most straightforward one is just using a string that can be parsed into a valid size. - -The second option is to provide a minimum size and an optional maximum one. The resulting size will -be between those thresholds. If the maximum is omitted or set to `null`, the device will grow as -much as possible, taking into account the available spaces and all the other specified sizes. - -It is also possible to specify "current" as a minimum or maximum size limit for partitions and -logical volumes that already exist in the system (so "current" can only be used for device -specifications that contain a `search` section). The usage of "current" and how it affects -resizing the corresponding devices is explained at a separate section below. - -If the size is completely omitted for a device that already exists (ie. combined with `search`), -then Agama would act as if both min and max limits would have been set to "current" (which implies -the partition or logical volume will not be resized). - -Moreover, if the size is omitted for a device that will be created, Agama will determine the size -limits when possible. There are basically two kinds of situations in which that automatic size -calculation can be performed. - -On the one hand, the device may directly contain a `filesystem` entry specifying a mount point. -Agama will then use the settings of the product to set the size limits. From a more technical point -of view, that translates into the following: - - - If the mount path corresponds to a volume supporting `auto_size`, that feature will be used. - - If it corresponds to a volume without `auto_size`, the min and max sizes of the volumes will be - used. - - If there is no volume for that mount path, the sizes of the default volume will be used. - - If the product does not specify a default volume, the behavior is still not defined (there are - several reasonable options). - -On the other hand, the size limits of some devices can be omitted if they can be inferred from other -related devices following some rules. - - - For an MD RAID defined on top of new partitions, it is possible to specify the size of all the - partitions that will become members of the RAID but is also possible to specify the desired size - for the resulting MD RAID and then the size limits of each partition will be automatically - inferred with a small margin of error of a few MiBs. - - Something similar happens with a partition that acts as the **only** physical volume of a new LVM - volume group. Specifying the sizes of the logical volumes could be enough, the size limits of the - underlying partition will match the necessary values to make the logical volumes fit. In this - case the calculated partition size is fully accurate. - - The two previous scenarios can be combined. For a new MD RAID that acts as the **only** physical - volume of a new LVM volume group, the sizes of the logical volumes can be used to precisely - determine what should be the size of the MD and, based on that, what should be the almost - exact size of the underlying new partitions defined to act as members of the RAID. - -The two described mechanisms to automatically determine size limits can be combined. Even creating -a configuration with no explicit sizes at all like the following example. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { "alias": "pv" } - ] - } - ], - "volumeGroups": [ - { - "name": "system", - "physicalVolumes": [ "pv" ], - "logicalVolumes": [ - { "filesystem": { "path": "/" } }, - { "filesystem": { "path": "swap" } } - ] - } - ] -} -``` - -Assuming the product configuration specifies a root filesystem with a minimum size of 5 GiB and a -max of 20 GiB and sets that the swap must have a size equivalent to the RAM on the system, then -those values would be applied to the logical volumes and the partition with alias "pv" would be -sized accordingly, taking into account all the overheads and roundings introduced by the different -technologies like LVM or the used partition table type. - -#### Under Discussion - -As explained, it should be possible to specify the sizes as a fixed value or as a range. But a -a parseable string like "40 GiB" may not be the only option to represent a size or a range limit. -The following two possibilities are also under consideration. - - - `{ "gib": 40 }` - - `{ "value": 40, "units": "gib" }` - -### Partitions Needed for Booting - -Using a `boot` entry makes it possible to configure whether (and where, using an alias) Agama -should calculate and create the extra partitions needed for booting. If the device is not -specified, Agama will take the location of the root file system as a reference. - -### Searching Existing Devices - -Many sections in the profile are used to describe how some devices must be created, modified or even -deleted. In the last two cases, it's important to match the description with one or more devices -from the system. - -If a description matches several devices, the same operations will be applied to -all. That's useful in several situations like applying the same partitioning schema to several disks -or deleting all partitions of a disk that match a given criteria. - -Matching is performed using a `search` subsection. The format is still under heavy discussion but -may look similar to this. - -``` -Search - condition [] - sort [] - max [] - ifNotFound [] - -Condition - -OperatorAnd - and: - -OperatorOr - or: - -Rule - property - value - operator [] - -Operator <'equal'|'notEqual'|'less'|'greater'|'lessOrEqual'|'greaterOrEqual'> - -Sort - property - order <'asc'|'desc'> - -NotFoundAction <'create'|'skip'|'error'> -``` - -By default, all devices in the scope fitting the conditions will be matched. The number of device -matches can be limited using `max`. The following example shows how several `search` sections could -be used to find the three biggest disks in the system, delete all linux partitions bigger than 1 GiB -within them and create new partitions of type RAID. - -```json -"storage": { - "drives": [ - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 3 - }, - "partitions": [ - { - "search": { - "condition": { - "and": [ - { "property": "id", "value": "linux" }, - { "property": "sizeGib", "value": 1, "operator": "greater" } - ] - } - }, - "delete": true - }, - { - "alias": "newRaidPart", - "id": "raid", - "size": { "min": "1 GiB" } - } - ] - } - ] -} -``` - -The example also serves to illustrate the scope of each search. That is, the devices from the system -that are considered as possible candidates. That obviously depends on the place in the profile of -the `search` section. A `search` section inside the description of an MD RAID will only match MD -devices and a `search` section inside the `partitions` subsection of that RAID description will only -match partitions of RAIDs that have matched the conditions of the most external `search`. - -A given device can never match two different sections of the Agama profile. When several sections -at the same level contain a search subsection, devices are matched in the order the sections appear -on the profile. - -```json -"storage": { - "drives": [ - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 1 - }, - "alias": "biggest" - }, - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 1 - }, - "alias": "secondBiggest" - } - ] -} -``` - -An empty search will match all devices in the scope, so the following example would delete all the -partitions of the chosen disk. - -```json -"storage": { - "drives": [ - { - "partitions": - { "search": {}, "delete": true } - } - ] -} -``` - -If there is not a single system device matching the scope and the conditions of a given search, then -`ifNotFound` comes into play. If the value is "skip", the device definition is ignored. If it's -"error" the whole process is aborted. The value "create", which cannot be used for a drive, will -cause the `search` section to be ignored if no device matches. As a consequence, a new logical -device (partition, LVM, etc.) will be created. - -Entries on `drives` are different to all other subsections describing devices because drives can -only be matched to existing devices, they cannot be simply created. If `search` is omitted for a -drive, it will be considered to contain the following one. - -```json -{ - "search": { - "sort": { "property": "name" }, - "max": 1, - "ifNotFound": "error" - } -} -``` - -#### Under Discussion - -Very often, `search` will be used to find a device by its name. In that case, the syntax could be -simplified to just contain the device name as string. - -```json -{ "search": "/dev/sda" } -``` - -Using a string as value for `search` may also be useful in other situations. Special values could be -used as aliases for typical cases: - - - Empty string or "\*" to match all devices (the same than an empty section) - - Something like "next" to represent the default search for drives (see above) - -If a simple string like "next" could be used to specify the standard search entry for drives, it -would make sense to simply make `search` mandatory for all drives instead of assuming a default one. - -Another possible improvement for that string-based format would be supporting regular expressions. -That would make it possible to use searchers like this. - -```json -{ "search": ".*" } -``` - -But regular expressions would not play well with libstorage-ng. Since not all device names are -stored in the devicegraph, it is is necessary to use functions like `find_by_any_name` in order to -perform an exhaustive search by name. - -Another apect under discussion is the format to specify conditions. Instead of the format described -above, it would be possible to use the key as name of the property, resulting in something like this. - -```json -{ - "search": { - "condition": { "sizeGib": 1, "operator": "greater" } - } -} -``` - -### Referencing Other Devices - -Sometimes is necessary to reference other devices as part of the specification of an LVM volume -group or RAID. Those can be existing system devices or devices that will be created as response to -another entry of the Agama profile. - -Aliases can be used for that purpose as shown in this example. - -```json -"storage": { - "drives": [ - { - "partitions": - { "size": "50 GiB", "id": "lvm", "alias": "newPV" } - } - ], - "volumeGroups": [ - { - "name": "newVG", - "physicalVolumes": [ "newPV" ], - "logicalVolumes": [ { "name": "data", "size": "20 GiB" } ] - } - ] -} -``` - -If a section that matches several existing devices contains an alias, that alias will be considered -to be a reference to all the devices. As a consequence, this two examples are equivalent. - -```json -"storage": { - "drives": [ - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 1, - }, - "alias": "biggest" - }, - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 1, - }, - "alias": "secondBiggest" - } - ], - "mdRaids": [ - { - "devices": [ "biggest", "secondBiggest" ], - "level": "raid0" - } - ] -} - -"storage": { - "drives": [ - { - "search": { - "sort": { "property": "sizeKib", "order": "desc" }, - "max": 2, - "min": 2 - }, - "alias": "big" - } - ], - "mdRaids": [ - { - "devices": [ "big" ], - "level": "raid0" - } - ] -} -``` - -#### Under Discussion - -In addition to aliases, a `search` section could be accepted in all the places in which an alias can -be used. In that case, the scope of the search would always be the whole set of devices in the -system (so the same conditions can be matched by a disk, a partition, an LVM device, etc.) and -`ifNotFound` could not be set to "create" (similar to what happens for drives in general). - -```json -"storage": { - "volume_groups": [ - { - "name": "newVG", - "physicalVolumes": [ - { "search": { "condition": { "property": "name", "value": "/dev/sda2" } } } - ], - "logicalVolumes": [ { "name": "data", "size": "20 GiB" } ] - } - ] -} -``` - -### Keeping an Existing File System or Encryption Layer - -The entries for both `encryption` and `filesystem` contain a flag `reuse` with a default value of -false. It can be used in combination with `search` to specify the device must not be re-encrypted -or re-formatted. - -### Deleting and Shrinking Existing Devices - -The storage proposal must make possible to define what to do with existing partitions and logical -volumes. Even with existing MD RAIDs or LVM volume groups. - -A `search` section allows to match the definition of a partition or an LVM logical volume with one -(or several) devices existing in the system. In order to provide the same capabilities than the -Guided proposal (see below) it must be possible to specify that a given partition or volume must be: - - - Deleted if needed to make space for the newly defined devices - - Deleted in all cases - - Shrunk to the necessary size to make space for new devices - - Shrunk or extended to a given size, maybe a range (not really possible in the current Guided - Proposal) - -It is even possible to express some combinations of the above, like "try to shrink it to make space -but proceed to delete it if shrinking it is not enough". - -Deletion can be achieved with the corresponding `delete` flag or the alternative `deleteIfNeeded`. -If any of those flags are active for a partition, it makes no sense to specify any other usage -(like declaring a file system on it). - -The following example deletes the partition with the label "root" in all cases and, if needed, keeps -deleting other partitions as needed to make space for the new partition of 30 GiB. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "search": { - "condition": { "property": "fsLabel", "value": "root" } - }, - "delete": true - }, - { "search": {}, "deleteIfNeeded": true }, - { "size": "30 GiB" } - ] - } - ] -} -``` - -Often some partitions or logical volumes are shrunk only to make space for the declared devices. But -since resizing is not a destructive operation, it can also make sense to declare a given partition -must be resized (shrunk or extended) and then formatted and/or mounted. - -In any case, note that resizing a partition can be limited depending on its content, the filesystem -type, etc. - -Combining `search` and `resize` is enough to indicate Agama is expected to resize a given partition -if possible. The keyword "current" can be used as min and/or max for the size range and it is always -equivalent to the exact original size of the device. The simplest way to use "current" is to just -specify that the matched device should keep its original size. That's the default for searched (and -found) devices if `size` is completely omitted. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "search": { - "condition": { "property": "fsLabel", "value": "reuse" } - }, - "size": { "min": "current", "max": "current" } - } - ] - } - ] -} -``` - -Other combinations can be used to specify how a device could be resized if possible. See the -following examples with explanatory filesystem labels. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "search": { - "condition": { "property": "fsLabel", "value": "shrinkIfNeeded" } - }, - "size": { "min": 0, "max": "current" } - }, - { - "search": { - "condition": { "property": "fsLabel", "value": "resizeToFixedSize" } - }, - "size": "15 GiB" - }, - { - "search": { - "condition": { "property": "fsLabel", "value": "resizeByRange" } - }, - "size": { "min": "10 GiB", "max": "50 GiB" } - }, - { - "search": { - "condition": { "property": "fsLabel", "value": "growAsMuchAsPossible" } - }, - "size": { "min": "current" } - }, - ] - } - ] -} -``` - -Of course, when the size limits are specified as a combination of "current" and a fixed value, the -user must still make sure that the resulting min is not bigger than the resulting max. - -Both `deleteIfNeeded` and a size range can be combined to indicate that Agama should try to make -space first by shrinking the partitions and deleting them only if shrinking is not enough. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "search": {}, - "size": { "min": 0, "max": "current" }, - "deleteIfNeeded": true - } - ] - } - ] -} -``` - -### Generating Partitions as MD RAID members - -MD arrays can be configured to explicitly use a set of devices by adding their aliases to the -`devices` property. - -```json -"storage": { - "drives": [ - { - "search": "/dev/sda", - "partitions": [ - { "alias": "sda-40", "size": "40 GiB" } - ] - }, - { - "search": "/dev/sdb", - "partitions": [ - { "alias": "sdb-40", "size": "40 GiB" } - ] - } - ], - "mdRaids": [ - { - "devices": [ "sda-40", "sdb-40" ], - "level": "raid0" - } - ] -} -``` - -The partitions acting as members can be automatically generated by simply indicating the target -disks that will hold the partitions. For that, the `devices` section must contain a `generate` -entry. - -```json -"storage": { - "drives": [ - { "search": "/dev/sda", "alias": "sda" }, - { "search": "/dev/sdb", "alias": "sdb" }, - ], - "mdRaids": [ - { - "devices": [ - { - "generate": { - "targetDisks": ["sda", "sdb" ], - "size": "40 GiB" - } - } - ] - "level": "raid0" - } - ] -} -``` - -As explained at the section about sizes, it's also possible to set the size for the new RAID letting -Agama calculate the corresponding sizes of the partitions used as members. That allows to use the -short syntax for `generate`. - -```json -"storage": { - "drives": [ - { "search": "/dev/sda", "alias": "sda" }, - { "search": "/dev/sdb", "alias": "sdb" }, - ], - "mdRaids": [ - { - "devices": [ { "generate": ["sda", "sdb" ] } ], - "level": "raid0", - "size": "40 GiB" - } - ] -} -``` - -### Generating Default Volumes - -Every product provides a configuration which defines the storage volumes (e.g., feasible file -systems for root, default partitions to create, etc). The default or mandatory product volumes can -be automatically generated by using a *generate* section in the *partitions* or *logicalVolumes* -sections. - -```json -"storage": { - "drives": [ - { - "partitions": [ - { "generate": "default" } - ] - } - ] -} - -``` - -The *generate* section allows creating the product volumes without explicitly writing all of them. -The config above would be equivalent to something like this: - -```json -"storage": { - "drives": [ - { - "partitions": [ - { "filesystem": { "path": "/" } }, - { "filesystem": { "path": "/home" } }, - { "filesystem": { "path": "swap" } } - ] - } - ] -} - -``` - -If any path is explicitly defined, then the *generate* section will not generate a volume for it. -For example, with the following config only root and swap would be automatically added: - -```json -"storage": { - "drives": [ - { - "partitions": [ - { "generate": "default" }, - { "filesystem": { "path": "/home" } } - ] - } - ] -} -``` - -The auto-generated volumes can be also configured. For example, for encrypting the partitions: - -```json -"storage": { - "drives": [ - { - "partitions": [ - { - "generate": { - "partitions": "default", - "encryption": { - "luks1": { "password": "12345" } - } - } - } - ] - } - ] -} -``` - -The *mandatory* keyword can be used for only generating the mandatory partitions or logical volumes: - -```json -"storage": { - "volumeGroups": [ - { - "name": "system", - "logicalVolumes": [ - { "generate": "mandatory" } - ] - } - ] -} -``` - -The *default* and *mandatory* keywords can also be used to generate a set of formatted MD arrays. -Assuming the default volumes are "/", "/home" and "swap", the following snippet would generate three -RAIDs of the appropriate sizes and the corresponding six partitions needed to support them. - -```json -"storage": { - "drives": [ - { "search": "/dev/sda", "alias": "sda" }, - { "search": "/dev/sdb", "alias": "sdb" }, - ], - "mdRaids": [ - { - "generate": { - "mdRaids": "default", - "level": "raid0", - "devices": [ - { "generate": ["sda", "sdb"] } - ] - } - } - ] -} -``` - -### Generating Physical Volumes - -Volume groups can be configured to explicitly use a set of devices as physical volumes. The aliases -of the devices to use are added to the list of physical volumes: - -```json -"storage": { - "drives": [ - { - "search": "/dev/vda", - "partitions": [ - { "alias": "pv2", "size": "100 GiB" }, - { "alias": "pv1", "size": "20 GiB" } - ] - } - ], - "volumeGroups": [ - { - "name": "system", - "physicalVolumes": ["pv1", "pv2"] - } - ] -} -``` - -The physical volumes can be automatically generated too, by simply indicating the target devices in -which to create the partitions. For that, a *generate* section is added to the list of physical -volumes: - -```json -"storage": { - "drives": [ - { - "search": "/dev/vda", - "alias": "pvs-disk" - } - ], - "volumeGroups": [ - { - "name": "system", - "physicalVolumes": [ - { "generate": ["pvs-disk"] } - ] - } - ] -} -``` - -If the auto-generated physical volumes have to be encrypted, then the encryption config is added to -the *generate* section: - - -```json -"storage": { - "drives": [ - { - "search": "/dev/vda", - "alias": "pvs-disk" - } - ], - "volumeGroups": [ - { - "name": "system", - "physicalVolumes": [ - { - "generate": { - "targetDevices": ["pvs-disk"], - "encryption": { - "luks2": { "password": "12345" } - } - } - } - ] - } - ] -} -``` - -### Using the Automatic Proposal - -On the first implementations, Agama can rely on the process known as Guided Proposal to calculate -all the needed partitions, LVM devices and file systems based on some general product settings and -some user preferences. That mechanism is offered as a temporary alternative to the more descriptive -syntax explained at previous sections of this document and it's implemented via a `guided` section -that conforms to the following specification. - -``` -Guided - device [TargetDevice] - boot [BootSettings] - encryption [EncryptionSettings] - space <'delete'|'resize'|'keep'> - volumes [Volume[]] - -TargetDevice - -TargetDisk - disk - -TargetNewLvm - newLvmVg - -TargetReusedLvm - reusedLvmVg - -Volume - mountPath - mountOptions - filesystem - autoSize - minSize - maxSize - snapshots - target - -VolumeTarget <'default'|NewPartition|NewVg|UseDevice|UseFilesystem> - -NewPartition - newPartition - -NewVg - newVg - -UseDevice - device - -UseFilesystem - filesystem -``` - -The `device` can be specified in several ways. The simplest one is using one of the strings "disk" -or "newLvmVg". In that case, the proposal will automatically select the first disk to be used as -target disk or as base to create the physical volumes. For example, this will create a default -partition-based installation on the first available disk. - -```json -"storage": { - "guided": { "device": "disk" } -} -``` - -And this will do the same, but creating a new LVM volume group on that first candidate disk. - -```json -"storage": { - "guided": { "device": "newLvmVg" } -} -``` - -It's also possible to use a device name to specify a concrete disk... - -```json -"storage": { - "guided": { - "device": { - "disk": "/dev/sda" - } - } -} -``` - -or to specify the set of disks where the LVM physical volumes can be created. - -```json -"storage": { - "guided": { - "device": { - "newLvmVg": ["/dev/vda", "/dev/vdb"] - } - } -} -``` - -Apart from specifying the main target device, device names must be used wherever a device is -expected, eg. when indicating a special target for a given volume. - -In principle, the list of volumes will have the same format than the existing HTTP API used by -the UI for calculating the storage proposal. That is, if the list is not provided the default -volumes will be created and if some aspects are omitted for a given volume they will be completed -with default values. In the future we may consider more advanced mechanisms to include or exclude -some given volumes or to customize a single volume without having to provide the full list of -volume mount paths. - -The `guided` section makes it possible to achieve the same results than using the Agama user -interface with only one exception. The Agama UI allows to indicate that a given set of partitions -can be resized if needed to allocate the volumes, without actually indicating how much those -partitions should be resized. The Guided Proposal algorithm decides whether to resize and how much -based on the other settings. Currently there is no way to express that in the auto-installation -profile. diff --git a/doc/autoyast_storage.md b/doc/autoyast_storage.md index 190a4559f6..740c21e605 100644 --- a/doc/autoyast_storage.md +++ b/doc/autoyast_storage.md @@ -1,13 +1,30 @@ -# Problems with the AutoYaST Storage Schema +# Agama and AutoYaST -The AutoYaST schema is far from ideal and it presents some structural problems. For that reason, -Agama offers its own storage schema following similar principles but a different approach at several -levels. +The AutoYaST schema to specify the storage setup is far from ideal and presents some structural +problems. Although Agama uses its own storage schema, an Agama profile can contain a special +`legacyAutoyastStorage` section which is a 1:1 representation of the XML AutoYaST profile. -This document explains some of the problems that caused the AutoYaST schema (or an hypothetical +## Implementation considerations for the AutoYaST specification + +In principle, implementing the legacy AutoYaST module is as simple as converting the corresponding +section of the profile into a `Y2Storage::PartitioningSection` object and use +`Y2Storage::AutoInstProposal` to calculate the result. + +But there are some special cases in which AutoYaST fallbacks to read some settings from the YaST +settings or to use some YaST mechanisms. Those cases should be taken into account during the +implementation. + +For example, AutoYaST relies on the traditional YaST proposal settings when "auto" is used to +specify the size of a partition or to determine the default list of subvolumes when Btrfs is used. +See also the sections "Automatic Partitioning" and "Guided Partitioning" at the AutoYaST +documentation for situations in which AutoYaST uses the standard YaST `GuidedProposal` as fallback. + +## Problems with the AutoYaST storage schema + +This section explains some of the problems that caused the AutoYaST schema (or an hypothetical compatible one) to be discarded as the main schema for Agama. -## Everything Is a Drive or a Partition Section +### Everything is a Drive or a Partition section This could seem a minor detail, but it has several implications: @@ -29,7 +46,7 @@ This could seem a minor detail, but it has several implications: ~~~ -## Directly Formatting Devices is Hammered +### Directly formatting devices is hammered A `` section is still needed for directly formatting a device, which shows the abuse of the schema. @@ -47,7 +64,7 @@ the schema. ~~~ -## Selecting Devices is Difficult and Limited +### Selecting devices is difficult and limited The AutoYaST schema allows selecting specific devices by using the `` property. This forces to use inverse logic when looking for a device. For example, if you want to select a disk @@ -101,7 +118,7 @@ allowing selecting a partition only by its number. Note that you could indicate the same partition number for deleting (``) and for reusing (``). -## Devices Are Created in a Indirect Way +### Devices are created in a indirect way For creating new LVM volume groups, RAIDS, etc, it is necessary to indicate which devices to use as logical volumes or as RAID members. In AutoYaST, the partitions have to indicate the device they are @@ -133,13 +150,13 @@ going to be used by. It would be more natural to indicate the used devices directly in the RAID or logical volume drive. -## Actions to make space must be very explicit +### Actions to make space must be very explicit There is no way to specify optional actions to be performed on the existing devices, like "resize a given partition as much as needed to make space for the new ones" or "delete a partition only if necessary" or "grow the existing partition to use the rest of the available space". -## MD RAIDs and LVM Volume Groups must be described exhaustively +### MD RAIDs and LVM Volume Groups must be described exhaustively To get a volume group on top of partitions distributed across several disks, the profile must specify the partitions that will serve as physical volumes on each disk, including exact sizes. @@ -153,3 +170,20 @@ partition level even if the usable size of the resulting MD RAID may not obvious Of course, the problem accumulates when defining an LVM volume group on top of an MD RAID that sits on top of some partitions. All the sizes may match (including all possible overheads and rounding) or the result will contain either wasted or surplus space. + +## The New Agama storage schema + +Agama offers its own storage schema (using a `storage` section instead of the mentioned +`legacyAutoyastStorage`) which is more semantic, comprehensive and flexible than the +AutoYaST one. + +The new schema allows: + +* To clearly distinguish between different types of devices and their properties. +* To perform more advanced searches for disks, partitions, etc. +* To indicate deleting and resizing on demand. + +The Agama schema is used by a new Agama specific proposal. This decouples the algorithm from the +AutoYaST one, making much easier to support new use cases and avoiding backward compatibility with +fringe AutoYaST scenarios. It also supports some features that are not available in the AutoYaST +proposal like deleting or resizing partitions on demand. diff --git a/doc/images/storage_ui/yast_guided_result.png b/doc/images/storage/yast_guided_result.png similarity index 100% rename from doc/images/storage_ui/yast_guided_result.png rename to doc/images/storage/yast_guided_result.png diff --git a/doc/images/storage_ui/agama_guided.png b/doc/images/storage_ui/agama_guided.png deleted file mode 100644 index f37d111c2d..0000000000 Binary files a/doc/images/storage_ui/agama_guided.png and /dev/null differ diff --git a/doc/images/storage_ui/automatic_size_example1.png b/doc/images/storage_ui/automatic_size_example1.png deleted file mode 100644 index da851e6d6d..0000000000 Binary files a/doc/images/storage_ui/automatic_size_example1.png and /dev/null differ diff --git a/doc/images/storage_ui/automatic_size_example2.png b/doc/images/storage_ui/automatic_size_example2.png deleted file mode 100644 index 0da03a65f3..0000000000 Binary files a/doc/images/storage_ui/automatic_size_example2.png and /dev/null differ diff --git a/doc/images/storage_ui/automatic_size_example3.png b/doc/images/storage_ui/automatic_size_example3.png deleted file mode 100644 index 052c75426b..0000000000 Binary files a/doc/images/storage_ui/automatic_size_example3.png and /dev/null differ diff --git a/doc/images/storage_ui/boot_config_popup.png b/doc/images/storage_ui/boot_config_popup.png deleted file mode 100644 index 01e1096d53..0000000000 Binary files a/doc/images/storage_ui/boot_config_popup.png and /dev/null differ diff --git a/doc/images/storage_ui/change_location_popup.png b/doc/images/storage_ui/change_location_popup.png deleted file mode 100644 index 810d7aaf6c..0000000000 Binary files a/doc/images/storage_ui/change_location_popup.png and /dev/null differ diff --git a/doc/images/storage_ui/default_device_popup.png b/doc/images/storage_ui/default_device_popup.png deleted file mode 100644 index 2f88ac5a59..0000000000 Binary files a/doc/images/storage_ui/default_device_popup.png and /dev/null differ diff --git a/doc/images/storage_ui/find_space.png b/doc/images/storage_ui/find_space.png deleted file mode 100644 index 42aa7e9892..0000000000 Binary files a/doc/images/storage_ui/find_space.png and /dev/null differ diff --git a/doc/images/storage_ui/transactional.png b/doc/images/storage_ui/transactional.png deleted file mode 100644 index 68177aeb74..0000000000 Binary files a/doc/images/storage_ui/transactional.png and /dev/null differ diff --git a/doc/storage.md b/doc/storage.md new file mode 100644 index 0000000000..5683596749 --- /dev/null +++ b/doc/storage.md @@ -0,0 +1,184 @@ +# Storage considerations + +This document describes several aspects of Agama's approach to storage configuration. + +All the user-facing information has been moved to the +[repository](https://github.com/agama-project/agama-project.github.io) containing the Agama +documentation. This document is maintained here for the following purposes. + + - Document the rationale behind some design decisions. + - Recap implementation details or other information that is too technical for user-oriented + documents. + - Record aspects that are still under discussion. + +## Agama and YaST + +This section describes some of the main differences between the Agama and YaST approaches. + +### Volumes in the YaST Proposal + +The YaST proposal heavily relies in the concept of the so-called volumes. Those volumes, that are +different for every product or system role, describe the partitions or LVM logical volumes to be +created during the process. + +In YaST, every volume specifies two different kinds of lower size limits. The so-called "desired +size" that is the smallest size that is recommeded for a normal usage of that volume and the "min +size" that is the lower threshold for the volume to be minimally useful. On top of that, every +volume has a "weight", used to adjust how the available space is distributed among the volumes. + +On the other hand, the maximum size for a given volume can be configured with the optional "max +size". But that value can be overridden if LVM is used by the also optional "max size LVM". + +Experience has shown that people in charge of defining the volumes for each product struggle to +grasp the concepts of desired size, min size and weight. The flexibility and level of customization +they provide doesn't seem to pay off for the confusion they introduce. + +Volumes at Agama will only have a minimum size and (optionally) a maximum one. No "desired size", +"weight" or "max size LVM". + +### The Initial Proposal + +Currently YaST tries really hard to present an initial proposal to the user, even if that implies +several subsequent executions of the `GuidedProposal`, each of them with a less ambitious +configuration. For that it relies on two features of the so-called volumes. + +- First of all, every volume specifies both a "min size" and a "desired size". +- On the other hand, some features of a volume are marked as optional in the control file. That + includes the usage of snapshots, the ability to expand based on the RAM size or even the existence + of the volume at all. + +YaST performs an initial execution of the `GuidedProposal` using the desired sizes as starting point +and with all the optional features set at their recommended values. If that fails, it runs +subsequent attempts until a proposal is possible. For that it fallbacks to the min sizes and +disables volumes (or volume features) in the order specified in the control file. It also explores +the possibility of using the different disks found on the system. + +That behavior almost guarantees that YaST can make a storage proposal so it's possible to install +with an empty AutoYaST profile or by simply clicking "next, next, next" in the interactive +installer. But it is not very self-explanatory. To somehow explain what happened, YaST shows a +sentence like these next to the result of the current proposal: + +- "_Initial layout proposed with the default Guided Setup settings_" +- "_Initial layout proposed after adjusting the Guided Setup settings_" (see screenshot). + +![Guided Setup result at YaST](images/storage/yast_guided_result.png) + +As mentioned before, Agama doesn't need to replicate all YaST behaviors or to inherit its +requirements and expectations. It's possible to adopt the same approach or to go all the way in the +other direction and try by default to execute the storage proposal only once, with: + + - A single disk as target (chosen by any criteria) + - The default product strategy for making space (eg. wiping the content of the disk) + - Using the default settings for all volumes + +If that execution of the proposal fails, then Agama could simply show a message like: +"it was not possible to calculate an initial storage layout". + +### Reusing LVM Setups + +For historical reasons, YaST tries to reuse existing LVM volume groups when making a proposal. That +behavior can be very confusing in many situations. To avoid the associated problems, the Agama +storage proposal will not automatically reuse existing LVM structures. + +To reuse existing volume groups the user must explicitly specify that. See the section "future +features". + +## Agama and AutoYaST + +The relationship between the Agama storage schema and the old AutoYaST format is described +at a [separate document](./autoyast_storage.md). + +## Calculating the omitted size of a file system + +If the size is omitted for a new device that directly contain a `filesystem` entry with a mount +point, Agama will then use the settings of the product to set the size limits. From a more +technical point of view, that translates into the following: + + - If the mount path corresponds to a volume supporting `auto_size`, that feature will be used. + - If it corresponds to a volume without `auto_size`, the min and max sizes of the volumes will be + used. + - If there is no volume for that mount path, the sizes of the default volume will be used. + - If the product does not specify a default volume, the behavior is still not defined (there are + several reasonable options). + +## Schema sections under discussion + +This section summarizes several aspects of the Agama storage schema that have been considered +but not implemented so far. + +### Specifying the Size of a Device + +The current schema makes it possible to specify the sizes as a fixed value or as a range. But a +a parseable string like "40 GiB" may not be the only option to represent a size or a range limit. +The following two possibilities are also under consideration. + + - `{ "gib": 40 }` + - `{ "value": 40, "units": "gib" }` + +### Searching Existing Devices + +Strings may be used as value for `search` to locate a device by its name or to search all existing +devices using "\*". But strings may be useful in other situations. + +For example, "next" (or any similar term) could be used to represent the default search for drives +(which is something like `{ "sort": { "property": "name" }, "max": 1, "ifNotFound": "error" }`. + +If a simple string like "next" could be used to specify the standard search entry for drives, it +would make sense to simply make `search` mandatory for all drives instead of assuming a default one. + +Another possible improvement for that string-based format would be supporting regular expressions. +That would make it possible to use searchers like this. + +```json +{ "search": ".*" } +``` + +But regular expressions would not play well with libstorage-ng. Since not all device names are +stored in the devicegraph, it is is necessary to use functions like `find_by_any_name` in order to +perform an exhaustive search by name. + +Another aspect under discussion is the format to specify conditions. Instead of the format described +above, it would be possible to use the key as name of the property, resulting in something like this. + +```json +{ + "search": { + "condition": { "sizeGib": 1, "operator": "greater" } + } +} +``` + +More formats for the conditions are being considered, like the one displayed at the next examples. + +```json +"condition": { "size": { "greater": "1 GiB" } } +``` + +```json +"condition": { "size": { "greater": "1 GiB", "smaller": "10 GiB" } } +``` + +```json +"condition": { "name": { "match": "^/dev/system" } } +``` + +### Referencing Other Devices + +In addition to aliases, a `search` section could be accepted in all the places in which an alias can +be used. In that case, the scope of the search would always be the whole set of devices in the +system (so the same conditions can be matched by a disk, a partition, an LVM device, etc.) and +`ifNotFound` could not be set to "create" (similar to what happens for drives in general). + +```json +"storage": { + "volume_groups": [ + { + "name": "newVG", + "physicalVolumes": [ + { "search": { "condition": { "property": "name", "value": "/dev/sda2" } } } + ], + "logicalVolumes": [ { "name": "data", "size": "20 GiB" } ] + } + ] +} +``` diff --git a/doc/storage_ui.md b/doc/storage_ui.md deleted file mode 100644 index 1393cd3271..0000000000 --- a/doc/storage_ui.md +++ /dev/null @@ -1,358 +0,0 @@ -# A Proposal for the Storage User Interface - -## Previous Considerations - -### Don't Take Mock-ups Too Seriously - -First of all, bear in mind the screenshots are far from being a faithfull representation of the -final look & feel. This document presents the concept focusing on the elements that should be there -and how they will interact. Something that is represented as a sentence in the screenshots can -become a tool-tip, a given icon can become a label, actions grouped in a drop-down can end up -being represented as separate buttons, etc. - -### Transactional Systems - -Agama is able to install transactional distributions like openSUSE MicroOS. There will be no option -in the Agama storage user interface to set whether the root file system of the installed system -should be transactional (also known as "immutable") or not. Since the implications go beyond the -file system settings, the nature of the system (transactional vs read-write) will be determined by -the selection of the operating system to install, "product" in Agama jargon. - -This document describes the Agama interface for a traditional (non-transactional) system. See the -section "interface changes for transactional systems" for the alternative. - -### Representation of the Result - -The result of the storage setup is represented in the mockups of this document as a section of the -storage page titled "result", which includes a list of (libstorage-ng) actions and a table -representing the final state of the affected devices. That is far from ideal, but a complete design -for a more convenient representation of the result is out of the scope of this proposal. - -In the long term, we may need to come with a better alternative to show the result. - -## General Workflow - -Having all the previous considerations in mind, let's describe how the general user interaction will -work. - -The summary page of Agama would display the result of the current storage proposal (or a -message about the failed initial calculation) and a link to modify that layout. That link will lead -to the page that allows to (re)configure and (re)calculate the storage proposal and that is -described at *The Proposal Page*. - -The Agama storage proposal will be the only mechanism to define the file systems of the new -operating system, including their mount points, subvolumes and options for formatting or mounting. -This proposal is based on the `GuidedProposal` implemented by the YaST libraries and presented as -"Guided Setup" in the YaST user interface. As such, the wanted file systems will actually be defined -as a set of so-called volumes very similar to the YaST ones (see *Volumes in the YaST Proposal* for -more information). The term "volume" is intentionally avoided in the user interface, using the term -"file system" instead. Nevertheless this document uses both terms indistinctly. - -Sometimes a previous setup may be needed in order to prepare the devices used by that proposal -mechanism. That includes actions like connecting to some iSCSI disks, activating and formating -DASDs, creating a software-defined RAID or setting an advanced LVM layout potentially including -several volume groups or thin-provisioned volumes. Those actions will in general modify the system -right away, instead of just planning actions to be performed during installation. Access to those -preliminary actions will be available from the Agama storage page. The general functionality is -briefly described at *Advanced Preparations*. - -## The Proposal Page - -### Overall Description of the Proposal - -The following interface will allow to configure the Agama storage setup for installation. Note the -mock-ups do not display an initial proposal, but the status after some manual changes done by the -user. - -![Initial storage screen](images/storage_ui/agama_guided.png) - -Every change to any of the configuration options will result in an immediate re-calculation of the -section that presents the result. Changes in the configuration of encryption, btrfs snapshots or the -target devices can also imply refreshing the description of the file systems. In a similar way, -changes in those volumes or the target device may result in a change in the number of disks -mentioned in the sentence about finding space. - -The table with the file systems actually represents the volumes used as input for the Agama variant -of the `GuidedProposal`. Compared to YaST, Agama turns the volumes into a much more visible concept. -The users will be able to see and adjust most of their attributes. Users could even define new -volumes that are not initially part of the configuration of the selected product. - -Pop-up dialogs will be used to modify the target device(s), the encryption configuration, the -booting setup or the strategy to find free space, as well as to add or edit a given volume. - -All file systems will be created by default at the chosen target disk or at the default LVM volume -group (in the case of LVM-based proposal). The user will be able to manually overwrite the location -of any particular volume. That has been done in the mock-up about both for the swap volume, which -will be created in an alternative disk, and for the `/home` one, that will reuse the existing file -system at the `vdb1` partition. Continue reading to understand all the possible options. - -Defining the settings and the list of volumes also defines, as a direct consequence, the disks -affected by the installation process. It may be needed to make some space in those disks. That -deserves a dedicated "find space" setting that is described below. - -### Device Selection - -As seen on the image above, the main device to install the system can be chosen at the very top of -the storage proposal page. Although a Linux installation can extend over several disks, the storage -proposal algorithm and its configuration is better understood if one device is chosen as the main -target one. That device can be a single disk (or equivalent device) or an LVM volume group. - -![Dialog to select installation device](images/storage_ui/default_device_popup.png) - -When the main target device is a disk, the volumes will be created by default as new partitions -there. If a new LVM volume group is chosen as installation device, the selection of disks indicates -which devices will be partitioned in order to allocate the physical volumes of the new volume group. -In that case, the file systems will be created by default as new LVM logical volumes at that new -volume group. - -### General Settings - -The device selection is followed by some global settings that define how the installation is going -to look and what are the possibilities in terms of booting and structuring the file systems. Those -settings include the usage of btrfs snapshots, which in YaST is presented relatively hidden as one -of the configuration options for the root file system. - -### File Systems - -The "settings" section also contains the table that displays the file systems to be created, volumes -in YaST jargon. The size of each volume is specified as a couple of lower and upper limits (the -upper one is optional in all cases). With the current approach of the YaST `GuidedProposal` there -are some volumes that may need to recalculate those limits based on the proposal configuration (eg. -whether Btrfs snapshots are enabled) or its relationship with others volumes. Their limits will be -set as "auto-calculated" by default. For more details, see the corresponding section below. - -If btrfs is used for the root file system, it will be possible to define subvolumes for it. Those -subvolumes are represented in the same table, nested on the entry of the root file system. They can -be removed and added. The subvolume entries are collapsed by default. - -If a subvolume becomes irrelevant due to the creation of another file system (let's say a `/var/lib` -subvolume exists but a new `/var` file system is added to the table), then it will not be created by -the proposal. That's known as "subvolume masking" in the YaST internals. It's still not clear how -that will be represented in the table. - -Although btrfs subvolumes don't have sizes. We might consider to add support for defining btrfs -quotas. In that case, the quota could be specified for each entry taking advantage of the already -existing column "size limits", although those quotas don't really affect the size calculations -performed by the proposal. - -All file systems will be created on the installation device by default. But it will be possible to -specify an an alternative location using the following form that offers several options. - -![Dialog to change a volume location](images/storage_ui/change_location_popup.png) - -When the option to reuse an existing device is chosen, size limits cannot be adjusted. The size of -the reused device will be displayed in the table of file systems in the corresponding column. - -### Configuration of Booting Partitions - -One of the main features of the `GuidedProposal` is its ability to automatically determine any extra -partition that may be needed for booting the new system, like PReP, EFI, Zipl or any other described -at the [corresponding YaST -document](https://github.com/yast/yast-storage-ng/blob/master/doc/boot-requirements.md). The -algorithm can create those partitions or reuse existing ones that are already in the system if the -user wants to keep them (see the section about finding space). The behavior of that feature can be -also be tweaked in the "settings" section of the page. - -![Dialog to configure booting](images/storage_ui/boot_config_popup.png) - -### Finding Space for the Volumes - -Similar to YaST, Agama will offer by default the option to automatically make space for the new -operating system. But the algorithm will be different and less configurable, offering basically -three automatic modes. - -As an alternative, the Agama proposal will offer a custom mode in which the user will explicitly -select which partitions to keep, delete or resize. - -That will result in up to four possibilities presented at a pop-up dialog if the user clicks on the -corresponding option at the botton of the "settings" section. - -- Delete everything in the disk(s). Obviously, all previous data is removed. -- Shrink existing partition(s). The information is kept, but partitions are resized as needed to make - enough space. -- Do not modify existing partition(s). The installation will only succeed if the disk(s) already - contains suitable free spaces. -- Custom. The user interface will allow the user to specify what to do with every individual partition - in the affected disks: delete it, keep it as it is or allow the algorithm to shrink it based on - the needs determined by the sizes of the volumes to allocate. - -![Section to find space](images/storage_ui/find_space.png) - -If the device chosen for installation is an already existing LVM volume group (see "device selection -and general settings"), that volume group will be displayed in a very similar way to any affected -disk, making it possible to specify what to do with the pre-existing logical volumes in a way that -is analogous to partitions in a disk. - -### Automatic Size Limits - -Currently there are cases in which the lower and upper limits of a given volume are adjusted for -the `GuidedProposal` based on the following aspects: - -- Whether snapshots are activated for the root volume -- Whether the size of the volume must be influenced by the RAM size (used for suspend in the case - of swap and for Kdump in the case of the root volume) -- Whether the given volume is marked as "fallback" for another one (eg. if the separate /home is - disabled then the upper limit of the root one disappears) - -To make that possible, the size limits of the volumes that are affected by one or several of those -circumstances will be set as "auto-calculated" by default. If that's the case, a tool-tip will be -available next to each set of limits to explain the rationale of the current values. - -Let's consider the following example in which some volumes are configured like this for the product -being installed: - -```yaml -volumes: -- mount_point: "/" - min_size: 5 GiB - max_size: 20 GiB - # Sizes are multiplied by 3 if snapshots are configured - snapshots_percentage: 200 - ... -- mount_point: "/home" - min_size: 10 GiB - max_size: unlimited - # If this volume is disabled we want "/" to increase - fallback_for_max_size: "/" - ... -``` - -The list could start with something like this. - -![Automatic Sizes Example Step 1](images/storage_ui/automatic_size_example1.png) - -The reason for the "auto-calculated" value would be explained to the user via a tool-tip (or similar -mechanism) with a text similar to this (very crude wording, to be refined): - -``` -These limits are affected by:: - - The configuration of snapshots - - The presence of a separate /home volume -``` - -As a consequence of all that, if the user deletes the /home volume then the new list would be (note -the change in the automatic size limits of the root volume). - -![Automatic Sizes Example Step 2](images/storage_ui/automatic_size_example2.png) - -If, on top of that, the user also disables snapshots the new resulting list would be. - -![Automatic Sizes Example Step 3](images/storage_ui/automatic_size_example3.png) - -Of course, at any point in time the user could modify the root volume and switch to fixed (ie. not -auto-calculated) limits. In that case, the entered values would be observed and would not be -automatically recalculated anymore, despite any configuration for the default volumes. - -### Interface Changes for Transactional Systems - -As explained at the beginning of this document, Agama can install some transactional distributions. -In those systems, it makes no sense to disable Btrfs snapshots, which are required to provide the -functionality. Is not only that snapshots are mandatory in transactional systems, they are actually -used with a different purpose when compared to read-write systems. - -Thus, if the system being installed is transactional, that will be clearly stated at the top of the -page. The setting to use btrfs snapshots will not be there and the root file system will be labeled as -"transactional" in the corresponding table. - -![Interface changes for transactional systems](images/storage_ui/transactional.png) - -## Advanced Preparations - -As mentioned above, in addition to the page for defining the proposal, Agama will offer interfaces to -perform some preparatory actions like managing DASDs or setting up complex RAID or LVM layouts. -Those interfaces will never replace the storage proposal as the only way to define the file systems of -the installed system. Instead, they will operate right away in the system to configure the devices -to be used by the proposal. - -Some of those interfaces already exist, like the one that allows to connect and disconnect to iSCSI -targets or the one to manage DASDs. Currently they can be reached through a special menu at the -Agama header. We may consider other mechanisms to make them more discoverable in the future. - -There will also be interfaces to: - -- Manage software-defined RAIDs -- Manage Bcache devices -- Define custom LVM setups -- Manipulate partitions in the disks or in any of the RAID and bcache devices - -Since all those actions are interrelated (eg. the user often creates partitions that are combined -into a RAID that is then used as an LVM physical volume), the final user interface will likely -resemble the traditional YaST Expert Partitioner. But, since the scope of such a tool will be -limited to preparing the disk for the proposal, it will not allow to format devices or to define -mount points for the target system. After defining all the actions to be performed, the changes will -be committed to the system before returning to the proposal page to define the location and settings -of each file system. - -## Comparison with the YaST Proposal - -### Volumes in the YaST Proposal - -The YaST proposal heavily relies in the concept of the so-called volumes. Those volumes, that are -different for every product or system role, describe the partitions or LVM logical volumes to be -created during the process. - -In YaST, every volume specifies two different kinds of lower size limits. The so-called "desired -size" that is the smallest size that is recommeded for a normal usage of that volume and the "min -size" that is the lower threshold for the volume to be minimally useful. On top of that, every -volume has a "weight", used to adjust how the available space is distributed among the volumes. - -On the other hand, the maximum size for a given volume can be configured with the optional "max -size". But that value can be overridden if LVM is used by the also optional "max size LVM". - -Experience has shown that people in charge of defining the volumes for each product struggle to -grasp the concepts of desired size, min size and weight. The flexibility and level of customization -they provide doesn't seem to pay off for the confusion they introduce. - -Volumes at Agama will only have a minimum size and (optionally) a maximum one. No "desired size", -"weight" or "max size LVM". - -### Reusing LVM Setups - -For historical reasons, YaST tries to reuse existing LVM volume groups when making a proposal. That -behavior can be very confusing in many situations. To avoid the associated problems, the Agama -storage proposal will not automatically reuse existing LVM structures. - -To reuse existing volume groups the user must explicitly specify that. See the section "future -features". - -### About the Initial Proposal - -Currently YaST tries really hard to present an initial proposal to the user, even if that implies -several subsequent executions of the `GuidedProposal`, each of them with a less ambitious -configuration. For that it relies on two features of the so-called volumes (volumes are already -described at *Volumes in the YaST Proposal*): - -- First of all, every volume specifies both a "min size" and a "desired size". -- On the other hand, some features of a volume are marked as optional in the control file. That - includes the usage of snapshots, the ability to expand based on the RAM size or even the existence - of the volume at all. - -YaST performs an initial execution of the `GuidedProposal` using the desired sizes as starting point -and with all the optional features set at their recommended values. If that fails, it runs -subsequent attempts until a proposal is possible. For that it fallbacks to the min sizes and -disables volumes (or volume features) in the order specified in the control file. It also explores -the possibility of using the different disks found on the system. - -That behavior almost guarantees that YaST can make a storage proposal so it's possible to install -with an empty AutoYaST profile or by simply clicking "next, next, next" in the interactive -installer. But it is not very self-explanatory. To somehow explain what happened, YaST shows a -sentence like these next to the result of the current proposal: - -- "_Initial layout proposed with the default Guided Setup settings_" -- "_Initial layout proposed after adjusting the Guided Setup settings_" (see screenshot). - -![Guided Setup result at YaST](images/storage_ui/yast_guided_result.png) - -As mentioned before, Agama doesn't need to replicate all YaST behaviors or to inherit its -requirements and expectations. It's possible to adopt the same approach or to go all the way in the -other direction and try by default to execute a variant of the `GuidedProposal` only once, with: - - - A single disk as target (chosen by any criteria) - - A simple strategy for making space (eg. wiping the content of the disk) - - Using the default settings for all volumes - -If that execution of the `GuidedProposal` fails, then Agama could simply show a message like: -"it was not possible to calculate an initial storage layout". - -The interface proposed in this document will work equally whatever approach is decided for the -initial storage proposal of Agama.