[RFC] add initial proposal for unattended questions by jreidinger · Pull Request #595 · agama-project/agama

jreidinger · 2023-05-29T21:02:59Z

Problem

With introduced questions mechanism it is impossible to have real unattended installation.

trello: https://trello.com/c/eMVlSQnU/3364-2-agama-research-design-a-mechanism-to-ask-questions-during-auto-installation

Solution

Create proposal how to modify questions and add support for unattanded installation to them.

coveralls · 2023-05-29T21:10:21Z

Pull Request Test Coverage Report for Build 5153284815

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 75.686%

Totals
Change from base Build 5081178664:	0.0%
Covered Lines:	5396
Relevant Lines:	6890

💛 - Coveralls

lslezak · 2023-05-30T13:41:23Z

doc/questions.md

+1. each question has ID that is hierarchical so partial match can be used. Example `storage.luks.password` or
+`software.repository.checksum`
+2. each question has defined default safe answer.
+3. questions can be up to 4 additional params that helps user to answer them. Example repository url and its checksum.


Why 4? What about using a single Hash as a parameter? That would also allow to name the parameters, like
{url: "https://example.com", checksum: "123456..."}

well, I consider 4 as reasonable number..but it is true, that hash can be also helpful and more flexible.

lslezak · 2023-05-30T13:50:29Z

doc/questions.md

+3. For questions without defined answer default answer will be used.
+4. All questions and answers ( together with source from where it comes )
+   will be logged for later audit ( maybe own file? or maybe directly write
+   it as answers.yml? )


I'm not sure if it is a good idea, you do not want to have your LUKS password to be stored in any file.
Even URLs might contain passwords, we have special URL::HidePassword() method in YaST for logging passwords.

OK, I can add there that sensitive data is filtered out and replaced by some <sensitive> string?

Yes, some filtering should be done.
RoR uses [FILTERED] IIRC...

lslezak · 2023-05-30T13:51:38Z

doc/questions.md

+1. I run unattended installation for first time and want to see what questions appear to be able to
+modify it to my needs for mass deployment. -> There is questions.log which writes down exact questions
+with all its params, ids, etc and used answer with note if it is automatic answer or one comming from
+file.


As written above, saving the user's answers might have security implications, that should only happen on explicit user request.

lslezak · 2023-05-30T14:25:32Z

doc/questions.md

+  GPG key that I know in advance. -> use answers.yml file
+3. I do mass deployment in environment where previous requirement was to have luks encryption with some
+   random password and this partition should not be reused -> do nothing as default answer is to skip such partition
+4. I have my own vendor iso and I want to preconfigure installer using CLI before showing web UI. And some actions can/will


I'm not sure if we should support this scenario. I'd expect the vendors would only touch /etc/agama.yaml and possibly some branding files. I feel that this is rather abusing the answers for some different case. We could configure Agama to death.

If we want to have some default answers configurable I'd prefer to configure them via /etc/agama.yaml and do not mix manual and automatic installation to some hybrid solution. That would be difficult to test and maintain, I'd rather not open that box...

E.g. if you want to accept an additional GPG key from a custom repository let's define that together with the URL in the config file.

well, basically question is if there is reason to have answers and interactive mode or not ( so basically answer only known questions and for rest ask user instead of using auto reply ).

doc/questions.md

mvidner

I'd like to know how this design affects the D-Bus API

doc/questions.md

Co-authored-by: Martin Vidner <mvidner@suse.cz>

mvidner · 2023-05-31T07:40:44Z

never trust robots that you do not program yourself :)

@jreidinger very true, you still have to proofread what DeepL gave you to check that it didn't change the meaning.

mvidner · 2023-05-31T09:26:31Z

So again, I'd like to know how this design affects the D-Bus API.

I can imagine a design, where all of the unattended handling just prevents some/all of the Questions from appearing on the bus so the API would stay the same.

If the API does change, I think it is worthwhile specifying now in this PR

joseivanlopez · 2023-05-31T11:09:27Z

My thoughts:

I agree with:
- Having a file with the provided answers.
- With a matching mechanism (e.g., what type of question, what device, what repo, etc).
- With a default answer (but optional).
I disagree with:
- Having modes (auto and manual).
- Having extra data (for matching) in the question itself.

How do I envision it?

Passing an optional (and encrypted?) file with the answers (similar to autoinstall profile).
Having a matching mechanism as proposed.
Having an optional default answer for each question.
A question is not asked if its answer is already known (why to ask for luks password if it was already provided by an answers file?).
No difference between auto and manual install. If the answer is known, then take it. Otherwise, ask a question.

mvidner · 2023-05-31T11:46:35Z

doc/questions.md

+#### Unattended Installation Features
+
+1. Questions contain values in addition to ids that can be matched. To match
+id and values, string, regexp or array can be used. Array lists all possible values and regexp is used for matching.


IMHO regexp (regular expressions) are overkill (incompatible dialects...) and I hope substrings or shell patterns will be enough

that is good suggestion. Maybe for the first phase we can omit regexp completely and if needed, we can extend it ( so some string that start with some fancy character to support advanced matching ).

mvidner · 2023-05-31T11:52:26Z

To avoid Mode in the service...
Another (crazy?) design: the service just asks a question on the bus, and there is an "auto answerer client" that answers all questions

jreidinger · 2023-05-31T15:19:29Z

My thoughts:

* I agree with:
  
  * Having a file with the provided answers.
  * With a matching mechanism (e.g., what type of question, what device, what repo, etc).
  * With a default answer (but optional).

* I disagree with:
  
  * Having modes (auto and manual).
  * Having extra data (for matching) in the question itself.

How do I envision it?

* Passing an optional (and encrypted?) file with the answers (similar to autoinstall profile).

* Having a matching mechanism as proposed.

* Having an optional default answer for each question.

* A question is not asked if its answer is already known (why to ask for luks password if it was already provided by an answers file?).

* No difference between auto and manual install. If the answer is known, then take it. Otherwise, ask a question.

Thanks for notes. I think there is a lot of agreement and just a few collisions.
I think the main one is how unattended installation should look like. For me, if we have unattended installation, it should really be unattended unless major and critical issue appear ( like not able to install anything, not able to partition, etc. ). And for that we have issues and not questions. Question is for me when installer missing some info, but should be able to continue without that info. And in unattended mode questions should never be asked from my POV.

Just to clarify few notes:

modes will be only in question object. So no global mode like in yast, just set questions object to unattended mode.
we can maybe use that extra data from questions in UI itself, as question can be more limited then data, so we can show it in some details for question or in CLI. E.g. consider some additional details about disk that is luks encrypted.
if we do not have extra data in questions for matching, we still need to somehow match answer and question, so if it is only in answer, it means that each object needs to do matching on its own, which can be quite messy.
in my proposal is also passing answers file, but if it is missing, just defaults are used. Note about encrypted is good, I will add TODO as we probably do not need it for first iteration
What you describe in the last point is basically how can happen my proposal if answers are provided and questions are not set to unattended mode. Question is if it is useful for users.

@imobachgs how do you see it? Especially about requirements for unattended mode and possible blocked by question that is not specified in advance.

imobachgs · 2023-05-31T15:30:42Z

doc/questions.md

+storage.* -> :skip.
+2. every question has a defined default secure answer. Why: it allows real unattended mode even if user does not specify all questions.
+3. questions can define additional params in map that help user to answer them. Example repository url and its checksum.
+   Partition identification for luks password. Why: Getting data only from question text is hard and very unreadable with regexp.


Not to mention translations...

jreidinger · 2023-05-31T15:41:29Z

To avoid Mode in the service... Another (crazy?) design: the service just asks a question on the bus, and there is an "auto answerer client" that answers all questions

It is of course possible and nothing prevents it. I just see some disadvantages:

default answer has to be in that client, so the clients needs to know about any possible question.
It is additional process that will need to run
If we use web UI as UI for user there is high chance, that it will quickly show questions and then it will again disappear.

imobachgs · 2023-05-31T15:54:53Z

About the "unattended mode", I see how it is useful but I also understand @joseivanlopez's concern.

I am a little bit late to the discussion but, IMHO, the important thing is to set the default behavior when an answer is not defined (using an answers.yml file or whatever mechanism). For instance, when performing an autoinstallation, you could set the behavior to use default. On a regular installation, the behavior could be ask the user.

So you do not need to model the concept of mode, but just to set the fallback behavior. Does it make any sense?

joseivanlopez · 2023-06-01T08:52:44Z

How I see it, auto-installation is exactly the same as the regular installation. The only two differences I see with auto and no auto are:

a) Auto-install automatically sets the given config.
b) Auto-install automatically starts the installation after setting the config.

What it is more, auto-install should not care about having user interaction or not. If we provide all the required info, then the auto-installation can proceed, otherwise it cannot. For example, if the "profile" does not contain user/root password, then the auto-installation cannot continue. I would do the same for questions/answers.

If we don't want to emit questions, then we need to configure our answers mechanism to do something when an answer is unknow (as Imo pointed above). For example (the format is wrong, only as idea):

questions: 
  default: continue (or abort)
  storage:
    default: abort
    luks:
     - device: /dev/sda
       password: linux123

With that approach, we would have a way to provide what to do in any case. And maybe we could also offer the answers mechanism with manual installation and CLI too. For example:

agama config set questions.default=continue
agama install

And this make me think, should questions belong to the "profile" instead of a separate file?

agama config load config.json # including settings and answers
agama config set questions.default=abort

joseivanlopez

LGTM

joseivanlopez · 2023-06-02T07:42:13Z

doc/questions.md

 ## Questions

-[Questions][] are a mechanism that allows the user to react to certain situations that occur in the installer.
+[Questions][] are a mechanism that allows the installer to ask for additional data which are needed without knowing it in advance.


which is needed

add initial proposal for unattended questions

e813d9d

lslezak reviewed May 30, 2023

View reviewed changes

mvidner reviewed May 30, 2023

View reviewed changes

doc/questions.md Outdated Show resolved Hide resolved

mvidner reviewed May 30, 2023

View reviewed changes

doc/questions.md Outdated Show resolved Hide resolved

jreidinger and others added 2 commits May 30, 2023 21:25

Update doc/questions.md

fca86f1

Co-authored-by: Martin Vidner <mvidner@suse.cz>

changes from review

6d2412c

mvidner reviewed May 31, 2023

View reviewed changes

imobachgs reviewed May 31, 2023

View reviewed changes

update from review

d64dc99

joseivanlopez approved these changes Jun 2, 2023

View reviewed changes

fix typo

a632713

jreidinger marked this pull request as ready for review June 2, 2023 08:07

jreidinger merged commit a2b3917 into master Jun 2, 2023

jreidinger deleted the questions_unattended_proposal branch June 2, 2023 08:30

jreidinger mentioned this pull request Jun 21, 2023

Change Questions D-Bus API to fit new proposal #637

Merged

2 tasks

imobachgs mentioned this pull request Aug 2, 2023

Release version 3 #683

Merged

Conversation

jreidinger commented May 29, 2023

Problem

Solution

Uh oh!

coveralls commented May 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 5153284815

💛 - Coveralls

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lslezak May 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mvidner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mvidner commented May 31, 2023

Uh oh!

mvidner commented May 31, 2023

Uh oh!

joseivanlopez commented May 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mvidner commented May 31, 2023

Uh oh!

jreidinger commented May 31, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreidinger commented May 31, 2023

Uh oh!

imobachgs commented May 31, 2023

Uh oh!

joseivanlopez commented Jun 1, 2023

Uh oh!

joseivanlopez left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

coveralls commented May 29, 2023 •

edited

Loading

lslezak May 30, 2023 •

edited

Loading

joseivanlopez commented May 31, 2023 •

edited

Loading