feat: add support for relative URLs

[imobachgs]

Problem

The relurl schema is used in several places, especially in AutoYaST profiles. It can be used to refer a script, a file, repositories, etc. They are useful because you do not need to specify the full URL, but it uses a URL which is relative to the profile URL.

The relurl is not directly supported by the FileFromUrl API, but it is implemented in different places. The goal of this PR is adding support to read files and scripts from relative locations.

relurl is an AutoYaST-specific invention.

Solution

Instead of porting relurl, introduce the concept of URL reference, well known from HTML <a href=...> and standardized in RFC3986 (Uniform Resource Identifier (URI): Generic Syntax).
A URL reference is either a URL (absolute), or a relative reference (typically just a path), resolved in relation to a base URL.

{
  files: [
    {
      destination: '/etc/issue.d/agama.issue',
      url: 'issue-sles',
    },
    {
      destination: '/etc/issue.d/readme.issue',
      url: 'http://192.168.122.1/agama/issue-readme',
    },
  ],
  scripts: {
    post: [
      {
        name: "disable-nginx",
        url: "disable-nginx.sh",
      },
      {
        name: "disable-firewall",
        url: "http://192.168.122.1/agama/disable-firewall.sh",
      },
    ],
  }
}

A new Url

The Url struct from the url crate does not support relative URLs. This PR uses fluent_uri but the old Url has not been replaced everywhere yet.

What is the base URL?

The first problem when using relative URLs is to determine which is the base URL. In this case, it is the profile's URL. We could store the URL somewhere (in the manager?) but the true is that it can change with every call to agama profile import. And what about agama config load?

For this first implementation, I have decided to inject the URL depending on the context:

• agama profile import: it uses the URL of the profile.
• agama config load: it injects the current directory. It will not work as expected when working remotely, but it should do when calling agama config load locally. Another option might be to just report an error about not being able to resolve the URL.

(Upcoming changes: according to #2289, agama config generate BASE_URL (the replacement for the most complex part of agama profile import) will be responsible for resolving any relative references, according to the explicit base given to it, and replace them with absolute URL in its JSON output. agama config load will not need to care.)

Other changes

• Fix handling of "file_owner" when importing a files definition.
• Improve test coverage of the Ruby part.

To do

• Add support for files
• Add support for scripts
• Reduce the amount of cloning (should resolve_url modify the receiver?).
• (optional) Implement YaST conversion (out of scope originally).

Testing

• Added a new unit test
• Tested manually

[mvidner]

Thanks! Commenting on the PR description so far, not the code yet:

My reason for wanting to retire relurl: is that I consider it a reinvented wheel, where the original wheel is a well understood concept from HTML+HTTP:

RFC 3986: Uniform Resource Identifier (URI): Generic Syntax
Section 5.2. Relative Resolution
https://datatracker.ietf.org/doc/html/rfc3986.html#section-5.2

We should tell the users: we just follow the standard (and ideally use a library that resolves relative URIs for us so we don't mess up and end up with a slightly incompatible wheel)

A new Url The Url struct from the url crate does not support relative URLs. This PR introduces our own Url concept, which is just an enum with a wrapper around the original Url (perhaps we should use a different name).

A search for rust relative uri resolution points me to http crate https://docs.rs/http/latest/http/uri/struct.Uri.html which claims to support relative ones.

[imobachgs]

A search for rust relative uri resolution points me to http crate https://docs.rs/http/latest/http/uri/struct.Uri.html which claims to support relative ones.

I checked that crate and it does not support URLs like this: ../something/else. That kind of URLs are rather common and I would like to get them supported.

[mvidner]

Another top hit from that search is https://docs.rs/fluent-uri/latest/fluent_uri/ which seems to target our use case.

[mvidner]

Thanks!
I think there are still some loose ends. In particular the case of the missing context.

[mvidner]

This looks OK but then I realize that url can be a relative UriRef in which case the user will get an error.
IMHO you forgot this case

[imobachgs]

I did not forgot, but I simply let Transfer to handle the error. In the future I would like to replace url::Url with fluent_uri::UriRef and have better handling here.

[mvidner]

Please add a comment about it.

[mvidner]

The code is good, but seriously we are missing documentation.

Looking at the "Solution" part of the PR description, it seems obvious to me that the minimal docs change should be the JSON schema. Please do :)

[imobachgs]

As mentioned in IRC, I decided to change the implementation a bit. The logic to resolve the URLs is now invoked from InstallSettings itself (in a single place). BTW I updated the documentation in the JSON schema. I hope it helps.

[mvidner]

The help for agama download needs adjusting

[mvidner]

Documentation! Again we are adding a secret feature. Please locate the 2 occurrences of relurl in Agama source, both are affected by this PR, one by this commit :)

[mvidner]

LGTM now, thank you!