feat!: implement "splicing" for --base-url + --root-dir

[katrinafyi]

when both --base-url and --root-dir are given, this implements the behaviour described in https://www.github.com/lycheeverse/lychee/issues/1718#issuecomment-3170382677. i've taken to calling this "splicing".

lychee/README.md

Lines 548 to 595 in ff72bd6

	-b, --base-url <BASE_URL>
	Remote base URL where the local root-dir will be hosted. If `--base-url` is
	specified, `--root-dir` must be specified as well.
	When both `--base-url` and `--root-dir` are specified, then links will be resolved
	*as if* the local root-dir was hosted at the given base-url.
	This is done by virtually "splicing" the root-dir onto the base-url path. This
	works in both directions: (1) links to subpaths of base-url will be resolved to
	local files within root-dir, with consideration to the relative subpath, and
	(2) links originating from local files which traverse outside of base-url will
	resolve to remote URLs on the internet.
	The two directions are demonstrated in the examples below. For these examples,
	suppose a base URL of `https://example.com/dir/` and root dir of `/tmp/root`.
	- (1) A link to `https://example.com/dir/sub/boop.html` will be resolved to
	the local file `/tmp/root/sub/boop.html` because it is a subpath of base-url.
	The relative subpath of `/sub/boop.html` is mapped into the root-dir.
	- (2) A link in `/tmp/root/index.html` to `../up.html` or `/up.html` will be
	resolved to the remote URL `https://example.com/up.html` because it traverses
	outside of base-url.
	--root-dir <ROOT_DIR>
	Root directory to use when checking local files. This option is required if
	absolute links appear in local files, otherwise those links will be flagged as
	errors. This must be an absolute path (i.e., one beginning with `/`).
	If specified, `--root-dir` acts according to three main rules:
	- Links are resolved *as if* the given root-dir was hosted at the root of a
	website. For example, with a root-dir of `/tmp`, a link in `/tmp/a/index.html`
	to `/page.html` would be resolved to `/tmp/page.html`.
	- `--root-dir` only applies to links originating from files which are subpaths
	of the given root directory. Other links will be unaffected (e.g., absolute
	links from files outside of root-dir will still fail to be found).
	- `--root-dir` also serves to limit parent path traversal. With a root-dir of
	`/tmp`, a link in `/tmp/index.html` to `../up.html` would be resolved to
	`/tmp/up.html` and not `/up.html`. This is because if `/tmp` was uploaded to
	a website root, traversing up beyond the root would not change the path.
	Additiionally, this option can be specified alongside `--base-url`. If both are
	given, the behavior is augmented to resolve links as if `--root-dir` was
	available at the remote URL of `--base-url`. See the help of `--base-url` for
	more information.

this gives a sensible interpretation when both flags are used together, enabling uses like this example from the docs:

lychee --base-url https://example.com/docs/ --root-dir $(pwd)/public/ "public/**/*.html"

this significantly changes the meaning of --base-url, introducing an association between --base-url and --root-dir. the base-url option now requires root-dir in order to know which directory to map to base-url.

something like the previous base-url behaviour is provided in --fallback-base-url, though it differs in some ways. in particular, it applies in slightly fewer situations; it does not apply where a more canonical base-url is available (see help).

--root-dir, when specified alone, is unchanged.

see also: full options help for the new options

notes for the reviewer:

• the majority of the logic is implemented in https://github.com/rina-forks/lychee/blob/easy-mode/lychee-lib/src/types/base_info.rs. this implements most of the URL parsing by calling methods on reqwest::Url, including local path handling which was previously done manually.
• additional url-manipulating functions are also added as ReqwestUrlExt in https://github.com/rina-forks/lychee/blob/easy-mode/lychee-lib/src/utils/url.rs.
• the old functions in lychee-lib/src/utils/request.rs are kept, but they are unused now. i've kept them around for now to minimise diff noise but i aim to delete before merging.

todo:

• comments
• tests, going through all the cases i've thought of
• e2e cli tests
• delete unused functions.
• re-enable werror.
• talk about trailing slash in base-url help.
• add check in mappinfs that if one side is a dir then the other side should be too
• check that fallback base url can then be remapped through root dir+base url
• think about how to do backwards compatibility. how do enable a global root-dir setting?

[katrinafyi]

need to fix https://github.com/rina-forks/lychee/pull/3/files#diff-91c9664581cb134f51a915a79781a5762de37142f67349f59060d277c0dc3a71R35 so it more strictly ensures a subpath relation

lychee/lychee-lib/src/utils/url.rs

Lines 35 to 41 in 15fc377

	fn strip_prefix(&self, prefix: &Url) -> Option<String> {
	prefix
	.make_relative(self)
	.filter(|subpath| !subpath.starts_with("../") && !subpath.starts_with('/'))
	// .inspect(|x| println!("subpathing {}", x))
	// .filter(|_| prefix.as_str().starts_with(self.as_str()))
	}

[katrinafyi]

problems with base-url:

• if require /docs/ then we might miss a url like /docs. does it matter. patch with remap?
• if require /docs then /docs/ cannot be easily constructed relatively. needs docs/ for example.

if directory base url, then go as normal. if file base url, then the only subpath is the file itself.