Skip to content

WIP: Get IPFS store working with StoreConfig refactor#78

Merged
Ericson2314 merged 62 commits intoipfs-binary-cache-developfrom
ipfs-mini-store-config-merge
Sep 23, 2020
Merged

WIP: Get IPFS store working with StoreConfig refactor#78
Ericson2314 merged 62 commits intoipfs-binary-cache-developfrom
ipfs-mini-store-config-merge

Conversation

@Ericson2314
Copy link
Member

No description provided.

meditans and others added 30 commits July 17, 2020 17:24
to each Store implementation. The generic regStore implementation will
only be for the ambiguous shorthands, like "" and "auto".

This also could get us close to simplifying the daemon command.
Removes duplicate websocket opening code, and also means we should be
able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
Add a new `init()` method to the `Store` class that is supposed to
handle all the effectful initialisation needed to set-up the store.
The constructor should remain side-effect free and just initialize the
c++ data structure.

The goal behind that is that we can create “dummy” instances of each
store to query static properties about it (the parameters it accepts for
example)
Don't let it just contain the value, but also the other fields of the
setting (description, aliases, etc..)
The default value is initialized when creating the setting and unchanged
after that
Rework the `Store` hierarchy so that there's now one hierarchy for the
store configs and one for the implementations (where each implementation
extends the corresponding config). So a class hierarchy like

```
StoreConfig-------->Store
    |                 |
    v                 v
SubStoreConfig----->SubStore
    |                 |
    v                 v
SubSubStoreConfig-->SubSubStore
```

(with virtual inheritance to prevent DDD).

The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
Using the `*Config` class hierarchy
Using virtual inheritance means that only the default constructors of
the parent classes will be called, which isn't what we want
When opening a store, only try the stores whose `uriSchemes()` include
the current one
Allow `-` and `.` in the RFC schemes as stated by
[RFC3986](https://tools.ietf.org/html/rfc3986#section-3.1).

Practically, this is needed so that `ssh-ng` is a valid URI scheme
So that it can be printed by `nix describe-stores`
Work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431 that was
already there in the code but was accidentally removed in the last
commits
It is apparently required for using `toJSONObject()`, which we do inside
the header file (because it's in a template).

This was accidentally working when building Nix itself (presumably because
`config.hh` was always included after `nlohman/json.hpp`) but caused a
(pretty dirty) build failure in the perl bindings package.
Instead make a separate header with the template implementation of
`BaseSetting<T>::toJSONObj` that can be included where needed
Add some necessary casts in the initialisation of the store's config
Doesn't test much, but at least ensures that the command runs properly
maljub01 and others added 27 commits September 16, 2020 17:56
Mac OS multi-user installations are currently broken because all requests
made by nix-daemon to the binary cache fail with:

```
unable to download ... Problem with the SSL CA cert (path? access rights?) (77).
```

This change ensures that the nix-daemon knows where to find the SSL CA cert file.

Fixes NixOS#2899 and NixOS#3261.
This isn't used anywhere except in the configure script of the Perl
bindings. I've changed the latter to use the C++ API's Settings object
at runtime.
Fix garbage collection of CA derivations
…elegate-regStore

Remove storetype delegate reg store -- contains NixOS#3736
find() returns an iterator so "!attr" doesn't work.
Since 108debe we allow a
`url`-attribute for the `github`-fetcher to fetch tarballs from
self-hosted `gitlab`/`github` instances.

However it's not used when defining e.g. a flake-input

    foobar = {
        type = "github";
        url = "gitlab.myserver";
        /* ... */
    }

and breaks with an evaluation-error:

    error: --- Error --------------------------------------nix
    unsupported input attribute 'url'
    (use '--show-trace' to show detailed location information)

This patch allows flake-inputs to be fetched from self-hosted instances
as well.
pkgs.fetchurl supports an executable argument, which is especially nice
when downloading a large executable. This patch adds the same option to
nix-prefetch-url.

I have tested this to work on the simple case of prefetching a little
executable:

1. nix-prefetch-url --executable https://my/little/script
2. Paste the hash into a pkgs.fetchurl-based package, script-pkg.nix
3. Delete the output from the store to avoid any misidentified artifacts
4. Realise the package script-pkg.nix
5. Run the executable

I repeated the above while using --name, as well.

I suspect --executable would have no meaningful effect if combined with
--unpack, but I have not tried it.
nix-prefetch-url: Add --executable flag
Fix the nix-daemon Mac OS SSL CA cert
Fix NixOS#4042

According to kristapsdz/lowdown@8aef9e9, we shouldn't need to use a fork anymore so we can switch back to upstream
They're still enabled in regular builds though.
libfetchers/github: allow `url` attribute
This reduces compilation time by ~15 seconds (CPU time).

Issue NixOS#4045.
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).

Issue NixOS#4045.
This reduces compilation time by 207s.

Issue NixOS#4045.
This cuts compilation time by ~49s.

Issue NixOS#4045.
@Ericson2314
Copy link
Member Author

I think it's probably best to just merge this then fix.

@Ericson2314 Ericson2314 merged commit c3f3452 into ipfs-binary-cache-develop Sep 23, 2020
@Ericson2314 Ericson2314 deleted the ipfs-mini-store-config-merge branch September 23, 2020 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants