plugins: support nix_plugin_entry, do some minor reworks

Plugins are already allowed to fail to load, let's be even more tolerant
of them failing to load by ignoring inaccessible ones with a warning.

This change also significantly reworks the docs for plugins.

This is a tiny extremely partial backport of the C API at
NixOS/nix#8699.

Fixes: https://git.lix.systems/lix-project/lix/issues/740
CC: https://git.lix.systems/lix-project/lix/issues/359

Change-Id: If4ee20c3daaf26c8184a415eef3e20ca5b5e7aef

Author: lf-

doc/manual/change-authors.yml

@@ -208,6 +208,9 @@ winter:
 xanderio:
   github: xanderio
 
+yorickvp:
+  github: yorickvp
+
 yshui:
   github: yshui
 

doc/manual/rl-next/plugin-files-nix-plugin-entry.md

@@ -0,0 +1,12 @@
+---
+synopsis: "Add `nix_plugin_entry` entry point for plugins"
+issues: [fj#740, fj#359]
+prs: [gh#8699]
+cls: [2826]
+category: Development
+credits: ["jade", "yorickvp"]
+---
+
+Plugins are an exceptionally rarely used feature in Lix, but they are important as a prototyping tool for code destined for Lix itself, and we want to keep supporting them as a low-maintenance-cost feature.
+As part of the overall move towards getting rid of static initializers for stability and predictability reasons, we added an explicit `nix_plugin_entry` function like CppNix has, which is called immediately after plugin load, if present.
+This makes control flow more explicit and allows for easily registering things that have had their static initializer registration classes removed.

lix/libstore/globals.cc

@@ -357,6 +357,10 @@ Paths PluginFilesSetting::parse(const std::string & str, const ApplyConfigOption
     return BaseSetting<Paths>::parse(str, options);
 }
 
+// C++ syntax so weird that it breaks the tree-sitter highlighter!
+// *Technically* the C linkage function pointer should be so annotated.
+// Does it actually matter? Almost certainly not!
+extern "C" using NixPluginEntry = void (*)();
 
 void initPlugins()
 {
@@ -365,20 +369,36 @@ void initPlugins()
         Paths pluginFiles;
         try {
             auto ents = readDirectory(pluginFile);
-            for (const auto & ent : ents)
+            for (const auto & ent : ents) {
                 pluginFiles.emplace_back(pluginFile + "/" + ent.name);
+            }
         } catch (SysError & e) {
-            if (e.errNo != ENOTDIR)
-                throw;
+            if (e.errNo != ENOTDIR) {
+                // I feel like it is more reasonable to skip plugins if they are
+                // inaccessible, since it is *already* the case that plugins
+                // are not guaranteed to load due to version mismatches etc
+                // causing dlopen failures.
+                warn("could not access plugin file '%s', skipping it: %s", pluginFile, e.msg());
+                continue;
+            }
             pluginFiles.emplace_back(pluginFile);
         }
         for (const auto & file : pluginFiles) {
             /* handle is purposefully leaked as there may be state in the
                DSO needed by the action of the plugin. */
             void *handle =
                 dlopen(file.c_str(), RTLD_LAZY | RTLD_LOCAL);
-            if (!handle)
-                warn("could not dynamically open plugin file '%s': %s", file, dlerror());
+            if (!handle) {
+                warn("could not dynamically open plugin file '%s', skipping it: %s", file, dlerror());
+                continue;
+            }
+
+            /* Older plugins use a statically initialized object to run their code.
+               Newer plugins can also export nix_plugin_entry() */
+            auto nix_plugin_entry = reinterpret_cast<NixPluginEntry>(dlsym(handle, "nix_plugin_entry"));
+            if (nix_plugin_entry) {
+                nix_plugin_entry();
+            }
         }
     }
 

lix/libstore/settings/plugin-files.md

@@ -4,24 +4,31 @@ internalName: pluginFiles
 settingType: PluginFilesSetting
 default: []
 ---
-A list of plugin files to be loaded by Nix. Each of these files will
-be dlopened by Nix, allowing them to affect execution through static
-initialization. In particular, these plugins may construct static
-instances of RegisterPrimOp to add new primops or constants to the
-expression language, RegisterStoreImplementation to add new store
-implementations, RegisterCommand to add new subcommands to the `nix`
-command, and RegisterSetting to add new nix config settings. See the
-constructors for those types for more details.
-
-Warning! These APIs are inherently unstable and may change from
-release to release.
-
-Since these files are loaded into the same address space as Nix
-itself, they must be DSOs compatible with the instance of Nix
-running at the time (i.e. compiled against the same headers, not
-linked to any incompatible libraries). They should not be linked to
-any Lix libs directly, as those will be available already at load
-time.
-
-If an entry in the list is a directory, all files in the directory
-are loaded as plugins (non-recursively).
+A list of plugin files to be loaded by Lix.
+
+Each of these files will be `dlopen`ed by Lix, allowing them to affect execution by registering various entities in Lix.
+After the plugins are loaded, they will have the function within them with signature `extern "C" void nix_plugin_entry(void)` called if it is defined.
+
+If an entry in the list is a directory, all files in the directory are loaded as plugins (non-recursively).
+
+FIXME(jade): We should provide a `nix_plugin_finalize()` that gets called at some point in teardown for use cases like nix-otel which need to be able to cleanup, flush things to network, etc, on exit without having to do that from life-after-main().
+
+In particular, these plugins may:
+- Construct static instances of `RegisterPrimOp` to add new primops or constants to the expression language (FIXME: will be replaced with an explicit function).
+- Add new store implementations with `StoreImplementations::add`.
+- Construct static instances of `RegisterCommand` to add new subcommands to the `nix` command (FIXME: will be replaced with an explicit function).
+- Construct static instances of `Setting` to add new Lix settings (FIXME: will be replaced with an explicit function).
+
+See the documentation for those symbols for more details.
+Note all the FIXMEs above: Lix is removing its usages of static initializers, see <https://git.lix.systems/lix-project/lix/issues/359>.
+
+Warning! These APIs are inherently unstable and may change in minor versions.
+It's recommended to, if you *are* relying on Lix's unstable C++ API, develop against Lix `main`, run `main` yourself, and be active in Lix development.
+
+Since these files are loaded into the same address space as Lix itself, they must be DSOs compatible with the instance of Lix running at the time (i.e. compiled against the same headers, not linked to any incompatible libraries, produced by the same nixpkgs).
+
+It's recommended that this setting *not* be used in `nix.conf` since it is almost always the case that there are multiple versions of the Nix implementation on a machine.
+In particular, CppNix (still true in 2.26 as of this writing) considers plugin load failure to be a hard error unlike Lix (since pre-2.90), which means that putting `plugin-files` in `nix.conf` causes random `nix` execution failures.
+Prefer instead to wrap the `nix` command using either the `NIX_CONFIG` environment variable or `--option plugin-files`.
+
+Plugins should not be linked to any Lix libs directly, as those will be available already at load time (FIXME: is it an actual problem if they are, assuming that there are not version mismatches?).

tests/functional/plugins/plugintest.cc

@@ -16,6 +16,8 @@ struct MySettings : Config
         "Whether the plugin-defined setting was set"};
 };
 
+bool entryCalled = false;
+
 MySettings mySettings;
 
 static GlobalConfig::Register rs(&mySettings);
@@ -27,6 +29,7 @@ static void maybeRequireMeowForDlopen() {
 
 static void prim_anotherNull (EvalState & state, const PosIdx pos, Value ** args, Value & v)
 {
+    assert(entryCalled);
     if (mySettings.settingSet)
         v.mkNull();
     else
@@ -38,3 +41,8 @@ static RegisterPrimOp rp({
     .arity = 0,
     .fun = prim_anotherNull,
 });
+
+extern "C" void nix_plugin_entry()
+{
+    entryCalled = true;
+}