Plugin: Add PluginDynamicLibrary::queryInterface

Documentation/Libraries/Plugin.md

@@ -1,6 +1,6 @@
 @page library_plugin Plugin
 
-@brief 🟥 Minimal dependency based plugin system with hot-reload
+@brief 🟨 Minimal dependency based plugin system with hot-reload
 
 [TOC]
 
@@ -10,35 +10,36 @@ Plugin library allows extending application compiling C++ source to executable c
 - Compile and link cpp files to dynamic library
 - Unload / recompile / reload Plugin
 - Reload all dependant plugins of a given one
+- PluginDynamicLibrary::queryInterface allows creating contracts between Plugins or Plugins and Host
+- Support creating libc++ and libc free plugins that only use Sane C++ Libraries
+- Allows toolchain customization through SC::PluginSysroot and SC::PluginCompiler
 - Forcefully unlock and delete dll being debugged from Visual Studio Debugger
 
 # Status
-🟥 Draft  
-This entire system is still in early stages, and it's an experiment at best,so it's not recommended for general use.
+🟨 MVP  
+This library is expected to work correctly on `macOS`, `Windows`, and `Linux` using `MSVC`, `clang` and `GCC` compiler toolchains.
 
 # Description
 The main use case is for now splitting applications in smaller pieces that can be hot-reloaded at runtime.  
 A secondary use case could be allowing customization on a delivered application (mainly on Desktop systems).
-Plugins are always meant to be delivered in source code form (.cpp) and they're compiled on the fly.
-A plugin is made of a single .cpp file and it declares itself through a special comment in the source code.
+Plugins are always meant to be delivered in source code form (`.cpp`) and they're compiled on the fly.
+A plugin is made of a single `.cpp` file and it declares itself through a special comment in the source code.
 Such comment can declare the name, the version, a description / category and a list of dependencies.  
 
 A plugin can be modified, unloaded, re-compiled and re-loaded to provide additional functionality.
 
 The list of dependencies makes it possible to find recursive dependencies and unload them before unload a plugin.
 
-The library doesn't use a build system, but it compiles the .cpp files directly, linking it with symbols exported from
+The library doesn't use a build system, but it compiles the `.cpp` files directly, linking it with symbols exported from
 the loading executable (using `bundle_loader` on macOS and linking library exported from loading executable on windows).
-Plugin Dynamic Libraries are compiled with `nostdlib` and `nostdlib++` and they include a stub the allows defining some 
-symbols needed due to not linking the C++ CRT.  
+Plugin Dynamic Libraries are compiled with `nostdlib` and `nostdlib++` and they include a stub the allows defining some symbols needed due to not linking the C++ CRT.  
+Some special build flags however allow using `libc`, `libc++` or other sysroot / compiler supplied windows.  
 The idea is that plugins only use functionality provided by the calling executable or by other plugins.
 
-On Windows, some extra care has been taken to force-unlock the .pdb file from visual studio debugger, that happens
-if the dll is being loaded on a program being debugged.
+On Windows, some extra care has been taken to force-unlock the `.pdb` file from visual studio debugger, that happens if the dll is being loaded on a program being debugged.
 
-As of today this is all implemented using native dynamic library mechanisms and they're loaded
-in process, so doing the wrong thing with memory or forgetting to clean everything during shutdown can yield to instabilities
-and eventually crash the main executable.
+As of today this is all implemented using native dynamic library mechanisms that are being loaded directly in the process.  
+Doing the wrong thing with memory or forgetting to clean everything during shutdown can quickly crash the main executable.
 
 # Examples
 
@@ -53,15 +54,21 @@ Other ideas include redistribute a minimal C++ toolchain (probably a customized
 the plugins without needing a system compiler or a sysroot, as all public headers of libraries in this project do not need
 any system or compiler header.
 
-🟨 MVP
-- Integrate with [FileSystemWatcher](@ref library_file_system_watcher) to get hot-reload during development.
-
 🟩 Usable Features:
-- Evaluate integration with [Build](@ref library_build) once it will gain capability to build standalone (without XCode or Visual Studio)
-- Create minimal clang toolchain to compile scripts on non-developer machines
+- Specify directory for compiled intermediate and output files
+- Parallel / Async compile and link
+- Improve error handling and reporting
+- Further customization of some build flags and features:
+    - Custom libraries to link (declared in the plugin)
+    - Custom include paths (declared in the plugin)
 
 🟦 Complete Features:
-- To be decided
+- Create minimal clang toolchain to compile scripts on non-developer machines
+- Integrate with [Build](@ref library_build) library (once it will gain capability to build standalone without needing Xcode or Visual Studio)
+- Evaluate possibility to achieve some minimal error recovery
+- Easily integration with some RPC mechanism
 
 💡 Unplanned Features:
 - Compile plugins to WASM ?
+- Deploy closed-source (already compiled) binary plugins
+- Allow plugin to be compiled with different compiler from the one used in the Host

Libraries/Plugin/Plugin.cpp

@@ -537,6 +537,8 @@ SC::Result SC::PluginDynamicLibrary::unload()
 #endif
     pluginInit  = nullptr;
     pluginClose = nullptr;
+
+    pluginQueryInterface = nullptr;
     return Result(true);
 }
 
@@ -614,6 +616,8 @@ SC::Result SC::PluginDynamicLibrary::load(const PluginCompiler& compiler, const
     SC_TRY_MSG(dynamicLibrary.getSymbol(buffer.view(), pluginInit), "Missing #PluginName#Init");
     SC_TRY(StringBuilder(buffer).format("{}Close", definition.identity.identifier.view()));
     SC_TRY_MSG(dynamicLibrary.getSymbol(buffer.view(), pluginClose), "Missing #PluginName#Close");
+    SC_TRY(StringBuilder(buffer).format("{}QueryInterface", definition.identity.identifier.view()));
+    SC_COMPILER_UNUSED(dynamicLibrary.getSymbol(buffer.view(), pluginQueryInterface)); // QueryInterface is optional
     return Result(true);
 }
 

Libraries/Plugin/Plugin.h

@@ -154,11 +154,16 @@ struct SC::PluginCompiler
     struct Internal;
 };
 
+/// @brief Holds include and library paths for a system toolchain, used to let plugins link to libc and libc++
 struct SC::PluginSysroot
 {
     SmallVector<StringNative<256>, 8> includePaths; ///< Path to system include directories
     SmallVector<StringNative<256>, 8> libraryPaths; ///< Path to system library directories
 
+    /// @brief Finds a reasonable sysroot for the given compiler
+    /// @param compiler The PluginCompiler::Type to constrain the compatible PluginSysroot to look for
+    /// @param[out] sysroot The PluginSysroot with filled in include and library path
+    /// @return Valid Result if sysroot has been found
     [[nodiscard]] static Result findBestSysroot(PluginCompiler::Type compiler, PluginSysroot& sysroot);
 };
 
@@ -167,11 +172,27 @@ struct SC::PluginDynamicLibrary
 {
     PluginDefinition     definition;     ///< Definition of the loaded plugin
     SystemDynamicLibrary dynamicLibrary; ///< System handle of plugin's dynamic library
+
+    /// @brief Try to obtain a given interface as exported by a plugin through SC_PLUGIN_EXPORT_INTERFACES macro
+    /// @param[out] outInterface Pointer to the interface that will be returned by the plugin, if it exists
+    /// @return true if the plugin is loaded and the requested interface is implemented by the plugin itself
+    template <typename T>
+    [[nodiscard]] bool queryInterface(T*& outInterface) const
+    {
+        if (pluginQueryInterface and instance != nullptr)
+        {
+            return pluginQueryInterface(instance, T::InterfaceHash, reinterpret_cast<void**>(&outInterface));
+        }
+        return false;
+    }
+
   private:
     void* instance                      = nullptr;
     bool (*pluginInit)(void*& instance) = nullptr;
     bool (*pluginClose)(void* instance) = nullptr;
 
+    bool (*pluginQueryInterface)(void* instance, uint32_t hash, void** instanceInterface) = nullptr;
+
     friend struct PluginRegistry;
     [[nodiscard]] Result load(const PluginCompiler& compiler, const PluginSysroot& sysroot, StringView executablePath);
     [[nodiscard]] Result unload();

Libraries/Plugin/PluginMacros.h

@@ -3,6 +3,7 @@
 #pragma once
 #include "../Foundation/Memory.h"
 #include "../Foundation/PrimitiveTypes.h"
+#include "../Strings/StringHashFNV.h"
 
 //
 // SC_PLUGIN_EXPORT
@@ -92,9 +93,9 @@
 #endif
 
 //
-// SC_DEFINE_PLUGIN
+// SC_PLUGIN_DEFINE
 //
-#define SC_DEFINE_PLUGIN(PluginName)                                                                                   \
+#define SC_PLUGIN_DEFINE(PluginName)                                                                                   \
     SC_PLUGIN_LINKER_DEFINITIONS                                                                                       \
     extern "C" SC_PLUGIN_EXPORT bool PluginName##Init(PluginName*& instance)                                           \
     {                                                                                                                  \
@@ -108,3 +109,36 @@
         delete instance;                                                                                               \
         return res;                                                                                                    \
     }
+
+#define SC_PLUGIN_EXPORT_INTERFACES(PluginName, ...)                                                                   \
+    extern "C" SC_PLUGIN_EXPORT bool PluginName##QueryInterface(PluginName* plugin, SC::uint32_t hash,                 \
+                                                                void** pluginInterface)                                \
+    {                                                                                                                  \
+        return SC::PluginCastInterface<PluginName, __VA_ARGS__>()(plugin, hash, pluginInterface);                      \
+    }
+
+namespace SC
+{
+template <typename PluginClass, typename... InterfaceClasses>
+struct PluginCastInterface;
+
+template <typename PluginClass>
+struct PluginCastInterface<PluginClass>
+{
+    bool operator()(PluginClass*, uint32_t, void**) { return false; }
+};
+
+template <typename PluginClass, typename InterfaceClass, typename... InterfaceClasses>
+struct PluginCastInterface<PluginClass, InterfaceClass, InterfaceClasses...>
+{
+    bool operator()(PluginClass* plugin, uint32_t hash, void** pluginInterface)
+    {
+        if (hash == InterfaceClass::InterfaceHash)
+        {
+            *pluginInterface = static_cast<InterfaceClass*>(plugin);
+            return true;
+        }
+        return PluginCastInterface<PluginClass, InterfaceClasses...>()(plugin, hash, pluginInterface);
+    }
+};
+} // namespace SC

Libraries/Plugin/Tests/PluginTest.cpp

@@ -7,6 +7,7 @@
 #include "../../Strings/StringBuilder.h"
 #include "../../Testing/Testing.h"
 #include "../../Threading/Threading.h"
+#include "PluginTestDirectory/TestPluginChild/Interfaces.h"
 
 namespace SC
 {
@@ -85,12 +86,24 @@ struct SC::PluginTest : public SC::TestCase
             const PluginDynamicLibrary* pluginParent = registry.findPlugin(identifierParent);
             SC_TEST_EXPECT(pluginChild->dynamicLibrary.isValid());
             SC_TEST_EXPECT(pluginParent->dynamicLibrary.isValid());
+
+            // Query two interfaces from the child plugins and check their expected behaviour
+            ITestInterface1* interface1 = nullptr;
+            SC_TEST_EXPECT(pluginChild->queryInterface(interface1));
+            SC_TEST_EXPECT(interface1 != nullptr);
+            SC_TEST_EXPECT(interface1->multiplyInt(2) == 4);
+            ITestInterface2* interface2 = nullptr;
+            SC_TEST_EXPECT(pluginChild->queryInterface(interface2));
+            SC_TEST_EXPECT(interface2 != nullptr);
+            SC_TEST_EXPECT(interface2->divideFloat(4.0) == 2.0);
+
+            // Manually grab an exported function and check its return value
             using FunctionIsPluginOriginal = bool (*)();
             FunctionIsPluginOriginal isPluginOriginal;
             SC_TEST_EXPECT(pluginChild->dynamicLibrary.getSymbol("isPluginOriginal", isPluginOriginal));
             SC_TEST_EXPECT(isPluginOriginal());
 
-            // Modify child plugin
+            // Modify child plugin to change return value of the exported function
             String     sourceContent;
             FileSystem fs;
             SC_TEST_EXPECT(fs.read(pluginScriptPath.view(), sourceContent, StringEncoding::Ascii));
@@ -109,7 +122,7 @@ struct SC::PluginTest : public SC::TestCase
             SC_TEST_EXPECT(registry.loadPlugin(identifierChild, compiler, sysroot, report.executableFile,
                                                PluginRegistry::LoadMode::Reload));
 
-            // Check child plugin modified
+            // Check child return value of the exported function for the modified plugin
             SC_TEST_EXPECT(pluginChild->dynamicLibrary.isValid());
             SC_TEST_EXPECT(pluginChild->dynamicLibrary.getSymbol("isPluginOriginal", isPluginOriginal));
             SC_TEST_EXPECT(not isPluginOriginal());

Libraries/Plugin/Tests/PluginTestDirectory/TestPluginChild/Interfaces.h

@@ -0,0 +1,20 @@
+// Copyright (c) Stefano Cristiano
+// SPDX-License-Identifier: MIT
+#pragma once
+#include "Libraries/Foundation/Function.h"
+#include "Libraries/Strings/StringHashFNV.h"
+
+struct ITestInterface1
+{
+    static constexpr const char*  InterfaceName = "ITestInterface1";
+    static constexpr SC::uint32_t InterfaceHash = SC::StringHashFNV("ITestInterface1");
+
+    SC::Function<int(int)> multiplyInt;
+};
+struct ITestInterface2
+{
+    static constexpr const char*  InterfaceName = "ITestInterface2";
+    static constexpr SC::uint32_t InterfaceHash = SC::StringHashFNV("ITestInterface2");
+
+    SC::Function<float(float)> divideFloat;
+};

Libraries/Plugin/Tests/PluginTestDirectory/TestPluginChild/TestPluginChild.cpp

@@ -1,21 +1,33 @@
 // Copyright (c) Stefano Cristiano
 // SPDX-License-Identifier: MIT
+#include "Interfaces.h"
 #include <Libraries/Containers/SmallVector.h>
 #include <Libraries/Plugin/PluginMacros.h>
 #include <Libraries/Strings/Console.h>
 #include <Libraries/Strings/String.h>
 SC::StringView externallyDefinedFunc();
 
-struct TestPluginChild
+struct TestPluginChild : public ITestInterface1, public ITestInterface2
 {
     SC::SmallVector<char, 1024 * sizeof(SC::native_char_t)> consoleBuffer;
 
     SC::Console console;
 
-    TestPluginChild() : console(consoleBuffer) { console.printLine("TestPluginChild original Start"); }
+    TestPluginChild() : console(consoleBuffer)
+    {
+        // Setup Interfaces table
+        ITestInterface1::multiplyInt.bind<TestPluginChild, &TestPluginChild::multiply>(*this);
+        ITestInterface2::divideFloat.bind<TestPluginChild, &TestPluginChild::divide>(*this);
+
+        console.printLine("TestPluginChild original Start");
+    }
 
     ~TestPluginChild() { console.printLine("TestPluginChild original End"); }
 
+    int multiply(int value) { return value * 2; }
+
+    float divide(float value) { return value / 2; }
+
     [[nodiscard]] bool init()
     {
         using namespace SC;
@@ -40,4 +52,5 @@ extern "C" SC_PLUGIN_EXPORT bool isPluginOriginal() { return true; }
 // Dependencies:  TestPluginParent
 //
 // SC_END_PLUGIN
-SC_DEFINE_PLUGIN(TestPluginChild)
+SC_PLUGIN_DEFINE(TestPluginChild)
+SC_PLUGIN_EXPORT_INTERFACES(TestPluginChild, ITestInterface1, ITestInterface2)

Libraries/Plugin/Tests/PluginTestDirectory/TestPluginParent/TestPluginParent.cpp

@@ -36,4 +36,4 @@ struct TestPluginParent
 //
 // SC_END_PLUGIN
 
-SC_DEFINE_PLUGIN(TestPluginParent)
+SC_PLUGIN_DEFINE(TestPluginParent)

README.md

@@ -39,7 +39,7 @@ Library
 [Foundation](https://pagghiu.github.io/SaneCppLibraries/library_foundation.html)                        | 🟩 Primitive types, asserts, limits, Function, Span, Result, Tagged Union
 [Hashing](https://pagghiu.github.io/SaneCppLibraries/library_hashing.html)                              | 🟩 Compute `MD5`, `SHA1` or `SHA256` hashes for a stream of bytes
 [Http](https://pagghiu.github.io/SaneCppLibraries/library_http.html)                                    | 🟥 HTTP parser, client and server
-[Plugin](https://pagghiu.github.io/SaneCppLibraries/library_plugin.html)                                | 🟥 Minimal dependency based plugin system with hot-reload
+[Plugin](https://pagghiu.github.io/SaneCppLibraries/library_plugin.html)                                | 🟨 Minimal dependency based plugin system with hot-reload
 [Process](https://pagghiu.github.io/SaneCppLibraries/library_process.html)                              | 🟩 Create child processes and chain them (also usable with [Async](https://pagghiu.github.io/SaneCppLibraries/library_async.html) library)
 [Reflection](https://pagghiu.github.io/SaneCppLibraries/library_reflection.html)                        | 🟩 Describe C++ types at compile time for serialization
 [Serialization Binary](https://pagghiu.github.io/SaneCppLibraries/library_serialization_binary.html)    | 🟨 Serialize to and from a binary format using [Reflection](https://pagghiu.github.io/SaneCppLibraries/library_reflection.html)