Improved support for NativeAOT Plugins

[LouChiSoft]

Right now if you want to setup up a plugin system in dotnet you have two options:

• Use JIT and have full access to the assembly loading features that dotnet has had for years. It's very robust, very featureful and you can do almost anything you want with it.
• Use native libraries with unmanaged functions and pointers. It's more rigid, brittle, prone to errors etc.

If you are making a standard runtime-dependant application it's clear which one you will want to use and there isn't really any reason to not use the JIT system. But if you are using a natively compiled application you have no option but to use unmanaged functions and pointers since without JIT the assembly loading features don't work. For me personally at least this puts me off wanting to write native libraries using C# as the second I do I lose the benefits of C# when it comes to actually consuming those libraries.

I know the main use case for AOT is applications and that you would normally use JIT libraries which are embedded in the application so that all the code is AOT'd together. But this isn't viable in all use cases. So I propose a first-party dotnet 'plugin' system and keyword that would work as follows:

Step 1: Define a plugin specification inside a shared library

public plugin MyPlugin {
    void InitializePlugin(PluginContext context);
    void TeriminatePlugin();

    void UpdatePlugin(PluginState state);
}

When this library is published with AOT it would produce 2 things; the native shared library and a .plugin called MyPlugin.plugin. MyPlugin.plugin contains a fixed layout and all the needed information for the types that make up the plugin's interface. In this case that's the MyPlugin definition and the definitions for PluginContext and PluginState. The purpose of the .plugin file is to be added as a reference to both your application that will load the plugins and the libraries that will implement it. Effectively replacing your standard assembly reference.

Step 2: Create an application that uses the plugin spec.

using MyPlugin;
using System.Plugin;

class MyApp
{
    static void Main(string[] args)
    {
        MyPlugin.PluginContext context = new MyPlugin.PluginContext();
        
        MyPlugin.MyPlugin plugin = Plugin.LoadPlugin<MyPlugin>("path to plugin");
        plugin.InitializePlugin(context);

        while (/*some exit condition*/) {
            plugin.UpdatePlugin(state);
        }

        plugin.TerminatePlugin;
        Plugin.UnloadPlugin(plugin);
    }
}

The example is pretty straight forward. As you can see the types used to load and interact with the plugin are only the types defined in the original plugin spec, which is why they would be fixed and stored in the .plugin file. This means that when compiling the application the compiler knows exactly they layout it needs to use to maintain compatibility. And by storing it in the .plugin file and not using a standardised ABI in the compiler like C++ you avoid having ABI breakage issues.

Step 3: The plugin creator can create their plugin

using MyPlugin;

class CustomPlugin: MyPlugin.MyPlugin {
    CustomPlugin() {
        // Initialise your class that implements the plugin.
        // This is called when the host calls `LoadPlugin`
    }

    // User's implementation of the Plugin interface
    void InitializePlugin(MyPlugin.PluginContext context){
        // Implementation
    }

    void TerminatePlugin() {
        // Implementation
    }

    void UpdatePlugin(MyPlugin.PluginState state){
        // Implementation
    }
}

I think a system like this or similar would go a long way to making natively compiled libraries more useful in C# for a wider range of use cases. If I'm honestly I would prefer a system that does the entire library and adds support for as many C# features as possible. But I think plugins would be a good first start

[hez2010]

If you are looking for an object-oriented interface for both the native and managed world to use in a plugin system, you will have to use COM as currently it's the only language-agnostic object-oriented native ABI that exists in the world.

[josephmoresena]

I recall that Java has something similar when generating headers for JNI libraries (from Java code). In this case, where the consumer is also C#, you should use something like a source generator at the time of native compilation, specifying the FFI that the C# code should consume (or some kind of interop assembly) when using your plugin. But that would only work for C#, so you’d still need to generate a .h file for C++, and so on for other languages... which is why I don’t see a built-in solution coming to the .NET SDK in the short term.

In fact, there’s no static way to link C# projects as native references. That could at least be a starting point.

On the other hand, keep in mind that even though it's CLR <-> Native AOT, what's in between is still the ABI—and in some cases, marshalling.

In this comment, I describe a scenario that clearly illustrates those same cases.

[LouChiSoft]

In this case, where the consumer is also C#, you should use something like a source generator at the time of native compilation

For me personally a C# only solution would be enough. I think a source code generator would be how you would do it. I did briefly look at trying to make a source code generator to do this. But having 0 experience with source code generators and also MSIL, which I assume you would need intimate knowledge of, I wouldn't even know where to begin

[josephmoresena]

Personally, I’ve used the official documentation along with support from AI tools — it’s really not that difficult.
I probably did some things incorrectly, since the way to create source generators has changed (that’s where AI support was key), but I think the results have been acceptable.
In fact, I recently had to use Mono.Cecil, with which I created a Task for MSBuild. This is useful for generating things post-compilation.

Did you want to create an .interop assembly or a .cs 'header' file? With this, I may can advise you on where to start.