Cross-platform C# / C++ interop

[llfab]

Goal of this show & tell

These insights are not specific to Avalonia. However, Avalonia supports cross-platform UI implementation. Not in every project you have the luxury to only handle .NET dependencies. Some projects require specific implementations in unmanaged code such as C++ and p/invoke those functions. I will describe our experiences and conclusions.

General technique

In the early phases of .NET the option to p/invoke system DLLs or own C++ libraries was DllImport. Its cross-platform correspondance is LibraryImport that allows for fewer automations and needs more manial control especially for non-primitive parameters in the interop functions like strucs.

Cross-platform LibraryImport

C-API

Use exported functions with either primitive parameters or pointers to structs

EXPORT int DoSomething(const MyInputStruct* input, MyOutputStruct* result);

Of course, you will have your C++ objects and their functions. Hence, the implementation of the C-API function must then call into your C++ objects

EXPORT int DoSomething(const MyInputStruct* input, MyOutputStruct* result)
{
    MyObject obj();
    result = obj.DoSomething(input);
    return 0;
}

Structs

Define the structs in the C++ layer.

typedef struct MyInputStruct {
    double X;
    double Y;
    double Z;
} MyInputStruct ;

Function and Type Interop

Define the native function using LibaryImport in the .NET code. And define the structs that match the definition in the C++ code.

[LibraryImport(MyLibLibraryName)]
public static partial int DoSomething(in MyInputStruct input, out MyOutputStruct output);

[StructLayout(LayoutKind.Sequential, Pack = 4)]
internal readonly struct MyInputStruct 
{
    private readonly double _x;
    private readonly double _y;

    public MyInputStruct () { }
}

Note: in order to match the memory layout of the C++ struct, you need to use StructLayout with LayoutKind.Sequential and specify the packing size.
The packing size is an optional parameter that you may not need if you target only one single platform. Typically, on Windows the packing
between C# and the C++ compiler is 8 bytes and matches implicitly. However, on Linux the packing may be different and especially
seems not to automatically match. Therefore, it is safer to specify the packing size explicitly. In our case, specifying the packing
was actually mandotory to make the code work on both Windows and Linux.

Also note, the inand out keywords are necessary to specify call-by-reference semantics for the parameters. A missing inleads to a
call-by-value interpretation. Again here, this can work on Windows but would be interpreted wrong on Linux. Therefore, the struct memory
would be marshalled incorrectly and the C++ code would not be able to read the data correctly.

Setting packing size in C++

In the above definition of the struct in C++, the packing size is not specified. Again, this may work nicely on Windows because MSVC comiler
and C# would use the same packing size. For cross-platform code, however, it is mandatory to define the packing size explicitly. As for some
platforms and compilers 8 bytes would not work, we found 4 bytes to be a safe choice.

#pragma pack(push, 4) // Ensure 4-byte alignment for C interface compatibility

// Define your structs here
// ...
// ...

#pragma pack(pop) // Restore previous packing alignment

Select the right native library to load

Use a DllImportResolver to guide the loading of the native library like in below simplified example.
This allows you to load the correct library depending on the platform and debug/release mode.

internal static partial class NativeMethods
{
    private const string MyLibLibraryName = "MyLib.dll";
    private const string MyLibWindowsLibraryName = "MyLib.dll";
    private const string MyLibWindowsLibraryDebugName = "MyLibd.dll";
    private const string MyLibLinuxLibraryName = "libMyLib.so";
    private const string MyLibLinuxLibraryDebugName = "libMyLibd.so";

    static NativeMethods()
    {
        NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, DllImportResolver);
    }

    private static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
    {
        if (libraryName.Equals(MyLibLibraryName, StringComparison.InvariantCultureIgnoreCase))
        {
            if (OperatingSystem.IsWindows())
            {
#if DEBUG
                return NativeLibrary.Load(MyLibWindowsLibraryDebugName, assembly, searchPath);
#else
                return NativeLibrary.Load(MyLibWindowsLibraryName, assembly, searchPath);
#endif
            }
            else if (OperatingSystem.IsLinux())
            {
#if DEBUG
                return NativeLibrary.Load(MyLibLinuxLibraryDebugName, assembly, searchPath);
#else
                return NativeLibrary.Load(MyLibLinuxLibraryName, assembly, searchPath);
#endif
            }
            else
            {
                Assert.Never("MyLibApi: Platform[{0}] not supported.", Environment.OSVersion.Platform);
            }
        }

        // Otherwise, fallback to default import resolver.
        return IntPtr.Zero;
    }

    // ...
}

C++ Build System

Obviously, .vcxproj cannot be used in a cross-platform manner. We found that CMake is a good choice to build the C++ library.
It is likely that you need to create platform-specific instructions to build the library correctly. That seems relatively easy
using CMake.

In our case, we added dummy projects to the C# solution that would run the CMake build commands as
pre-build steps. This way, the C++ library is built automatically when the C# project is built.

[maxkatz6]

I think with this approach you have C# <-> C interop, not C++. As you have to use extern C methods.
COM-like interop would be another option allowing to keep "classes" concept.

[llfab]

I forgot to add the link of the C-API to the C++. Did so now.

Indeed, C-API is one of the options to call into your C++ code which works nicely if you have a relatively slim interface that you can make a flat C-API thing. If you have a e.g. math library with hundreds of objects and functions, this approach is not suitable.

[llfab]

We have used this approach for an image processing lib in C++ that has decent size. However, the interface is relatively slim with some 20 functions or so. It helped us for both:

• be able to carry legacy code forward
• some image processing functions can be implemented more efficiently in C++ e.g. using OpenMP or in some cases ceres-solver for which there is not corresponding C# solution

We would potentially have tried a pure C# impl if it was on a blank piece of paper. But since all this algos have already been there and were perf optimized that was the shorter route.

We had these working on Windows but when extending to Liniux we found above details to be relevant (and hard to find in the web).

[llfab]

Note for above simplified examples:

• Your struct handling and marshalling becomes more complex when you have elements of dynamic size. E.g. we have some image data elements in structs that need manual marshalling code for the data handling but it still works over interop
• Another special problem is, if dynamic data is being created in the unmanaged layer (in the C++ code) and returned in your output data of the C-API functions. We found that a pattern that works is to use a FreeMemory function. We are using that heavily for returned image data of variable length.

// C# interop
[LibraryImport(FluoroLibLibraryName)]
public static partial void FreeMemory(IntPtr dataPtr);

// C++
EXPORT void FreeMemory(void* data_ptr)
{
    free(data_ptr);
}

[thevortexcloud]

In the early phases of .NET the option to p/invoke system DLLs or own C++ libraries was DllImport. Its cross-platform correspondance is LibraryImport that allows for fewer automations and needs more manial control especially for non-primitive parameters in the interop functions like strucs.

This is not quite accurate. The major difference is a source generator that can generate things ahead of time instead of entirely at runtime. If you look at the code the source generator generates, in many cases it will generate fairly similar code to what you would get if you manually did a DllImport. Both work cross platform.

https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation

Note: in order to match the memory layout of the C++ struct, you need to use StructLayout with LayoutKind.Sequential and specify the packing size.

You can also use an explicit layout, it's just more tedious. It does however save you from your code breaking for no obvious reason if you decide to run some formatting tool on your C# code.

Another special problem is, if dynamic data is being created in the unmanaged layer (in the C++ code) and returned in your output data of the C-API functions.

The windows API approach to solve this to call a method twice. The first time returns the size, and then you allocate some buffer for the second call. For linked lists it just returns a pointer to the next value, so you just keep reading until you run out of valid pointers. But there is probably better patterns, I am not a C expert.

[llfab]

DllImport vs LibraryImport
Maybe is was not 100% precise. I know that this finally does a DllImport. Still LibraryImport does "less", like not being open for either calling funcA or funcW but needing a decision on compile time. My experience is that both work nicely on Windows but LibraryImport ist needed or at least the better choice for full control and therefore for x-plat.

Struct Alignment
Yes, you can do explicit but I find it even more cumbersome.

The Win-API approach is feasible especially for fine granular functions (GetX()). We had thought about that too. However, for very high level functions that compose a lot of your C++ functions below it seemed not a great fit. That is why we decided for burdening the caller with the "free"-task. Hence, the returning struct contains dynamic data. The caller must be aware of that and after having consumed the data (typically this is a copy in to a managed element) the caller must free the allocated resources. The best-fit approach may be quite application specifc.

[llfab]

It is all valuable input, I found it difficult at the time to collect all these specifics. Especially, the accidently missing "in" in one of the functions that would not cause a problem on Windows but would add a lot of alignment and padding bytes on Linux.