How to convert complex struct into Span<byte>? And how about reverse?

[Flithor]

I'm looking for a faster way to serialize and deserialize stream and binary.
For simple struct, MemoryMarshal.Cast<T, byte> works just fine.

But for complex struct, I'm stuck.

Sample struct here:

[StructLayout(LayoutKind.Explicit, Pack = 1, Size = 1)]
struct ExplicitStruct
{
    [FieldOffset(0)]
    public byte num1_1;
    [FieldOffset(1)]
    public byte num1_2;
    [FieldOffset(2)]
    public byte num1_3;
    [FieldOffset(4)]
    public byte num1_4;
}

[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct ArrMemberStruct
{
    public ExplicitStruct num1;
    public int num2;
    public byte num3;
    public long num4;
    public decimal num5;
}
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
struct DataStruct
{
    public DataStruct(string name, ArrMemberStruct[] members)
    {
        header = 0xFF;
        this.name = name;
        var nameCharSpan = name.AsSpan();
        var nameByteSpan = MemoryMarshal.Cast<char, byte>(nameCharSpan);
        name_length = (byte)nameByteSpan.Length;
        this.members = members;
        var membersSpan = MemoryMarshal.Cast<ArrMemberStruct, byte>(members);
        data_length = (byte)members.Length;
        check_sum = (byte)(header + name_length + nameByteSpan.Sum() + data_length + membersSpan.Sum());
    }
    public byte header;
    public byte name_length;
    public string name;
    public byte data_length;
    public ArrMemberStruct[] members;
    public byte check_sum;
}
//The Sum extension methods:
public static int Sum(this Span<byte> bytes)
{
    int sum = 0;
    foreach (byte b in bytes)
        sum += b;
    return sum;
}
public static int Sum(this ReadOnlySpan<byte> bytes)
{
    int sum = 0;
    foreach (byte b in bytes)
        sum += b;
    return sum;
}

The sample data here:

var data = new DataStruct("Testing Object", new[] {
  new ArrMemberStruct {
    num1 = new ExplicitStruct {
      num1_1 = 1,
      num1_2 = 1,
      num1_3 = 2,
      num1_4 = 3
    },
    num2 = 134255,
    num3 = 33,
    num4 = long.MinValue,
    num5 = 4.99999999233m
  },
  new ArrMemberStruct {
    num1 = new ExplicitStruct {
      num1_1 = 1,
      num1_2 = 1,
      num1_3 = 3,
      num1_4 = 5
    },
    num2 = 63444,
    num3 = 99,
    num4 = long.MaxValue,
    num5 = 523635.99999999233m
  }
});

---

I have tried MemoryMarshal.Cast<T, byte> but not work, it throw an exception:

System.ArgumentException: Cannot use type 'MyNameSpace.DataStruct'. Only value types without pointers or references are supported.
  + System.ThrowHelper.ThrowInvalidTypeWithPointersNotSupported(System.Type)
  + System.Runtime.InteropServices.MemoryMarshal.Cast<TFrom, TTo>(ReadOnlySpan<TFrom>)

What should I do?
Does this mean I need to declare a Span<byte> with "exact length" and then write each member into it?
And convert Span<byte> to complex struct need to process each member slice manually?

[huoyaoyuan]

As the exception message says, Only value types without pointers or references are supported. Your DataStruct type contains a managed array, which is a reference.

[DaZombieKiller]

You cannot treat a managed structure (one containing refs, classes, arrays, etc) as a sequence of bytes and write it to a file, because managed types are not blittable. Arrays, strings, etc are not stored inline in the structure, they are GC-tracked pointers to data on the managed heap. They will have no meaning once they are written to disk. I would suggest using an existing binary serialization framework when you have a complex scenario like this.

[Clockwork-Muse]

As a side note - while you can pack and blit on single byte boundaries, there's usually a performance penalty for doing unaligned (that is, not aligned to the native word size) operations. Unless you're explicitly mimicking an existing protocol or file type, you may want to reorder fields.

[gfoidl]

Instead of using structs should these be classes, and for Type <-> Span having a serializer?
Yep, then it's not just one (unaligned) write, but multiple of them. Therefore you can handle the array, other non-simple types, etc.

PS: with the struct, and when the array length is known in advance, then you could use fixed sized buffers.