Inconsistent struct layouts can break cross-architecture usage

[antoine-sac]

When trying to run distributed-llama across multiple architectures (e.g., x86_64 as the server and 32-bit ARMv7 as the workers), I’m seeing corrupted data fields in the transmitted structs.

This usually leads to huge, incorrect string lengths (sometimes gigabytes) and a subsequent std::bad_alloc.

Everything runs fine if both ends share the same architecture (e.g., all x86_64 or all ARMv8), but fails specifically with 32-bit ARM.

It looks like the issue comes from sending entire structs over the wire via:

write(socket, &myStruct, sizeof(myStruct));

and then doing a corresponding read(...) on the other side. Different ABIs (especially 32-bit vs. 64-bit, but not only) can have different alignment rules for struct fields. This causes the fields to be written and read at mismatched offsets. The same approach is harmless if both ends are compiled for the exact same architecture, but it is fragile.

It breaks for me if I mix 32-bit ARM with 64-bit x86 or ARMv8, but it could also happen anytime struct layouts differ across compilers or OSes – for instance, between Windows x64 (MSVC) and Linux x64 (GCC), or even different compiler flags on the same architecture.

Proposed Fix

A more robust solution is to serialize fields explicitly, rather than writing the raw struct. For example:

uint32_t ft = (uint32_t)myStruct.floatType;
network->write(socket, &ft, sizeof(ft));
network->write(socket, &myStruct.x, sizeof(myStruct.x));
network->write(socket, &myStruct.y, sizeof(myStruct.y));

And on the other side:

uint32_t ft, xVal, yVal;
network->read(socket, &ft, sizeof(ft));
network->read(socket, &xVal, sizeof(xVal));
network->read(socket, &yVal, sizeof(yVal));
myStruct.floatType = (NnFloatType)ft;
myStruct.x = xVal;
myStruct.y = yVal;

This ensures consistent layouts, independent of how the compiler packs fields in a struct. Alternatively, a standard serialization library like protobuf would handle alignment, endianness, and versioning issues automatically.