AVX512 Instructions Help

[waylaa]

I am trying to re-write a C++ project that heavily uses AVX512 because of the Keccak algorithm in C#, and these are the instructions that it uses:

_mm512_maskz_loadu_epi64
_mm512_ternarylogic_epi64
_mm512_permutexvar_epi64
_mm512_permutex2var_epi64
_mm512_rol_epi64
_mm512_rolv_epi64
_mm512_xor_si512
_mm512_unpacklo_epi64
_mm512_unpackhi_epi6
_mm512_mask_blend_epi64
_mm512_mask_blend_epi64
_mm512_mask_storeu_epi64
_mm512_maskz_set1_epi64
_mm512_setr_epi64

I cannot find most of them. Are some of them missing? I would appreciate any help.

[tannergooding]

C Intrinsic	.NET API	Notes
_mm512_maskz_loadu_epi64	N/A	Not exposed yet, was missed due to requiring more complex mask support
_mm512_ternarylogic_epi64	Avx512F.TernaryLogic	.NET 9 has support for recognizing many x op y op z patterns and emitting ternary logic implicitly
_mm512_permutexvar_epi64	Avx512F.PermuteVar8x64
_mm512_permutex2var_epi64	Avx512F.PermuteVar8x64x2
_mm512_rol_epi64	Avx512F.RotateLeft
_mm512_rolv_epi64	Avx512F.RotateLeftVariable
_mm512_xor_si512	Avx512F.Xor	Just use x ^ y instead
_mm512_unpacklo_epi64	Avx512F.UnpackLow
_mm512_unpackhi_epi6	Avx512F.UnpackHigh
_mm512_mask_blend_epi64	Avx512F.BlendVariable	Alternatively just use Vector512.ConditionalSelect
_mm512_mask_storeu_epi64	N/A	Not exposed yet, was missed due to requiring more complex mask support
_mm512_maskz_set1_epi64	N/A	Long term this will be supported via ConditionalSelect(mask, Vector512.Create(value), Vector512.Zero)
_mm512_setr_epi64	Vector512.Create(...)	We don't expose set, only setr as it makes sense to pass parameters from element 0 to element count

In general masking support is not done in the same way as it's done in C/C++. Instead, we internally track whether something is a mask or not and continue exposing everything as a Vector512<T>. This helps immensely when writing cross platform code or code that needs to support multiple ISAs (SSE2, SSE4.1, AVX2, AVX512, AVX10, etc) as the code and general masking concepts are consistent. We then pattern match usages like ConditionalSelect(mask, x + y, merge) to emit a masked x + y instruction on hardware that supports it and otherwise to emit the natural fallback (which would be to use something like the add; blendv sequence instead).

For places that have different behavior when a mask is used and therefore cannot be pattern matched (like maskz_loadu) we will eventually expose explicit APIs that take in the mask as a Vector512<T> (or appropriately sized vector). If that input is actually a mask, then you get the optimal codegen as expected. We didn't land the full support in .NET 8 simply due to the sheer size/scope of AVX512 support; it is something we have partially landed in .NET 9 so far and are continuing to work on. -- It's also worth noting that in general masked instructions are more expensive than unmasked instructions and so you still want to be selective with how such code is used. You still in most scenarios want to restrict masking primarily to outside our core loop only when handling leading or trailing elements if at all possible (which it should be for most algorithms).

[waylaa]

Thank you for your detailed reply. I am wondering if there is any alternative to the N/A table items (_mm512_maskz_set1_epi64, _mm512_mask_storeu_epi64, _mm512_maskz_loadu_epi64). Like for example resorting to plain simple math operations or AVX2, etc.

Because right now, according to the table above, I cannot implement these (and some more) in C#:

const static __m512i K12RoundConst0 = _mm512_maskz_set1_epi64(1, 0x000000008000808bULL);
const static __m512i K12RoundConst1 = _mm512_maskz_set1_epi64(1, 0x800000000000008bULL);
const static __m512i K12RoundConst2 = _mm512_maskz_set1_epi64(1, 0x8000000000008089ULL);
const static __m512i K12RoundConst3 = _mm512_maskz_set1_epi64(1, 0x8000000000008003ULL);
const static __m512i K12RoundConst4 = _mm512_maskz_set1_epi64(1, 0x8000000000008002ULL);
const static __m512i K12RoundConst5 = _mm512_maskz_set1_epi64(1, 0x8000000000000080ULL);
const static __m512i K12RoundConst6 = _mm512_maskz_set1_epi64(1, 0x000000000000800aULL);
const static __m512i K12RoundConst7 = _mm512_maskz_set1_epi64(1, 0x800000008000000aULL);
const static __m512i K12RoundConst8 = _mm512_maskz_set1_epi64(1, 0x8000000080008081ULL);
const static __m512i K12RoundConst9 = _mm512_maskz_set1_epi64(1, 0x8000000000008080ULL);
const static __m512i K12RoundConst10 = _mm512_maskz_set1_epi64(1, 0x0000000080000001ULL);
const static __m512i K12RoundConst11 = _mm512_maskz_set1_epi64(1, 0x8000000080008008ULL);

Also, I believe it would be immensely helpful to provide a table in the .NET AVX documentation like the one you created covering all supported intrinsics.

[neon-sunset]

For quick lookup by intrinsic name, you can use https://marketplace.visualstudio.com/items?itemName=EgorBogatov.Disasmo:

If possible, it may be easier to write cross-platform SIMD implementation first and then accelerate parts of the algo with platform-specific instructions instead.

[waylaa]

Useful extension indeed. However, '_mm512_maskz_set1_epi64' is not yet implemented in C#. I made this for now (although i haven't tested it if it works at intended):

    public static unsafe Vector512<ulong> Set1WithMask(ulong mask, ulong value)
    {
        Vector512<ulong> result = Vector512<ulong>.Zero;

        ulong* resultPtr = (ulong*)&result;

        for (int i = 0; i < 8; i++)
        {
            if ((mask & (1UL << i)) != 0)
            {
                resultPtr[i] = value;
            }
        }

        return result;
    }

[En3Tho]

@waylaa In case of const static I think you can just resort to something like public static Vector512<ulong> K12RoundConst0 => Vector512.Create(...). Afaik it should be properly inlined and optimized.

If you need to do this dynamically then maybe AVX2 has something that can do similar thing on 2x256?

[tannergooding]

const static __m512i K12RoundConst0 = _mm512_maskz_set1_epi64(1, 0x000000008000808bULL);

This looks like it's just Vector512.CreateScalar(0x000000008000808BUL), given that maskz_set1 will set any elements to that value if the bit is set and clear it to zero otherwise, thus only setting element 0 to it and leaving all elements zeroed.

It's notably much better to just use such values inline, rather than placing them into a static readonly, as the JIT will recognize them as constant.

I am wondering if there is any alternative to the N/A table items

In general all maskz_name(mask, ...) is doing is name(...) & mask. If the mask bit is true, then the result of name(...) is kept and otherwise the element is cleared to zero. So assuming mask is the result of something like CompareEqual and is therefore AllBitsSet or Zero per-element, it works exactly as expected. If you are expanding from a bitmask, then bit that is 1 becomes AllBitsSet and other bits stay Zero for the respective elements.

All mask_name(src, mask, ...) operations are then just doing ConditionalSelect(mask, ..., src) because name(...) is kept if the mask bit is true and otherwise the element from src is taken.

There are many cases where doing a masked operation is "worse" than just doing a more natural thing. For example choosing CreateScalar(...) is preferred over maskz_set1(1, ...). Similarly there are many cases where masking can be avoided altogether by simply rethinking how you work with the data.

mask_storeu and maskz_load can be emulated in most circumstances using the above logic and when the direct JIT support is added for that pattern. However, there is a little-bit of nuance if the load/store would be potentially out of bounds (cause an AccessViolation) as the masked instruction will suppress faults while the manual pair of operations would not. This is why I mentioned that we need to expose explicit APIs for these.

Also, I believe it would be immensely helpful to provide a table in the .NET AVX documentation like the one you created covering all supported intrinsics.

Providing a table is fairly complex as not all things map 1-to-1. It can be done, but there isn't a super high priority around it and the doc page would be massive without the ability to easily search, etc.

Even for C/C++, most compilers have stopped providing docs and instead just defer to https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html. That, unfortunately, doesn't currently have any .NET information but it would ultimately be up to Intel to provide such info.

We do, however, currently provide some minimal information on each individual API page around what instruction and C/C++ intrinsic it maps to and most names should be a fairly intuitive mapping as both C intrinsics follow a fairly standard pattern and the .NET intrinsics follow a fairly similar pattern as well.

For example. C intrinsics are in the form of {size}{prefix}{name}{dataType}(...) where:

• {size} is _mm for 128-bit, _mm256 for 256-bit, and _mm512 for 512-bit.
• {prefix} is typically empty but can sometimes be _mask meaning you merge the result with src or _maskz meaning you merge the result with zero
• {name} is then the actual operation being done like _add or _storeu (meaning store unaligned)
• {dataType} then indicates the type of data you operate on, some common type are
  • epi8 (packed signed 8-bit integers)
  • epi16 (packed signed 16-bit integers)
  • epi32 (packed signed 32-bit integers)
  • epi64 (packed signed 64-bit integers)
  • epu8 (packed unsigned 8-bit integers)
  • epu16 (packed unsigned 16-bit integers)
  • epu32 (packed unsigned 32-bit integers)
  • epu64 (packed unsigned 64-bit integers)
  • pd (packed 64-bit float; aka packed double)
  • ph (packed 16-bit float; aka packed half)
  • ps (packed 32-bit float; aka packed single)
  • sd (scalar 64-bit float; aka scalar double)
  • sh (scalar 16-bit float; aka scalar half)
  • ss (scalar 32-bit float; aka scalar single)

C does it this way because C doesn't have overload resolution and doesn't allow APIs to be instances of types. So everything is global, it is up to the user to know if it is safe to call an API or not based on the target machine they're compiling for (have to manually do CPUID checks), and they need to disambiguate the type of data being operated on by name. This is also why they have at least 3 types per vector size __m128, __m128d, and __m128i.

For .NET, we typically follow the form of {Isa}.{Name}(...) where:

• {Isa} is the name of the instruction set the API was exposed in, like Sse or Avx
• {Name} is the name of the actual operation being done like Add

In .NET we do it this way because we do have overload resolution, so we can categorize the APIs based on when they were exposed. We can expose {Isa}.IsSupported properties that do the CPUID checks for the user (and implicitly treat those as constant based on the target machine), we can then just use a single name like Add for any data type being operated on because the data type is clearly indicated by it taking Vector128<float> or Vector128<byte> or Vector128<...>. We can make it so that Sse2.{Name} exposes everything from Sse2 and everything from Sse (since SSE2 implies SSE support).

In .NET we then take it a step further and give you access to operators like x + y so that you don't have to remember whether Sse.Add for Vector128<float>, but Sse2.Add for other types (like double or int) and that its Avx.Add for Vector256<float> and Vector256<double> but Avx2.Add for the integer types and so on. This also ends up allowing you to write one code path that works for other architectures like Arm64 or WASM which have completely different naming schemes, ISAs, etc. We expose a great deal of cross platform APIs that should be the preferred mechanism to write code in most scenarios.