Built FlashMLA for Windows and Nvidia sm_120 (workstation/50s) cards

[IISuperluminaLII]

I will PR this into this repo at some point, but I think I got a working source for windows MSVC with CUTLASS 4.1 and CUDA 12.9

For now I will just leave this here for crawlers and indexers, have not bench-marked (fully) anything. Will get results at some point.

FYI. at the end I only got sm_120(ONLY) to build so sm_100a will not have support as with sm_90 (I hate Nvidia sm) (Thanks for attending my Ted talk)

https://github.com/IISuperluminaLII/FlashMLA_Windows_sm120

[IISuperluminaLII]

I almost switched to Linux and was about to give up and just started trying a bit of everything here and there everywhere. I just wanted to play games and run some HP bayesian optimizations 😒

But after gighting with 100+ TMEM layout errors (DUE TO NVIDIA ONLY GIVING 99KB OF SRAM TO WORKSTATION CARDS)......... backwards was literal nightmare to implement (LITERALLY NOT A SINGLE TILE CONFIGURATION WORKED!!!!!!!)

I just threw a million darts on the board and hoped something would work, and from testing it seems like ehhh.... might have worked. Again I will need* to test this extensively against B100s and such

As you can see backwards still has some issues, working on it still.... forward seems to be acceptable although I would need an expert to comment on those parts. Backward is very close, should have it done osson! MIght ust be an issue with random vs realistic gradient values.... not sure

[IISuperluminaLII]

Currently only have Inference working, BWD is quite complicated so it should be done soon. MSVC sucks sooo much