Scalable MatMul-free Language Modeling

[iplayfast]

From this paper https://arxiv.org/abs/2406.02528 pdf -> https://arxiv.org/pdf/2406.02528
and this code
https://github.com/ridgerchu/matmulfreellm
Just found this and wondered if there is any opinion on integrating into llama.cpp

[iplayfast]

https://arstechnica.com/information-technology/2024/06/researchers-upend-ai-status-quo-by-eliminating-matrix-multiplication-in-llms/
It looks like it is scaleable. I'm wondering what if it would be possible to convert from current models to thistype of model. It seems that it is as good as regular models, only smaller and doesn't use matrix multiplication.
I think that means faster, can run on cpu easier, bigger models in same space, multiple models in same space etc etc.
Conversion might even be able to happen in realtime as the model is loaded. (just a thought).