Interesting neural network compression technology

[igor-yusupov]

Hi! I found an interesting service for running language models with 8 billion parameters on user devices. Maybe it will be useful for you and you will find some tricks for your engine: link and code

[robertknight]

Looks interesting. Thanks for the info!