Google's new model: recurrent-Gemma (Griffin architecture) - faster inference and lower memory usage when inferencing over long contexts

[joseph777111]

@ggerganov Google just dropped a new model architecture: Griffin, which outperforms transformers. And, it offers efficiency advantages including faster inference and lower memory usage when inferencing over longer contexts. google/gemma.cpp has implemented a C++ version of the model (for use and reference).

Paper: Google’s Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

recurrentgemma-2b-it

[phymbert]

Already requested here:

• Support for RecurrentGemma (Gemma with Griffin Architecture) #6564