LM-Infinite: Simple On-the-Fly Length Generalization for LLM

[Jipok]

https://www.reddit.com/r/LocalLLaMA/comments/165zpn9/r_lminfinite_simple_onthefly_length/
https://arxiv.org/pdf/2308.16137.pdf

Any chance this will be implemented in llama.cpp? As far as I understand, nothing in theory prevents this from being done.

[kyegomez]

Hey, I have a very basic implementation here
https://github.com/kyegomez/LM-Infinite

[Glaciohound]

Hello, we have a more detailed implementation here:

https://github.com/Glaciohound/LM-Infinite.

Probably you will be interested in checking out.