[New Feature] Seek MLA Supported by Smooth #86
Description
Will MLA that used in DeepSeek-V2 (https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat and https://arxiv.org/abs/2405.04434) be supported by activation smooth method?
{{ message }}
Will MLA that used in DeepSeek-V2 (https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat and https://arxiv.org/abs/2405.04434) be supported by activation smooth method?