Does vLLM support flash attention?

[zhaoyang-star]

Flash attention is an important optimizing method but I found no flash attention impls in vLLM code base. So does vLLM support flash attention?

[zhyncs]

Flash attention is an important optimizing method but I found no flash attention impls in vLLM code base. So does vLLM support flash attention?

vLLM use xformers's memory_efficient_attention_forward, so it makes indirect use of flash attention.