Releases: lucidrains/memory-efficient-attention-pytorch
Releases · lucidrains/memory-efficient-attention-pytorch
0.0.23
add tests, fix bugs
0.0.22
oops, actually add cosine sim flash attention
0.0.21
oops, actually add cosine sim flash attention
0.0.20
add cosine sim attention variant of flash attention
0.0.19
fix bugs
0.0.18
complete flash attention algorithm in plain pytorch (for educational …
0.0.17
directly calculate triu shift value for causal mask for memory savings
0.0.16
materialize causal mask only when needed, to reduce peak memory usage…
0.0.15
chunk feedforward for entirely memory efficient transformer
0.0.14
max for numerical stability should be taken after masking