v0.3.1
What's Changed
- [Misc] Change grid to support long ctx by @zhiyuan1i in #528
- [RWKV7] Reduce CPU overhead by @zhiyuan1i in #529
- [Tokenshift] Support SP and cache by @zhiyuan1i in #531
- [RWKV7] Use tokenshift to save cache by @zhiyuan1i in #532
- [RWKV7] Fix the issue of RWKV7 initialization with BFloat16 data type on CPU. by @zhiyuan1i in #538
- [CI] Add compatibility check by @zhiyuan1i in #536
- [ShortConv] Support cache in prefill by @zhiyuan1i in #535
- [WIP] Add Log-Linear Attention by @2022tgoel in #524
- [Cache] Upgrade to transformer>= v4.48[skip test] by @zhiyuan1i in #541
- [Misc.] Set env var TRITON_F32_DEFAULT to
ieeewhen tf32 is not supported on NVIDIA by @KevlarKanou in #544 - [CI] Fix mirror for building triton by @zhiyuan1i in #543
- Log-Linear Attention Tests by @2022tgoel in #542
- [CI] Add proxy config for git by @zhiyuan1i in #548
- [Conv] Fix warning issue by @zhiyuan1i in #549
- [Misc.] Eliminate recompilation in layer-norm kernels caused by dynam… by @zhiyuan1i in #545
- [Misc.] Add activations for non-cuda Backends by @zhiyuan1i in #174
- [TMA] Accelerate solve_tril with TMA descriptors[skip test] by @zhiyuan1i in #550
- [CI] Upgrade to latest casual-conv1d and fix triton build for 3.4.x by @zhiyuan1i in #551
- [CI] Fix support for Intel GPU by @zhiyuan1i in #554
- [Fix] Fix Triton Error for HeadDim < 16[skip test] by @zhiyuan1i in #556
- [GLA] Fix simple_gla Test by @zhiyuan1i in #558
- [CI] Fix CI script errors[skip test] by @zhiyuan1i in #566
- require transformers <= 4.53.3 by @richardodliu in #570
- [Deps] Adopt transformers>4.53.3 by @zhiyuan1i in #571
- [Misc.] Clean codes and make mypy happy by @zhiyuan1i in #572
- [Models]: Add MoM by @WKX933 in #442
- [MoM]Fix lint by @JusenD in #573
- [Refactor] Apply GradientCheckpointingLayer to all model layers by @yzhangcs in #575
- [Mamba] Fix errors in Triton backend by @zhiyuan1i in #576
New Contributors
- @2022tgoel made their first contribution in #524
- @KevlarKanou made their first contribution in #544
- @richardodliu made their first contribution in #570
- @WKX933 made their first contribution in #442
Full Changelog: v0.3.0...v0.3.1