Version v0.1.11rc2 Release Today!

[binmakeswell]

What's Changed

Autoparallel

• [autoparallel] fix bugs caused by negative dim key ([autoparallel] fix bugs caused by negative dim key #1808) by YuliangLiu0306
• [autoparallel] fix bias addition module ([autoparallel] fix bias addition module #1800) by YuliangLiu0306
• [autoparallel] add batch norm metainfo ([autoparallel] add batch norm metainfo #1815) by Boyuan Yao
• [autoparallel] add conv metainfo class for auto parallel ([autoparallel] add conv metainfo class for auto parallel #1796) by Boyuan Yao
• [autoparallel]add essential CommActions for broadcast oprands ([autoparallel]add essential CommActions for broadcast oprands #1793) by YuliangLiu0306
• [autoparallel] refactor and add rotorc. ([autoparallel] refactor and add rotorc. #1789) by Super Daniel
• [autoparallel] add getattr handler ([autoparallel] add getattr handler #1767) by YuliangLiu0306
• [autoparallel] added matmul handler ([autoparallel] added matmul handler #1763) by Frank Lee
• [autoparallel] fix conv handler numerical test ([autoparallel] fix conv handler numerical test #1771) by YuliangLiu0306
• [autoparallel] move ckpt solvers to autoparallel folder / refactor code ([autoparallel] move ckpt solvers to autoparallel folder / refactor code #1764) by Super Daniel
• [autoparallel] add numerical test for handlers ([autoparallel] add numerical test for handlers #1769) by YuliangLiu0306
• [autoparallel] update CommSpec to CommActions ([autoparallel] update CommSpec to CommActions #1768) by YuliangLiu0306
• [autoparallel] add numerical test for node strategies ([autoparallel] add numerical test for node strategies #1760) by YuliangLiu0306
• [autoparallel] refactor the runtime apply pass and add docstring to passes ([autoparallel] refactor the runtime apply pass and add docstring to passes #1757) by YuliangLiu0306
• [autoparallel] added binary elementwise node handler ([autoparallel] added binary elementwise node handler #1758) by Frank Lee
• [autoparallel] fix param hook issue in transform pass ([autoparallel] fix param hook issue in transform pass #1755) by YuliangLiu0306
• [autoparallel] added addbmm handler ([autoparallel] added addbmm handler #1751) by Frank Lee
• [autoparallel] shard param and buffer as expected ([autoparallel] shard param and buffer as expected #1753) by YuliangLiu0306
• [autoparallel] add sequential order to communication actions ([autoparallel] add sequential order to communication actions #1735) by YuliangLiu0306
• [autoparallel] recovered skipped test cases ([autoparallel] recovered skipped test cases #1748) by Frank Lee
• [autoparallel] fixed wrong sharding strategy in conv handler ([autoparallel] fixed wrong sharding strategy in conv handler #1747) by Frank Lee
• [autoparallel] fixed wrong generated strategy for dot op ([autoparallel] fixed wrong generated strategy for dot op #1746) by Frank Lee
• [autoparallel] handled illegal sharding strategy in shape consistency ([autoparallel] handled illegal sharding strategy in shape consistency #1744) by Frank Lee
• [autoparallel] handled illegal strategy in node handler ([autoparallel] handled illegal strategy in node handler #1743) by Frank Lee
• [autoparallel] handled illegal sharding strategy ([autoparallel] handled illegal sharding strategy #1728) by Frank Lee

Kernel

• [kernel] added jit warmup ([kernelfusion] Added JIT warmup #1792) by アマデウス
• [kernel] more flexible flashatt interface ([kernel] more flexible flashatt interface #1804) by oahzxl
• [kernel] skip tests of flash_attn and triton when they are not available ([kernel] skip tests of flash_attn and triton when they are not available #1798) by Jiarui Fang

Gemini

• [Gemini] make gemini usage simple ([Gemini] make gemini usage simple #1821) by Jiarui Fang

Checkpointio

• [CheckpointIO] a uniform checkpoint I/O module ([CheckpointIO] a uniform checkpoint I/O module #1689) by ver217

Doc

• [doc] polish diffusion README ([doc] polish diffusion README #1840) by binmakeswell
• [doc] remove obsolete API demo ([doc] remove obsolete API demo #1833) by binmakeswell
• [doc] add diffusion ([doc] add diffusion in README #1827) by binmakeswell
• [doc] add FastFold ([doc] add FastFold in README #1766) by binmakeswell

Example

• [example] remove useless readme in diffusion ([example] remove useless readme in diffusion #1831) by Jiarui Fang
• [example] add TP to GPT example ([example] add TP to GPT example #1828) by Jiarui Fang
• [example] add stable diffuser (add colossal diffuser #1825) by Fazzie-Maqianli
• [example] simplify the GPT2 huggingface example ([example] simplify the GPT2 huggingface example #1826) by Jiarui Fang
• [example] opt does not depend on Titans ([example] opt does not depend on Titans #1811) by Jiarui Fang
• [example] add GPT by Jiarui Fang
• [example] add opt model in lauguage ([example] add opt model in lauguage #1809) by Jiarui Fang
• [example] add diffusion to example ([example] add diffusion to example #1805) by Jiarui Fang

Nfc

• [NFC] update gitignore remove DS_Store ([NFC] update gitignore remove DS_Store #1830) by Jiarui Fang
• [NFC] polish type hint for shape consistency ([NFC] polish type hint for shape consistency #1801) by Jiarui Fang
• [NFC] polish tests/test_layers/test_3d/test_3d.py code style ([NFC] polish tests/test_layers/test_3d/test_3d.py code style #1740) by Ziheng Qin
• [NFC] polish tests/test_layers/test_3d/checks_3d/common.py code style ([NFC] polish tests/test_layers/test_3d/checks_3d/common.py code style #1733) by lucasliunju
• [NFC] polish colossalai/nn/metric/_utils.py code style ([NFC] polish colossalai/nn/metric/_utils.py code style #1727) by Sze-qq
• [NFC] polish tests/test_layers/test_3d/checks_3d/check_layer_3d.py code style ([NFC] polish tests/test_layers/test_3d/checks_3d/check_layer_3d.py co… #1731) by Xue Fuzhao
• [NFC] polish tests/test_layers/test_sequence/checks_seq/check_layer_seq.py code style ([NFC] polish tests/test_layers/test_sequence/checks_seq/check_layer_s… #1723) by xyupeng
• [NFC] polish accuracy_2d.py code style ([NFC] polish accuracy_2d.py code style #1719) by Ofey Chan
• [NFC] polish .github/workflows/scripts/build_colossalai_wheel.py code style ([NFC] polish .github/workflows/scripts/build_colossalai_wheel.py code #1721) by Arsmart1
• [NFC] polish _checkpoint_hook.py code style ([NFC] polish _checkpoint_hook.py code style #1722) by LuGY
• [NFC] polish test_2p5d/checks_2p5d/check_operation_2p5d.py code style ([NFC] polish test_2p5d/checks_2p5d/check_operation_2p5d.py code style #1718) by Kai Wang (Victor Kai)
• [NFC] polish colossalai/zero/sharded_param/init.py code style ([NFC] polish colossalai/zero/sharded_param/__init__.py code style #1717) by CsRic
• [NFC] polish colossalai/nn/lr_scheduler/linear.py code style ([NFC] polish colossalai/nn/lr_scheduler/linear.py code style #1716) by yuxuan-lou
• [NFC] polish tests/test_layers/test_2d/checks_2d/check_operation_2d.py code style ([NFC] polish check_operation_2d.py code style #1715) by binmakeswell
• [NFC] polish colossalai/nn/metric/accuracy_2p5d.py code style ([NFC] polish colossalai/nn/metric/accuracy_2p5d.py code style #1714) by shenggan

Fx

• [fx] add a symbolic_trace api. ([fx] add a symbolic_trace api. #1812) by Super Daniel
• [fx] skip diffusers unitest if it is not installed ([fx] skip diffusers unitest if it is not installed #1799) by Jiarui Fang
• [fx] Add linear metainfo class for auto parallel ([fx] Add linear metainfo class for auto parallel #1783) by Boyuan Yao
• [fx] support module with bias addition ([fx] support module with bias addition #1780) by YuliangLiu0306
• [fx] refactor memory utils and extend shard utils. ([fx] refactor memory utils and extend shard utils. #1754) by Super Daniel
• [fx] test tracer on diffuser modules. ([fx] test tracer on diffuser modules. #1750) by Super Daniel

Hotfix

• [hotfix] fix build error when torch version >= 1.13 (fix build error when torch version >= 1.13 #1803) by xcnick
• [hotfix] polish flash attention ([flashatt] polish code and fix test #1802) by oahzxl
• [hotfix] fix zero's incompatibility with checkpoint in torch-1.12 ([hotfix] fix zero's incompatibility with checkpoint in torch-1.12 #1786) by HELSON
• [hotfix] polish chunk import ([hotfix] polish chunk import #1787) by Jiarui Fang
• [hotfix] autoparallel unit test ([hotfix] autoparallel unit test #1752) by YuliangLiu0306

Pipeline

• [Pipeline]Adapt to Pipelinable OPT ([Pipeline]Adapt to Pipelinable OPT #1782) by Ziyue Jiang

Ci

• [CI] downgrade fbgemm. ([CI] downgrade requirements fbgemm. #1778) by Super Daniel

Compatibility

• [compatibility] ChunkMgr import error ([compatibility] ChunkMgr import error #1772) by Jiarui Fang

Feat

• [feat] add flash attention ([feat] add flash attention #1762) by oahzxl

Fx/profiler

• [fx/profiler] debug the fx.profiler / add an example test script for fx.profiler ([fx/profiler] debug the fx.profiler / add an example test script for fx.profiler #1730) by Super Daniel

Workflow

• [workflow] handled the git directory ownership error ([workflow] handled the git directory ownership error #1741) by Frank Lee

Full Changelog: v0.1.11rc2...v0.1.11rc1

---

This discussion was created from the release Version v0.1.11rc2 Release Today!.