NVIDIA BioNeMo Framework v2.6

New Features

• Adds support for AMPLIFY doi:10.1101/2024.09.23.614603 pre-training and inference, offering a 70% speedup over the xformers-based attention backend with similar final perplexity values at 1M pre-training steps. (4.23 for 120M, 3.05 for 350M). The model is fully compatible with existing weights on HuggingFace.
• Adds alpha support for LoRA fine-tuning to for ESM2 models. Inference and fine-tuning are enabled along with resumption from a checkpoint.

Updates & Improvements

• Blackwell support, tested on B200 systems.
• Fixed Grace CPU support, released ARM compatible container.

What's Changed

• hotfix: docker build in CI by @dorotat-nv in #756
• updated version for 2.5 release by @dorotat-nv in #755
• update evo2 partial conv max steps by @dorotat-nv in #736
• bump ruff to 0.9.10 and reformat files as necessary by @pstjohn in #751
• Updated file paths for images in SCDL README by @polinabinder1 in #758
• hotfix CI: failing test test_train_evo2_stops by @dorotat-nv in #761
• adding v2.5 release notes by @dorotat-nv in #764
• switch to GHA runners by @pstjohn in #734
• Jwilber/update evo2 readme and assets by @jwilber in #759
• Aligning directories with tensorboard logs for ESM2 and Evo2 by @dorotat-nv in #740
• Remove Evo2 PR announcement now that everything is merged by @jstjohn in #772
• switch cache-from flags by @pstjohn in #773
• Add AMPLIFY model and huggingface conversion scripts by @pstjohn in #640
• parallelize test stages in GitHub CI by @dorotat-nv in #768
• D3pm blackwell testing stability fix by @nvdreidenbach in #743
• fix: attempts to update geneformer notebooks by @jomitchellnv in #745
• [hotfix] setting as "ignore" the failing notebook geneformer_cellxgene_tutorial.ipynb by @dorotat-nv in #779
• Remove xformers install by @pstjohn in #781
• Remove outdated evo2 tutorial that is now in the submodule by @jstjohn in #783
• Blackwell compatibility changes by @trvachov in #707
• Add AMPLIFY inference by @pstjohn in #775
• Pin griffe to 1.6.2 by @pstjohn in #789
• hotfix: evo2 divergence - downgrade TE to v1.13 by @dorotat-nv in #791
• Updates tflops chart for Geneformer. by @jomitchellnv in #785
• change checkpoint name pattern by @farhadrgh in #786
• Revert commit 67a869b (TE_VERSION=v1.13 fix) by @dorotat-nv in #795
• [cye/subpack-gpu-testing] Add GPU runner to testing job. by @cspades in #776
• Dockerfile improvements for ARM by @trvachov in #777
• Remove llama-index from container to fix CVEs by @trvachov in #800
• Bump 3rdparty/NeMo from cc8ff45 to 384ff02 by @dependabot in #792
• Add local clone script by @nvdreidenbach in #787
• Fix ARM docker build by @trvachov in #801
• [cye/ml-subpackage-ci] Onboard bionemo-llm and bionemo-noodles to the sub-package CI. by @cspades in #809
• Update README.md link by @nvdreidenbach in #812
• Have dependabot update our docker base image by @pstjohn in #813
• Add .codecov.yml status checks by @pstjohn in #618
• Add AMPLIFY model documentation, minor type fixes by @pstjohn in #788
• Remove import guard in bionemo-llm by @pstjohn in #804
• Bump rust from 1.82.0 to 1.86.0 by @dependabot in #819
• Bump crossbeam-channel from 0.5.13 to 0.5.15 in /sub-packages/bionemo-noodles by @dependabot in #818
• Pbinder/geneformer partial conv by @polinabinder1 in #802
• [cye/rapids-sc-install] Add rapids_singlecell import to BioNeMo FW container image. by @cspades in #816
• Biopharma mailing list docs addition. by @trvachov in #822
• unify the implementation of early training termination across BioNeMo subpackages and update benchmarks by @dorotat-nv in #803
• Fix bitsandbytes issue on ARM by @trvachov in #824
• Fixes for AMPLIFY QA scripts by @pstjohn in #825
• updated configs for benchmarks by @dorotat-nv in #833
• Remove temporary pins in docs build by @pstjohn in #828
• Adding baseline metrics for benchmarking ESM2 model by @ShevaNguyen in #831
• Updates docs for geneformer training, inference, and cellxclassification by @jomitchellnv in #823
• Add pre commit to verify test status by @pstjohn in #841
• fix geneformer image paths by @jomitchellnv in #839
• fix geneformer image links by @jomitchellnv in #844
• ESM2 PEFT by @polinabinder1 in #766
• Pbinder/esm2 document by @polinabinder1 in #846
• h11 CRIT vuln fix by @trvachov in #847
• Docs fix by @trvachov in #826

New Contributors

• @ShevaNguyen made their first contribution in #831

Full Changelog: v2.5...v2.6