v0.1.4

What's Changed

• adding basic wandb support following what tensorboard was doing by @amazingvince in #12
• Config fixes by @pszemraj in #13
• Docs and utils by @pszemraj in #14

New Contributors

• @amazingvince made their first contribution in #12

Full Changelog: v0.1.3...v0.1.4

configuration improvements

default configurations closer to paper, fix gradient clipping, automatically set relative attention params (for longer seq len than 512), support custom wordpiece tokenizers and more

What's Changed

• Cleanup extra files by @pszemraj in #9
• pad vocab for CUDA by @pszemraj in #10
• Improve training config by @pszemraj in #11

Full Changelog: v0.1.2...v0.1.3

speedups, serialization fixes

What's Changed

• Speedups by @pszemraj in #4
• Save load fix by @pszemraj in #8

Full Changelog: v0.1.1...v0.1.2

streaming dataset support

pretrain with streaming hf datasets

What's Changed

• Fixes and updates by @pszemraj in #1
• Dtype fix by @pszemraj in #2
• Streaming dataset by @pszemraj in #3

New Contributors

• @pszemraj made their first contribution in #1

Full Changelog: https://github.com/pszemraj/annotated-mpnet/commits/v0.1.1