Skip to content

v0.10.0-alpha: Merge pull request #172 from david-thrower/171-upgrade-tf-2190

Latest
Compare
Choose a tag to compare
@david-thrower david-thrower released this 12 Apr 03:56
6dfb427

Summary of changes:

  • Replaced the text embedding base model with an interleaved Rotaty Positional Embedding iRoPE in the Phishing detection NLP proof of concept
  • Proof of concept that the entire model can scale at O(n) timing as sequence length increases infinitely.

Benchmarks:

seq_len     val_bianry_accuracy       min/model        total_min         timing_relative_to_1024                      Commit_SHA
3072          0.955                    65.942           329.715                    2.817                     4bc217b36d1baf8b9a81fe53000e9e13e6e87801
1536          0.96                     37.27             186.36                    1.591                     286ba81a1e51493d748ded727bd602a4398248a8
1024          0.952                    23.42             117.08                    1.0                       9893bfc55d4f7b753eff79e0c5c70e4992c61085
  • Upgraded tf to 2.19.0
  • Upgraded Jax
  • Upgraded model architecture in both phishing CICD example and CIFAR10 example to accommodate tf 2.19.0.
  • Removed obsolete BERT embedding CICD test.