Summary of changes:
- Replaced the text embedding base model with an interleaved Rotaty Positional Embedding iRoPE in the Phishing detection NLP proof of concept
- Proof of concept that the entire model can scale at O(n) timing as sequence length increases infinitely.
Benchmarks:
seq_len val_bianry_accuracy min/model total_min timing_relative_to_1024 Commit_SHA
3072 0.955 65.942 329.715 2.817 4bc217b36d1baf8b9a81fe53000e9e13e6e87801
1536 0.96 37.27 186.36 1.591 286ba81a1e51493d748ded727bd602a4398248a8
1024 0.952 23.42 117.08 1.0 9893bfc55d4f7b753eff79e0c5c70e4992c61085
- Upgraded tf to 2.19.0
- Upgraded Jax
- Upgraded model architecture in both phishing CICD example and CIFAR10 example to accommodate tf 2.19.0.
- Removed obsolete BERT embedding CICD test.