MT5, Adafactor optimizer, additional schedulers

MT5, Adafactor optimizer, additional schedulers

Breaking change

• T5Model now has a required model_type parameter ("t5" or "mt5")

Added

• Added support for MT5
• Added support for Adafactor optimizer
• Added support for various schedulers:
  • get_constant_schedule
  • get_constant_schedule_with_warmup
  • get_linear_schedule_with_warmup
  • get_cosine_schedule_with_warmup
  • get_cosine_with_hard_restarts_schedule_with_warmup
  • get_polynomial_decay_schedule_with_warmup

Changed

• T5Model now has a required model_type parameter ("t5" or "mt5")

Fixed

• Fixed issue with class weights not working in ClassificationModel when using mult-GPU training