Skip to content

Latest commit

 

History

History
88 lines (73 loc) · 7.99 KB

STATUS.MD

File metadata and controls

88 lines (73 loc) · 7.99 KB

SparseML Integration Project Status Page

Feature status tables related to required and target features for SparseML sparsification aware training integrations

Base Training

NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
cli ✅ ✔️ ✅ ✅ ✅
api ✅ ✔️ ✅ ✅ ✅
dense_training ✅ ✔️ ✅ ✅ ✅
gradient_accumulation ❌ ❌ ✅ ✔️ ✅
DP ✅ ✔️ ✅ ✅ ✅
DDP ✅ ✔️ ✅ ✅ ✅

Sparsification

Features related to sparsification integration. Notes:

  • Recipe support should be optional
  • AMP must be disabled during QAT. (scaler._enabled = False) Distillation:
  • distillation_teacher kwarg must be passed to manager initialzation
  • Call loss = manager.loss_update(...) after loss is computed
NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
recipe ✅ ✅ ✅ ✅ ✅
recipe_args ✅ ❌ ✅ ✅ ✅
EMA ❌ ❌ ✅ ❌ ✅
AMP ✅ ❌ ✅ ✅ ✅
distillation ❌ ❌ ✅ ✅ ✅

Datasets

NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
use_standard_datasets ✅ ✔️ ✅ ✅ ✅
train_val_test_datasets ✅ ✔️ ✅ ✅ ✅
auto_download_datasets ✅ ✔️ ✅ ✔️ ❌

Checkpoints

Features related to checkpoints. Notes:

  • best_* checkpoints can only be saved after the entire sparsification step completes
  • update_architecture_from_recipe requires a call to apply_structure() on a torch model before loading sparsified checkpoint
  • staged_recipes requires manager.compose_staged(...) before checkpoint save
NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
original_integration_checkpoints ✅ ✅ ✅ ✅ ✅
sparsezoo_checkpoints ✅ ✅ ✅ ✅ ✅
best_checkpoint ✅ ✅ ✅ ✅ ✅
best_pruned_checkpoint ❌ ❌ ❓ ❌ ❌
best_pruned_quantized_checkpoint ❌ ❌ ❓ ❌ ❌
recipe_saved_to_checkpoint ❌ ❌ ❓ ❌ ❌
update_architecture_from_recipe ✅ ✅ ✅ ✅ ✅
staged_recipes ✅ ✅ ✅ ✅ ✅

Logging

Logging units for x axis in logging should be number of optimizer steps. Notably: num_optimizer_steps = num_batches / gradient_accum_steps. So when gradient_accumuluation is not used, the x axis will be number of batches trained on.

NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
stdout ✅ ✔️ ✅ ✅ ✅
weights_and_biases ❌ ❌ ✅ ✅ ✅
tensorboard ✅ ❌ ✅ ✅ ✅

Export

PyTorch export features should use ModuleExporter and only require specifying checkpoint path and necessary configuration files

NM IC OpenPifPaf YOLOv5 Transformers Torchvision IC
cli ✅ ✅ ✅ ✅ ✅
api ✅ ✅ ✅ ✅ ✅
one_shot ✅ ✅ ✅ ✅ ✅
onnx ✅ ✅ ✅ ✅ ✅
torch_script ❌ ❌ ✔️ ❌ ❌
static_batch_size ❌ ❌ ✔️ ❌ ❌
dynamic_batch_size ✅ ✅ ✅ ✅ ✅
static_input_shape ✅ ✅ ✅ ✅ ✅
dynamic_input_shape ❌ ❌ ❌ ❌ ❌
save_to_simple_deployment_directory ✅ ✅ ✅ ✅ ✅
save_to_sparsezoo_directory ❌ ❌ ❌ ❌ ❌

Key

  • ✅ - implemented by neuralmagic integration
  • ✔️ - implemented by underlying integration
  • ❌ - not implemented yet
  • ❓ - not sure, not tested, or to be investigated