Time-series regression problem for a Multi-Incremental-Task

[ESarikayaPTW]

Hello,

I am trying to use Avalanche for a time-series regression problem with a LSTM network. The task can be described as a Multi-Incremental-Task (MIT), since I have a repitition of task labels over time.

The data I use is attached here to reproduce the code below:
Dataset.zip

Code

import torch
from torch.optim import SGD
from torch.nn import L1Loss
from avalanche.models import MTSimpleSequenceClassifier
from avalanche.training.supervised import Naive
from avalanche.benchmarks.generators import tensors_benchmark

#Imports a list of 24 tensor datasets, each including (x,y)
datasets = torch.load('experiences.pkl')

#Imports a list of task labels (t)
task_labels = torch.load('task_labels.pkl')

scenario = tensors_benchmark(
    train_tensors=datasets,
    test_tensors=datasets,
    task_labels=task_labels
)

train_stream = scenario.train_stream
test_stream = scenario.test_stream

model = MTSimpleSequenceClassifier(
    input_size=14,
    hidden_size=10,
    rnn_layers=1,
    batch_first=True
)

optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9)

criterion = L1Loss() #mean absolute error (MAE)

cl_strategy = Naive(
    model,
    optimizer,
    criterion,
    train_mb_size=32,
    train_epochs=8,
    eval_mb_size=32
)

results = []
for experience in train_stream:
    cl_strategy.train(experience)
    results.append(cl_strategy.eval(test_stream))

Error

C:\Users\Lokal_Sarikaya\Desktop\Features\venv\Scripts\python.exe C:/Users/Lokal_Sarikaya/Desktop/Features/gitrequest.py
-- >> Start of training phase << --
Traceback (most recent call last):
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\gitrequest.py", line 45, in <module>
    cl_strategy.train(experience)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\training\templates\base_sgd.py", line 146, in train
    super().train(experiences, eval_streams, **kwargs)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\training\templates\base.py", line 115, in train
    self._before_training_exp(**kwargs)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\training\templates\base_sgd.py", line 237, in _before_training_exp
    self.check_model_and_optimizer()
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\training\templates\observation_type\batch_observation.py", line 30, in check_model_and_optimizer
    self.model = self.model_adaptation()
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\training\templates\observation_type\batch_observation.py", line 16, in model_adaptation
    avalanche_model_adaptation(model, self.experience)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\models\utils.py", line 19, in avalanche_model_adaptation
    module.adaptation(experience)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\models\dynamic_modules.py", line 367, in adaptation
    self.classifiers[tid].adaptation(experience)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\Lokal_Sarikaya\Desktop\Features\venv\lib\site-packages\avalanche\models\dynamic_modules.py", line 238, in adaptation
    self.active_units = torch.zeros(new_nclasses, dtype=torch.bool)
TypeError: zeros() received an invalid combination of arguments - got (float, dtype=torch.dtype), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of SymInts size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)


Process finished with exit code 1

My Questions

1. Is this the right approach for my problem?
2. It seems that the model is only suited for classification tasks. Can I use it for regression or do you recommend to use other models?
3. the error message links me to 'dynamic_modules.py'. There it seems to adapt the model's architecture by the number of new "new_nclasses", which seems to be determined by the size of my target vector. What I would actually expect here is that the adaption is only triggered for a new "task_label"

Thank you in advance 🙏

[AntonioCarta]

MTSimpleSequenceClassifier is using a multi-head classifier. You can look at the MultiHeadClassifier code and adapt it for regression easily if you want a separate output head for each regression task.