the accuracy rate described in the comments of the example does not match the results obtained from the actual test

[Aik197]

Hello my friend!
The accuracy rate described in the comments of the example does not match the results obtained from the actual test the when I am running the sample code of lwf_mnist.py. and I noticed that the dataset of lwf_mnist is split_mnist dataset which is class incremental therefore the number of labels in the stream will keep increasing, but model used in the example has a default output category of 10, does this mean that the model has no ability to extend the output FC layer cause we have not overrriden a certain method like "adaptation" to enable the model/strategy expand itself?
The result of my run is that after each experience is trained, the recognition rate of the previous experience drops to 0%.
Looking forward to your answer!

[AndreaCossu]

Hi @Aik197 . The model has already 10 output units, so in this case you don't need to perform adaptation. In case, you can substitute the final layer with an IncrementalClassifier so that you can set up an initial number of classes and then they will expand over time.

For the performance of LwF on Split MNIST, I recommend to use the master version of avalanche and check its performance by running the experiment present in the continual learning baselines repository. You can find it under experiments/split_mnist/lwf.py. LwF may under-perform when using avalanche v0.2.1 but it should work on the master branch. Let us know!

[Aik197]

ohh，thank you for your reply. Based on my exp, I think LWF does not performance well in nc scenes, and I would also like to ask what is the difference between version 0.2.1 and the master? According to my observation, in the evalutaion session, version 0.2.1 prints out the evaluation results for all tasks everytime if you use acc_metrics and set stream opt 2 True, while master prints the average accuracy of all tasks, does this means the version 0.2.1 is a latest version which fixed the errors but we still recommend to use the matser cause it's stability?