Performance Discrepancy in Netsim Dataset Training

[shinmc12]

I hope this message finds you well. I am currently working on replicating the training process described in the paper and have set up the environment accordingly. However, I noticed a significant performance discrepancy when training on the Netsim dataset.

Specifically, the paper reports an AUROC of 0.688 ± 0.051 for the Netsim dataset, whereas my own training results yielded an AUROC of 0.5347511574, which is notably lower.

Could you kindly clarify:

1. If there were any updates or modifications to the implementation that could explain this performance difference?
2. Whether there are specific hyperparameter settings optimized for the Netsim dataset that I might need to adjust in my setup?

I would greatly appreciate any guidance or insights you could provide on this matter. Thank you for your time and support.