Matrix Profile XXI (MERLIN algorithm for anomalies ) & XXII (SWAMP a.k.a. Motifs under DTW)

[udf2457]

I was reading about these two innovations presented at ICDM 2020:

• Matrix Profile XXI: MERLIN: Parameter-Free Discovery of Arbitrary Length Anomalies in Massive Time Series Archives. Takaaki Nakamura, Makoto Imamura, Ryan Mercer and Eamonn Keogh. ICDM 2020 [pdf,author's Matlab reference implementation]
• Matrix Profile XXII: Exact Discovery of Time Series Motifs under DTW. Sara Alaee, Ryan Mercer, Kaveh Kamgar, Eamonn Keogh. ICDM 2020 [pdf, author's Matlab reference implementation]

It would be really neat to see these implemented in stumpy , perhaps especially the MERLIN algorithm.

[seanlaw]

@udf2457 Welcome to the STUMPY community and thank you for the suggestions. We have some contributors who are planning to work on Merlin. However, I haven't dug into it yet but it appears that MERLIN does not seem to use matrix profiles directly (do you know anything about this?). We'll need to explore but it will likely be happening in the fall.

Unfortunately, SWAMP is likely outside of the scope of STUMPY as we do not have the expertise to implement a fast DTW.

[udf2457]

@seanlaw Thanks for the swift reply. In relation to MERLIN I believe you are correct about it not using MP directly, their paper makes two key observations in that area:

1. They observe that MP detects anomalies as a by-product of the motif discovery process of MP
2. They point out that MP is more computationally intensive than is required if your goal is anomaly detection rather than motifs.

[seanlaw]

@udf2457 Thank you for the context. I have added a new issue to track the development of MERLIN. Since it sounds like you have a reasonable understanding of the paper and if you are available/interested, I encourage you to participate and provide your feedback where possible (i.e., to help talk through the development, to provide feedback on the user API, to review the PR, etc). This will really help make the feature better.

I should probably preface all of this by saying that there have been times where we were unable to reproduce the published work due to missing information or inconsistencies between the paper and the reference Matlab code. However, that is really the goal of the STUMPY project. We "try" to figure it out so that users don't have to reinvent the wheel from only looking at a picture of the wheel :)

[udf2457]

@seanlaw Have a fair bit on at $work at the moment, so can't make any promises, but if I get the opportunity I'll certainly try to drop by now and then.

[seanlaw]

@seanlaw Have a fair bit on at $work at the moment, so can't make any promises, but if I get the opportunity I'll certainly try to drop by now and then.

Sounds good! Every bit helps