Function to select a bespoke BirdNET model and/or exact versions of BirdNET model

[KristianBell]

I have developed a multi-class frog model using BirdNET and it would be great to be able to select my own custom model within Chirpity. I dont feel like it would be a big task given Chirpity can already well understand the TFLite format. Also, there are occasions when a new survey requires backwards consistency with an old survey and it would also be good to be able to select older versions of BirdNET for these occasions.

[Mattk70]

HI @KristianBell,

I have been working toward the ability to import custom models into Chirpity for some time - see for example #48. There are a couple of blockers.

Firstly, Chirpity does NOT use TFlite models, it uses TFJS models. A TFJS model can be converted from TensorFlow SavedModel format, so if you train a model using BirdNET Analyzer and save as a Raven format model instead of TFlite it could work.

It is also possible for Chirpity to run Tensorflow SavedModels directly, without conversion, as there is a Node.js binding for this. HOWEVER, for some reason, whilst this will run the original BirdNET 2.4, and older versions, attempting to run models trained and saved in the newer versions of BirdNET Analyzer causes errors. I have raised a question over on the BirdNET repo about that here.

I also did some digging, and suspect that the problem is that the Node.js bindings for Tensorflow are quite old (they appear to have been compiled against Tensorflow 2.9.1). I have tried to get some insight into this from the Tensorflow team here.

There is a package that runs TFLite models in the browser, but it is very out of date, no longer supported and does not work with BirdNET TFLite models.

Whilst those issues are outstanding, it is unlikely there will be a simple option to load custom models into Chirpity. Since the TFJS converter is a Python utility, it would be difficult to include this in the Chirpity package, and the conversion of BirdNET models is far from straightforward due to their use of custom layers and Ops.

So, unfortunately, it is a very difficult task indeed.

[KristianBell]

Thanks Matt, I can see it is much more difficult than first glance would suggest and I appreciate the reply. Have you had a look at the possibility of integrating Perch models instead? Google has a great little tool for developing custom recognisers but they dont seem remotely interested in then making it easy to use any custom model on fresh data.

[Mattk70]

HI Kristian, yes I've done the same experiments with Perch. Perch V1 loads, but more recent versions throw the same error as BirdNET with the Node bindings. You would think the Tensorflow team would show some interest, since both Perch and TensorFlow come from Google.