help understanding classifier training and improving the results

[jeremylindsell]

I used the BirdNET-Analyzer-GUI to train a custom classifier for a species of frog after watching the very helpful video HERE (thank you Stefan @kahst). I used 1000 three-second call samples (there is only one call type for the species) and 1000 NOISE samples and left the training settings at their defaults (100 epochs, batch size 32, learning rate 0.01 - I've rather little idea what they do).

The result from the training didn't appear as I expected (i.e. a curve converging towards an asymptote). Instead it was a horizontal straight line with no inflections; and a reported precision of 0.54, which sounded uninspiring.

In fact, despite that low number the classifier still did a useful job on a dataset. I set the minimum confidence to 0.8 (I chose this arbitrarily but it seemed to reduce the false presence rate) and it missed rather few calls in a collection of 1440 one minute recordings. But it did miss some real stand out cases. And I also found that the time stamp was a bit off to one side of the actual call (usually a second or two too early). Real life precision was indeed about 0.6 with lots of false presence.

So I could feasibly output a bunch of PNGs from all the detections, manually throw out the false cases, and quietly ignore the fact that there are missed calls becuase they are relatively few. But the call of this frog is so characteristic and consistent (and easy to spot by eye in a spectrogram!) that I don't believe the results can't be better.

Rather than stumble around trying different settings and combinations of traiing files can anyone suggest why the training didn't seem to have a period of convergence and the best steps I can try in order to improve the classifier? More calls training data? More noise data? Adjusting those default training values in a specific way?

Thank you so much

[jeremylindsell]

PS I do get a warning when I launch the GUI. I've no idea how consequential it is or whether I can resolve it:

WARNING:tensorflow:AutoGraph is not available in this environment: functions lack code information. This is typical of some environments like the interactive Python shell. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#access-to-source-code for more information.

[kahst]

I suspect that the learning rate you have chosen is too high. Try lowering it, halving it each run and see if that helps. I also recommend the new autotune feature in the latest version of the GUI.