Live Analysis

[mcguirepr89]

Hello, all

I wanted to share this as a feature you can expect to see at some point

vokoscreen-2021-10-12_15-25-15.mp4

[nilspupils]

Great you are doing BirdNET Demo! Was pleasantly surprised about the german language at first ;)

[DD4WH]

I seem not to be able to watch the video!?

[mcguirepr89]

I sometimes can watch it, and sometimes I cannot. Github servers are a little wishy-washy sometimes. I'll try and repost and see if you can see it

vokoscreen-2021-10-12_15-25-15.mp4

[ghost]

QUESTION: Does this work in real-time with the microphone?

[mcguirepr89]

Yes -- truly real-time

[mcguirepr89]

What is needed for this to work well

Right now, this works, but not as it should, and I think all the kinks are due to old JavaScript

Broken

• The audio stream does not play through the web interface. Something about Chrome's updated autoplay policies and changes in the AudioContext webkit are also warnings that are displayed in the web console.
• The realtime spectrogram has appeared to have worked momentarily for a time or two, but it was before I did any deeper debugging, and now I can't get it working at all anymore and wonder if it ever really did.
• This seems to work for about 30-45 minutes per browser before python paste stops pushing new information. I think this might have something to do with the incredibly high number of error messages by that point (in the thousands), or something about caching that is too esoteric for me to even guess properly.

Working

• The actual detections work great and lightening fast. The original code is only meant to work on a very small WHITELIST, that I went ahead and replaced with the entire /model/labels.txt from the BirdNET-Lite repo. This was just to see how to change its output beyond its DEMO purposes. There seems to be an easy-enough scheme available in there as well to use localization for the browser.
• The audio stream that is supposed to play in the browser during the live detection works at the source URL the detections page is supposed to use. This can easily replace my livestream.service, though right now it stays in mono for the detections.
• The species in Stefan's WHITELIST also had photos to go along with their detection. For those species, even after doing away with the WHITELIST altogether, their photo displaying function is intact. That is to say, any species for which you add an image entitled with its corresponding species "label" from the "labels.txt" file will display as intended.

Pretty Doable

• The python script reads the model and a json formated version of the labels, or something like that. It seems it would be doable enough to create one from the BirdNET-Lite labels.txt so that this can run on the BirdNET-Lite tflite model.

HELP WANTED

• I don't know anything about JavaScript, and the issues that keep this form working flawlessly are going to be nearly impossible for me to find on my own.
• The python can also be optimized for our database, thought I would want a more streamlined approach to this. I can write bash scripts in my sleep, but I am a hacky python developer and an even hackier and green javascripter