Playback pauses due to buffer level going to zero

[scottveirs]

Describe the bug
As a live listening app user monitoring Bush Point hydrophone after troubleshooting a power loss, I experienced pauses or silent gaps in the playback on my Macbook within the Chrome browser.

To Reproduce
Steps to reproduce the behavior:

1. Go to Bitmovin player aimed at Bush Point data from today
2. Click on 'Play' (and you may need to unmute the player)
3. Listen for a few minutes and monitor the buffer level plot
4. Notice that the audio playback pauses for a few seconds whenever the buffer level decreases to zero.

Expected behavior
Continuous playback (assuming that the audio data is continuous!)

Screenshots
Here's a shot of the buffer level time series:

And here are two shots showing that when I'm experiencing the pauses within the live app (as oppose to the Bitmovin test player), the HLS segments seem to be available in S3 and downloaded with less than 1 min latency by my browser.

Desktop (please complete the following information):

• Hardware: MacBook Pro (16-inch, 2019)
• OS: OSX v.12.6
• Browser Chrome Version 107.0.5304.87 (Official Build) (x86_64)

Additional context
I think this has been going on for a while (6 months off and on?), and maybe mostly for Bush Point -- but why that would be I do not know! It may be related to a lurking, hard-to-reproduce behavior of the the live app UI... possible playback performance differences based on which feed you load first, or perhaps the sequence of feeds you visit, i.e. if you load Orcasound Lab first sometimes, it seems the playback button does not function.

[scottveirs]

A weird, novel performance issue this morning: listening to Bush Point had pretty good playback -- just intermittent skips or silent periods of 0.5-2s -- but then player stopped. Network tab showed first a live000.ts and then a seemingly random, much earlier segment being loaded over and over again (live028.ts):

Previously, the player had been working on segments up around live1900.ts ...

[scottveirs]

Noticed via Quilt that some of the .ts segments are much smaller than the others. This suggests a bug or other issue on the Raspberry Pi causing data drops. The player may be responding in unpredictable ways to these shortened, possibly corrupt files...

Normal size seems to be about 198kB --

Here are a few shorter ones (<1kB and 74kB) --

[scottveirs]

Last notes, looking briefly at htop on the Bush Point node via Dataplicity, it seems that ffmpeg is pushing one core pretty hard. Why doesn't threading work to spread load?

[paulcretu]

Noticed via Quilt that some of the .ts segments are much smaller than the others. This suggests a bug or other issue on the Raspberry Pi causing data drops. The player may be responding in unpredictable ways to these shortened, possibly corrupt files...

This issue might be a candidate for moving over to orcanode… seems like it's unclear if the problem is how the stream is generated or how the player is handling it. Will leave the issue here for now until we can investigate (and see if it's still happening in v3)

[paulcretu]

No reports of anything similar happening lately, moving this issue to orcanode for now. If it doesn't happen anymore, we should close it out as stale (though there are some good orcanode questions in here to investigate)