Improve handling of default device switching

[tomfa]

Problem
Bad handling of default system microphone switch.

Context
It seems to me like switching the default system microphone causes recording/transcription to silently break, as shown in the video below. It does not seem to "un-break" if the default microphone switches back to the original.

This may seem like an edge case, so click here for a few examples

• Your partner turns on your bluetooth headset in the neighbouring room, but quickly shuts it off again.
• Your colleague turns on a conference bluetooth speaker in the neighbouring room. It connects to you shortly before they take over the BT-connection to the speaker (and your system swaps back to the correct mic).
• Your accidentally disconnect your (default) mic, and reconnects it.

To reproduce
(I tried this on Chrome, Mac)

1. Connect your (default) BlueTooth speaker to your machine.
2. Open the nextjs-real-time-transcription example app
3. Start recording, verify that it transcribes well.
4. Disconnect the (default) BT device.
5. Verify bug by seeing the lack of transcription.
6. Reconnect the (default) BT device.
7. Verify still present bug with lack of transcription.

Note: same bug if you move step 1 to 4 and replace 6 with the original 4

Expected behavior
I would expect a hickup, a detection of device switch, and a recovery.

I realize this may be technically challenging. If so, I'd hope for either:

• Remove the "default" device-id device from the deviceList array if s likely duplicate is found within the list with a regular id
• simply a console warning when selecting "default" device with a link to some docs about it

Workaround
We will no longer allow user to select the default microphone.

Video
In the video below, I say "1 minute, 2 minutes, 3 minutes... 10 minutes" on step 3, 5 and 7 in the reproduce list above. Only the first round is transcribed.

which-minutes-you-say.small.mov

Sidenote: The video is from noisy surroundings on bad wifi. But I have verified this issue previously in better conditions

[TudorCRL]

Hi @tomfa, thanks for raising this with us, once again great to get your feedback and thanks for all the details! We'll triage this at our standup tomorrow morning and get back to you with more details after that if that's okay?

[TudorCRL]

Hey @tomfa, we had a chat about this this morning, so thought I'd give you an update from our end. We're currently working on a big update of our docs site which should be landing in the next few weeks. Because of that the team's quite busy at the moment, and we think this issue is not an urgent, must-fix problem for now. For that reason we'll probably revisit it in a few weeks time once the docs work has settled down. Hopefully that works okay for you? Do let me know if there is some urgency in this on your end!

[tomfa]

Sounds good to me 👌

[mnemitz]

Hi @tomfa . Thanks again for the detailed issue report, and apologies for the delayed response!

The UX points you list are good, and I agree with them. I checked out the video you sent, and reproduced the issue locally.

I discovered (as you may already realize) that the cause of the issue there isn't that the change in default device changes the actual input, but just that the disconnection of the bluetooth device cuts off all audio input, but leaves the recorder still running on silence.

Confusingly then, the disabled select element shows the new default device, even though it isn't in use. It's worth noting that even if we did filter default in this case, it would still flip to a new value once the previously selected device disconnects.

So really there are three issues:

1. Having default as a device ID option leads to confusing UI with the select dropdown:
   • This can be solved by your suggestion to filter default where it aliases another device, which I'm happy to add to the library
2. Disconnecting an input device mid-recording leaves the PCMRecorder running indefinitely with no actual input:
   • ⏫ This is the biggest issue IMO
3. The notion of which device is "selected" is vague in the example:
   • Ambiguity between device currently recording, and device selected for next time you start recording.

Proposal

We'd like to keep these libraries as flexible as possible to different use-cases, so I would propose the following changes to the library:

1. Remove default from the device list where it aliases another device (I think this only exists in Chrome anyway)
2. On device disconnect mid-recording, stop recording and emit an event which can be used to notify the user, or potentially restart the session.
3. Add a deviceID field which can be set on the PCMRecorder class, to unambiguously represent which device is currently recording
4. Improve the example NextJS app to better handle the above issues

If you have any thoughts at all on this, we'd love to hear them. I'll hopefully have some time in the next week or two to look into these changes. Also happy to review any PRs if you'd like to implement any changes. Cheers!

[tomfa]

I think all of your proposals are good 👌 I will probably not contribute a PR on this issue, unfortunately.