SpeechmaticsRealtimeError - Socket not ready to receive audio

[vlashada]

I am getting the error SpeechmaticsRealtimeError - Socket not ready to receive audio when using a setup very similar to the documentation in nextjs realtime transcription.

I think the problem is that sometimes the code below is executed when it should not. Is there any more robust way of starting and stopping and making sure everything is stopped when I run a callback? Is there for instance any way of checking that it is valid to call stop transcription and stop recording? It does not seem to affect performance, but it is really messing with my logging.

  useEffect(() => {
    return () => {
      stopTranscription();
      stopRecording();
    };
  }, [stopTranscription, stopRecording]);

[mnemitz]

Thanks for reaching out. Would you mind confirming that you're using the latest versions of @speechmatics/browser-audio-input-react and @speechmatics/real-time-client-react, or if not which versions you are using?

I agree having this cleanup in a useEffect makes it a little bit error prone. The reason it was in the example was to ensure that if the user navigates to another page in the app without disconnecting, the connection doesn't hang open.

Depending on your use-case, you could potentially do away with this useEffect cleanup entirely, and allow the user to fully control stopping transcription (i.e. just via a button), but that would assume the user isn't performing any client-side navigation. Would something like that suit your use case?

I will investigate changing the example to not rely on this useEffect, and maybe try intercepting navigation events or something similar (although this isn't super simple with app router unfortunately.

[klolo]

I have the same problem:

720chunk-DGJPC7GG.js?v=d8e01145:111 Uncaught SpeechmaticsRealtimeError: Socket not ready to receive audio
    at RealtimeClient.sendAudio (chunk-DGJPC7GG.js?v=d8e01145:111:13)
    at @speechmatics_real-time-client-react.js?v=d8e01145:75:14
    at PCMRecorder.onAudio (@speechmatics_browser-audio-input-react.js?v=d8e01145:419:7)`

with versions:

    "@speechmatics/browser-audio-input-react": "^2.0.0",
    "@speechmatics/real-time-client-react": "^1.0.0",

My component (I tried to create the simplest working component in React...no idea how to do it correctly...documentation seems to be not completed):

import React, { useState, useCallback, useEffect } from 'react';
import {
  useRealtimeTranscription,
  type RealtimeTranscriptionConfig,
} from '@speechmatics/real-time-client-react';
import { usePCMAudioListener, usePCMAudioRecorderContext } from '@speechmatics/browser-audio-input-react';
import { fetchJWT } from './RecordingButton';

const SpeechmaticsRealtimeTranscriber = () => {
  const [transcript, setTranscript] = useState<string>('');
  const [isRecording, setIsRecording] = useState<boolean>(false);
  const [error, setError] = useState<string | null>(null);

  const { startTranscription, stopTranscription, sendAudio, socketState } = 
    useRealtimeTranscription();
  const { startRecording, stopRecording } = usePCMAudioRecorderContext();
  usePCMAudioListener(sendAudio);

  const config: RealtimeTranscriptionConfig = {
    transcription_config: { 
      language: 'en', 
      output_locale: 'en-GB',
      enable_partials: true
    }
  };

  const startSession = useCallback(async () => {
    try {
      setError(null);
      const jwt = await fetchJWT();
      
      await startTranscription(jwt, config);
      await startRecording({});
      
      setIsRecording(true);
    } catch (err) {
      setError(`Błąd inicjalizacji: ${JSON.stringify(err)}`);
      console.error(err);
    }
  }, [startTranscription, startRecording]);

  const stopSession = useCallback(async () => {
    try {
      await stopRecording();
      await stopTranscription();
      setIsRecording(false);
    } catch (err) {
      setError(`Cannot stop: ${JSON.stringify(err)}`);
    }
  }, [stopRecording, stopTranscription]);

  useEffect(() => {
    const handleMessage = (event: MessageEvent) => {
      try {
        console.log('event', event);
        
        if (event.data.metadata?.transcript) {
          setTranscript(prev => prev + ' ' + event.data.metadata.transcript);
        }
      } catch (err) {
        console.error('Error parsing message:', err);
      }
    };

    window.addEventListener('message', handleMessage);
    return () => window.removeEventListener('message', handleMessage);
  }, []);

  return (
    <div className="transcriber-container">
      <h2>Transkrypcja mowy w czasie rzeczywistym</h2>
      
      <div className="controls">
        <button 
          onClick={isRecording ? stopSession : startSession}
          disabled={socketState === 'connecting'}
          className={`record-btn ${isRecording ? 'recording' : ''}`}
        >
          {isRecording ? '⏹ stop' : '⏺ record'}
        </button>
        
        <div className="status">
          Status: {socketState || 'nieaktywny'}
          {error && <span className="error">{error}</span>}
        </div>
      </div>
      
      <div className="transcript-box">
        <h3>Transkrypcja:</h3>
        <p>{transcript || <em>first start recording...</em>}</p>
      </div>
    </div>
  );
};

export default SpeechmaticsRealtimeTranscriber;`

[vlashada]

Is there any nice way to check if the recording status is actually recording/transcribing?

[mnemitz]

Thanks both for reaching out as well. @vlashada The libraries expose two fields which monitor the recording and transcription status:

Recording status

For recording, the isRecording boolean is exported directly from the hook (see README here). There's no need to keep that in a useState.

import { usePCMAudioRecorderContext } from "@speechmatics/browser-audio-input-react";

// ...

  const { isRecording } = usePCMAudioRecorderContext();

Transcription status

For tracking the transcription connection status, the socketState is returned from the hook as well:

  const { socketState } =
    useRealtimeTranscription();

@klolo For your code, it looks like you're listening for MessageEvents through window.addEventListener. Instead you'll want to use the useRealtimeEventListener hook to listen for Websocket messages (see example code here). I'll update the README for the real-time-client-react package to include this function.

I'm not sure why you're getting the socket error though. It's possible you're getting an authentication error after the websocket starts up. Could you try including this code to see if it shows any errors?:

import { useRealtimeEventListener } from "@speechmatics/real-time-client-react";

const SpeechmaticsRealtimeTranscriber = () => {
  // ...

  // Log incoming errors from the WebSocket
  useRealtimeEventListener('receiveMessage', ({ data }) => {
    if (data.message === 'Error') {
      console.error('Websocket error: ', data);
    }
  });
}

[TudorCRL]

@klolo hope you're doing well! Any update on this? Let us know if we can help in any way! Otherwise, I'll close this issue shortly as resolved and/or stale