usePCMAudioPlayerContext().analyser always undefined

[yovizn]

I tried using Flow in my Next.js projects, and I also did the same thing in examples/nextjs-flow. I tried to run the analyzer with usePCMAudioPlayerContext, but the analyzer never appeared. I have rendered FlowProvider on the server. startConversation() is working properly. I can also successfully connect to Speechmatics. Everything works except usePCMAudioPlayerContext().analyser.

This is the implementation of my AudioProvider:

"use client";

import { PCMAudioRecorderProvider } from "@speechmatics/browser-audio-input-react";
import { PCMPlayerProvider } from "@speechmatics/web-pcm-player-react";
import { useEffect, useMemo, useSyncExternalStore } from "react";

type SpeechmaticsProviderProps = {
  children: React.ReactNode;
};

export const RECORDING_SAMPLE_RATE =
  typeof navigator !== "undefined" && navigator.userAgent.includes("Firefox") ? undefined : 16_000;

export function SpeechmaticsProvider({ children }: SpeechmaticsProviderProps) {
  const { inputAudioContext, playbackAudioContext } = useAudioContexts();

  return (
    <PCMAudioRecorderProvider
      audioContext={inputAudioContext}
      workletScriptURL="/js/pcm-audio-worklet.min.js"
    >
      <PCMPlayerProvider audioContext={playbackAudioContext}>{children}</PCMPlayerProvider>
    </PCMAudioRecorderProvider>
  );
}

// This hook returns audio contexts for recording and playback.
// In practice they will be the same AudioContext, except in Firefox where sample rates may differ
// See bug tracked here: https://bugzilla.mozilla.org/show_bug.cgi?id=1725336https://bugzilla.mozilla.org/show_bug.cgi?id=1725336
// TODO: If/when the bug is fixed, we can use the same audio context for both recording and playback
function useAudioContexts() {
  const hydrated = useHydrated();
  const inputAudioContext = useMemo(
    () => (hydrated ? new window.AudioContext({ sampleRate: RECORDING_SAMPLE_RATE }) : undefined),
    [hydrated],
  );

  const playbackAudioContext = useMemo(() => {
    const isFirefox = navigator.userAgent.toLowerCase().indexOf("firefox") > -1;
    return isFirefox
      ? new window.AudioContext({ sampleRate: RECORDING_SAMPLE_RATE })
      : inputAudioContext;
  }, [inputAudioContext]);

  useCleanupAudioContext(inputAudioContext);
  useCleanupAudioContext(playbackAudioContext);

  return { inputAudioContext, playbackAudioContext };
}

// Lets us know if we're rendering client side or not
const useHydrated = () =>
  useSyncExternalStore(
    () => () => {},
    () => true,
    () => false,
  );

// Close audio context when component unmounts
function useCleanupAudioContext(context?: AudioContext) {
  useEffect(() => {
    return () => {
      if (context && context.state === "running") {
        context.close();
      }
    };
  }, [context]);
}

Can you help me?

[mnemitz]

Thanks for getting in touch. Your implementation of the Provider looks correct.

I've tried running your code in the NextJS Flow example in this repo. I've pushed those changes to a branch called issue/174-repro: https://github.com/speechmatics/speechmatics-js-sdk/tree/issue/174-repro/examples/nextjs-flow. It's just using the code you provided, plus adding an AudioVisualizer to the right-hand section.

Locally for me, it works fine in Chrome. Can you confirm whether it works on your end?

EDIT: Apologies, I misread and thought you were referring to the recorder (not the player). The code on this branch of the example shows the player visualizer working properly as well. Can you confirm whether it works for you?

https://github.com/speechmatics/speechmatics-js-sdk/tree/issue/174-repro/examples/nextjs-flow