🐛 [Bug?] Skia + useSkiaFrameProcessor crash with “SkiaSGRoot of undefined” in worklet

[Poovit-B]

What's happening?

When I use useSkiaFrameProcessor and reference any Skia object or API, the app crashes immediately with the following error:

'TypeError: Cannot read property \'S\' of undefined', { [Component Stack] name: 'Component Stack' }
(NOBRIDGE) ERROR  TypeError: Cannot read property 'S' of undefined
(NOBRIDGE) ERROR  TypeError: Cannot read property 'SkiaSGRoot' of undefined

"react-native": "0.76.9",

Reproduceable Code

const frameSkiaProcessor = useSkiaFrameProcessor(
    frame => {
      'worklet';
      frame.render();
      if (!pluginFrame) {
        return;
      }

      runAtTargetFps(targetFps, () => {
        'worklet';
        const resizedFrame = resize(frame, {
          scale: {
            width: MODEL_SIZE,
            height: MODEL_SIZE,
          },
          crop: {
            y: 0,
            x: 0,
            width: MODEL_SIZE,
            height: MODEL_SIZE,
          },
          pixelFormat: 'rgb',
          dataType: 'float32',
        });

        const outputs = pluginFrame.runSync([resizedFrame]);
        const detection_boxes = outputs[1];
        const detection_classes = outputs[3];
        const detection_scores = outputs[0];

        const minConf = 0.7;
        const frameWidth = MODEL_SIZE;
        const frameHeight = MODEL_SIZE;
        let detections = [];

        // Loop over all detections and draw detection box if confidence is above minimum threshold
        for (let i = 0; i < detection_scores.length; i++) {
          if (detection_scores[i] > minConf && detection_scores[i] <= 1.0) {
            // Get bounding box coordinates and draw box
            const ymin = Math.max(1, detection_boxes[i * 4] * frameHeight);
            const xmin = Math.max(1, detection_boxes[i * 4 + 1] * frameWidth);
            const ymax = Math.min(
              frameHeight,
              detection_boxes[i * 4 + 2] * frameHeight,
            );
            const xmax = Math.min(
              frameWidth,
              detection_boxes[i * 4 + 3] * frameWidth,
            );

            // Add detection to list
            const objectName = LAUNDRY_SYMBOLS_LABELS[detection_classes[i]]; // Look up object name from "labels" array using class index
            const label =
              LAUNDRY_SYMBOL?.[objectName]?.[`label_${i18n.locale}`]; // Look up object name from "labels" array using class index
            const percent = Math.round(detection_scores[i] * 100);
            detections.push({
              objectName,
              percent,
              xmin,
              ymin,
              xmax,
              ymax,
            });

            const rect = Skia.XYWHRect(xmin, ymin, xmax - xmin, ymax - ymin);
            frame.drawText(label, xmin, ymin - 5, paintText, font);
            frame.drawRect(rect, paint);
          }
        }

        detections = detections.filter(
          (v, i, a) =>
            a.findIndex(t => t.objectName === v.objectName) === i &&
            v?.objectName,
        );

        onWorkletProcesser({detections});
      });
      return;
    },
    [pluginFrame, lists, isActive],
  );


const ReanimatedCamera = Reanimated.createAnimatedComponent(Camera);
Reanimated.addWhitelistedNativeProps({
  zoom: true,
});
              <ReanimatedCamera
                ref={cameraRef}
                style={[
                  StyleSheet.absoluteFill,
                  {
                    flex: 1,
                    width: WIDTH,
                    height: WIDTH,
                  },
                ]}
                device={device}
                isActive={isActive}
                // isActive={false}
                pixelFormat={pixelFormat}
                format={format}
                frameProcessor={Platform.select({
                  ios: frameSkiaProcessor,
                  android: frameProcessor,
                })}
                animatedProps={animatedProps}
                enableZoomGesture={false}
                // onTouchEnd={onTouchEnd}
              />

Relevant log output

'TypeError: Cannot read property \'S\' of undefined', { [Component Stack] name: 'Component Stack' }
Warning: TypeError: Cannot read property 'SkiaSGRoot' of undefined

This error is located at:
    in Canvas (created by SkiaCameraCanvasImpl)
    in SkiaCameraCanvasImpl (created by Camera)
    in CameraView (created by Camera)
    in Camera (created by AnimatedComponent(Camera))
    in AnimatedComponent(Camera)
    in Unknown (created by PinchGestureHandler)
    in PinchGestureHandler (created by SymbolMain)
    in RCTView (created by View)
    in View (created by GestureHandlerRootView)
    in GestureHandlerRootView (created by SymbolMain)
    in RCTView (created by View)
    in View (created by SymbolMain)
    in RCTView (created by View)
    in View
    in StyledNativeComponent (created by Styled(View))
    in Styled(View) (created by SymbolMain)
    in SymbolMain (created by SceneView)
    in StaticContainer
    in EnsureSingleNavigator (created by SceneView)
    in SceneView (created by StackNavigator)
    in RCTView (created by View)
    in View (created by CardContainerInner)
    in RCTView (created by View)
    in View (created by CardContainerInner)
    in RCTView (created by View)
    in View
    in CardSheet (created by Card)
    in RCTView (created by View)
    in View (created by Animated(View))
    in Animated(View) (created by PanGestureHandler)
    in PanGestureHandler (created by PanGestureHandler)
    in PanGestureHandler (created by Card)
    in RCTView (created by View)
    in View (created by Animated(View))
    in Animated(View) (created by Card)
    in RCTView (created by View)
    in View (created by Card)
    in Card (created by CardContainerInner)
    in CardContainerInner (created by CardStack)
    in RNSScreen (created by Animated(Anonymous))
    in Animated(Anonymous)
    in Suspender (created by Freeze)
    in Suspense (created by Freeze)
    in Freeze (created by DelayedFreeze)
    in DelayedFreeze
    in InnerScreen (created by Screen)

    in Screen (created by MaybeScreen)
    in MaybeScreen (created by CardStack)
    in RNSScreenContainer (created by ScreenContainer)
    in ScreenContainer (created by MaybeScreenContainer)
    in MaybeScreenContainer (created by CardStack)
    in RCTView (created by View)
    in View (created by CardStack)
    in CardStack (created by HeaderShownContext)
    in FrameSizeProvider (created by SafeAreaProviderCompat)
    in RCTView (created by View)
    in View (created by SafeAreaProviderCompat)
    in SafeAreaProviderCompat (created by StackView)
    in RCTView (created by View)
    in View (created by GestureHandlerRootView)
    in GestureHandlerRootView (created by StackView)
    in StackView (created by StackNavigator)
    in PreventRemoveProvider (created by NavigationContent)
    in NavigationStateListenerProvider (created by NavigationContent)
    in NavigationContent
    in Unknown (created by StackNavigator)
    in StackNavigator (created by StackSymbol)
    in StackSymbol (created by SceneView)
    in StaticContainer
    in EnsureSingleNavigator (created by SceneView)
    in SceneView (created by StackNavigator)
    in RCTView (created by View)
    in View (created by CardContainerInner)
    in RCTView (created by View)
    in View (created by CardContainerInner)
    in RCTView (created by View)
    in View
    in CardSheet (created by Card)
    in RCTView (created by View)
    in View (created by Animated(View))
    in Animated(View) (created by PanGestureHandler)
    in PanGestureHandler (created by PanGestureHandler)
    in PanGestureHandler (created by Card)
    in RCTView (created by View)
    in View (created by Animated(View))
    in Animated(View) (created by Card)
    in RCTView (created by View)
    in View (created by Card)
    in Card (created by CardContainerInner)
    in CardContainerInner (created by CardStack)
    in RNSScreen (created by Animated(Anonymous))
    in Animated(Anonymous)
    in Suspender (created by Freeze)
    in Suspense (created by Freeze)
    in Freeze (created by DelayedFreeze)
    in DelayedFreeze
    in InnerScreen (created by Screen)
    in Screen (created by MaybeScreen)
    in MaybeScreen (created by CardStack)
    in RNSScreenContainer (created by ScreenContainer)
    in ScreenContainer (created by MaybeScreenContainer)
    in MaybeScreenContainer (created by CardStack)
    in RCTView (created by View)
    in View (created by CardStack)
    in CardStack (created by HeaderShownContext)
    in FrameSizeProviderInner (created by FrameSizeProvider)
    in FrameSizeProvider (created by SafeAreaProviderCompat)
    in RCTView (created by View)
    in View (created by SafeAreaProviderCompat)
    in SafeAreaProviderCompat (created by StackView)
    in RCTView (created by View)
    in View (created by GestureHandlerRootView)
    in GestureHandlerRootView (created by StackView)
    in StackView (created by StackNavigator)
    in PreventRemoveProvider (created by NavigationContent)
    in NavigationStateListenerProvider (created by NavigationContent)
    in NavigationContent
    in Unknown (created by StackNavigator)
    in StackNavigator (created by RootRoutes)
    in ThemeProvider
    in EnsureSingleNavigator
    in BaseNavigationContainer
    in NavigationContainerInner (created by RootRoutes)
    in RCTView (created by View)
    in View
    in StyledNativeComponent (created by Styled(View))
    in Styled(View) (created by RootRoutes)
    in ToastProvider (created by RootRoutes)
    in RootRoutes (created by App)
    in PortalProviderComponent (created by BottomSheetModalProviderWrapper)
    in BottomSheetModalProviderWrapper (created by App)
    in RCTView (created by View)
    in View (created by GestureHandlerRootView)
    in GestureHandlerRootView (created by App)
    in ThemeProvider (created by App)
    in RNCSafeAreaProvider (created by SafeAreaProvider)
    in SafeAreaProvider (created by App)
    in Provider (created by App)
    in App
    in RCTView (created by View)
    in View (created by AppContainer)
    in RCTView (created by View)
    in View (created by AppContainer)
    in AppContainer
    in cleanz24CustomerApp(RootComponent)

Camera Device

{
  "supportsLowLightBoost": false,
  "position": "back",
  "hardwareLevel": "full",
  "formats": [],
  "minExposure": -8,
  "hasFlash": true,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "minZoom": 1,
  "isMultiCam": false,
  "name": "Back Camera",
  "sensorOrientation": "portrait",
  "supportsRawCapture": false,
  "supportsFocus": true,
  "hasTorch": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "maxZoom": 121.875,
  "minFocusDistance": 15,
  "maxExposure": 8
}

Device

iPhone 13

VisionCamera Version

^4.7.0

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here 🍻

Thank you for your detailed report! It looks like you have provided a lot of useful information regarding the issue you are facing. However, to assist you better, could you please try reproducing this issue in the VisionCamera example app? This will help us confirm if it's a bug or something specific to your setup. Additionally, providing the relevant logs from Xcode or adb logcat related to the error will be very helpful for mrousavy to diagnose the issue.

If you need help with gathering those logs, just let me know! And remember, sponsoring the project can help get more attention to your issue. You can do that here.

Note: If you think I made a mistake, please ping @mrousavy to take a look.