Skip to content

Using a USDZ file to full body motion #243

@pedrointercap

Description

@pedrointercap

Hi there!

I'm struggling in rendering an usdz file for a full body motion capture.

The code below runs, but the object is not rendered in the screen.

Any clues on how to solve this?

import 'package:flutter/material.dart';
import 'package:arkit_plugin/arkit_plugin.dart';
import 'package:vector_math/vector_math_64.dart';

class BodyTrackingScreen extends StatefulWidget {
  @override
  _BodyTrackingScreenState createState() => _BodyTrackingScreenState();
}

class _BodyTrackingScreenState extends State<BodyTrackingScreen> {
  late ARKitController arkitController;
  ARKitNode? avatarNode;

  @override
  void dispose() {
    arkitController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: ARKitSceneView(
        configuration: ARKitConfiguration.bodyTracking,
        onARKitViewCreated: onARKitViewCreated,
      ),
    );
  }

  void onARKitViewCreated(ARKitController controller) {
    arkitController = controller;

    // Load the USDZ avatar
    final avatar = ARKitReferenceNode(
      url: 'assets/models/avatar.usdz',
      scale: Vector3(0.1, 0.1, 0.1), // Adjust scale as needed
    );

    arkitController.add(avatar);
    avatarNode = avatar;

    // Listen for body tracking updates
    arkitController.onNodeUpdate = (update) {
      if (update.node.name == "bodyAnchor" && avatarNode != null) {
        updateAvatarPose(update.node.transform);
      }
    };
  }

  void updateAvatarPose(Matrix4 bodyTransform) {
    // Map ARKit's body joints to your avatar's skeleton
    // This requires your USDZ skeleton to match ARKit's joint hierarchy
    if (avatarNode != null) {
      avatarNode!.transform = bodyTransform;
    }
  }
}

Besides, my pubspec.yaml allows access to the assets like this:

  assets:
    - assets/models/

My final goal is this: https://developer.apple.com/documentation/arkit/capturing-body-motion-in-3d

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions