babylon-mmd

Babylon.js loader and runtime for MikuMikuDance (MMD) models and animations. It supports PMX/PMD model formats, VMD/VPD animation formats, and provides a runtime for physics, IK, morphs, and more.

Playground Demo

Credits

Music: メランコリ・ナイト

Model: YYB Hatsune Miku_10th

Motion / Camera: https://www.nicovideo.jp/watch/sm41164308

2025-07-21.06-01-26.mp4

Credits

Music: ピチカートドロップス

Model: YYB Piano dress Miku

Stage: ガラス片ドーム

Motion: https://www.nicovideo.jp/watch/sm31508557

Camera: https://www.nicovideo.jp/watch/sm36273873

Documentation

Please refer to the documentation for detailed instructions on how to use it.

Usage

Here is how to load and animate an MMD model using babylon-mmd. For more details, please refer to the Documentation.

// side effects that register the loader
import "babylon-mmd/esm/Loader/pmxLoader";

// side effects that register the animation runtime
import "babylon-mmd/esm/Runtime/Animation/mmdRuntimeCameraAnimation";
import "babylon-mmd/esm/Runtime/Animation/mmdRuntimeModelAnimation";

async function build(canvas: HTMLCanvasElement, engine: Engine): Scene {
    // If you don't want full SDEF support on shadow / depth rendering, you can comment out this line as well. While using SDEF can provide similar results to MMD, it comes with a higher cost.
    SdefInjector.OverrideEngineCreateEffect(engine);

    const scene = new Scene(engine);
    scene.ambientColor = new Color3(0.5, 0.5, 0.5);

    const camera = new MmdCamera("mmdCamera", new Vector3(0, 10, 0), scene);

    const hemisphericLight = new HemisphericLight("light1", new Vector3(0, 1, 0), scene);
    hemisphericLight.intensity = 0.3;
    hemisphericLight.specular = new Color3(0, 0, 0);
    hemisphericLight.groundColor = new Color3(1, 1, 1);

    const directionalLight = new DirectionalLight("DirectionalLight", new Vector3(0.5, -1, 1), scene);
    directionalLight.intensity = 0.7;
    
    CreateGround("ground1", { width: 60, height: 60, subdivisions: 2, updatable: false }, scene);
    
    const mmdWasmInstance = await GetMmdWasmInstance(new MmdWasmInstanceTypeMPR());
    const physicsRuntime = new MultiPhysicsRuntime(mmdWasmInstance);
    physicsRuntime.setGravity(new Vector3(0, -98, 0));
    physicsRuntime.register(scene);
    
    // MMD runtime for solving morph, append transform, IK, animation, physics
    const mmdRuntime = new MmdRuntime(scene, new MmdBulletPhysics(physicsRuntime));
    mmdRuntime.register(scene);
    
    // For synced audio playback
    const audioPlayer = new StreamAudioPlayer(scene);
    audioPlayer.source = "your_audio_path.mp3";
    mmdRuntime.setAudioPlayer(audioPlayer);
    
    // You can also run the animation before it loads. This will allow the audio to run first.
    mmdRuntime.playAnimation();

    // create a youtube-like player control
    new MmdPlayerControl(scene, mmdRuntime, audioPlayer);
    
    const vmdLoader = new VmdLoader(scene);

    const cameraAnimation = await vmdLoader.loadAsync("camera_motion", "your_camera_motion_path.vmd");
    const cameraRuntimeAnimationHandle = camera.createRuntimeAnimation(cameraAnimation);
    camera.setRuntimeAnimation(cameraRuntimeAnimationHandle);
    mmdRuntime.addAnimatable(camera);

    const assetContainer = await LoadAssetContainerAsync("path/to/your_file.pmx", scene);
    assetContainer.addAllToScene();
    const mmdMesh = assetContainer.meshes[0] as MmdMesh;

    const mmdModel = mmdRuntime.createMmdModel(mmdMesh, {
        materialProxyConstructor: MmdStandardMaterialProxy
    });
    const modelMotion = await vmdLoader.loadAsync("model_motion", "your_model_motion_path.vmd");
    const modelRuntimeAnimationHandle = mmdModel.createRuntimeAnimation(modelMotion);
    mmdModel.setRuntimeAnimation(modelRuntimeAnimationHandle);

    return scene;
}

Implementation status

Parser

• PMX Parser
• PMD Parser
• VMD Parser
• VPD Parser
• Optimized Custom PMX like format
• Optimized Custom VMD like format

PMX / PMD Loader

Mesh

• Geometry load
• Build Bone / MorphTarget

Material / Shading

• Basic material parameters load
• Custom MMD toon fragment shader
• Spherical Deformation support
• WebGL 1.0 / 2.0 support
• WebGPU support

VMD / VPD Loader

• Basic animation load / custom retargeting
• Load animation using the Babylon.js Animation container
• Babylon.js Animation Humanoid to MMD bone retargeting

Animation Runtime

• Basic animation load
• MMD morph system support
• Solve Append transform
• Solve IK
• Play audio / sync with animation
• Basic animation player UI
• Animation blending in custom animation runtime
• Force Humanoid Model Support

Physics Runtime

• Solve Rigid body / Joint
• WASM based runtime for threaded bullet physics and MMD IK

Not planned features

• PMX 2.1 support (because 2.1 spec not implemented in MMD)
• Self shadow, Ground shadow spec support

Support this project

Advanced technical support is available for those supporting the project

Contact me via email or discord

email: hjnam2014@gmail.com

discord: noname0310

References

PMX (Polygon Model eXtended) 2.0, 2.1 File Format Specifications

blender_mmd_tools

Saba: OpenGL Viewer (OBJ PMD PMX)

three.js MMDLoader