|
1 |
| -# ProjectBabble |
2 |
| -A Mouth tracking system for VR, 100% Opensource! |
3 | 1 |
|
4 |
| -Currently the architecture is a modified efficientnetv2-b0 training on blenshapes from ARKit! The model outputs 45 blenshapes of which most cross over with SRanipals blendshapes. |
| 2 | + |
5 | 3 |
|
6 |
| -### How to run |
7 |
| -Go to the releases tab, download the latest exe. Clone this repo, and then move the calibration file (calib.json) to the same location as the exe. Next get the .onnx model in the repo and move it to the same location as the exe. Run the exe and enjoy! |
| 4 | +# Table of Contents |
| 5 | +- [What is Babble?](#what-is-babble) |
| 6 | +- [Features](#features) |
| 7 | +- [How do I set it up?](#setup-more-detailed-wiki-and-setup-video-coming-soon) |
| 8 | +- [Useful Links](#useful-links) |
| 9 | + |
8 | 10 |
|
9 |
| -### PLEASE HIT THE TURN OFF CALIBRATION BUTTON UNTILL FURTHER NOTICE |
10 |
| -It ruins the model's quality |
| 11 | +## What is Babble? |
11 | 12 |
|
12 |
| -### Warranty |
13 |
| -Lol |
| 13 | +<p align="center"> |
| 14 | +Babble is an opensource mouth tracking project designed to work with any existing VR headset. We strive to make our models robust to different lighting, cameras, image qualities, and facial structures! |
| 15 | +</p> |
14 | 16 |
|
15 |
| -### Credit |
| 17 | +## Features |
| 18 | +- 100% Opensource! 🌟 |
| 19 | +- Fast and robust! 🚀 |
| 20 | +- Works with most existing blendshape standards! ⚙️ |
| 21 | +- Constantly updated and modified! 🔧 |
16 | 22 |
|
17 |
| -Thank you to dfgHiatus#7426 for providing the unity demo as well as the intergrations with other apps! |
| 23 | +## Setup (More detailed wiki and setup video coming soon!) |
| 24 | +To install babble is fairly simple! Head over to the releases tab and download the EXE located there. Run the EXE and install babble! After that, you can test with a USB WEBCAM by doing the following steps: |
18 | 25 |
|
19 |
| -Thank you to Rames The Generic#3540 who made the dataset, tested and evaluated the performence for the models, and for being awesome! |
| 26 | +- Run the babble app |
| 27 | +- Enter 0 into the camera address bar |
| 28 | +- Enter cropping mode where your camera feed should be |
| 29 | +- Crop your mouth out of the frame |
| 30 | +- Open VRCFT |
| 31 | +- Install the babble module |
| 32 | +- Test in VRC with a VRCFT-compatible avatar |
| 33 | + |
| 34 | + |
| 35 | +## Useful links |
| 36 | +- [Our Discord!](https://discord.gg/XAMZmjBktk) |
| 37 | +- [Wandb Runs](https://wandb.ai/summerai/ProjectBabble) |
20 | 38 |
|
21 |
| -Copyright (c) 2022 Sameer Suri |
|
0 commit comments