lala-companion

3D personified desktop assistants, tuned for you, powered by AI vision and voice.

Early access in-development.

Features

• 3D VRM react-three-fiber avatars.
• Always on microphone, voice to voice AI conversations.
• Screen vision with GPT4-V. Self operating computer functions.
• Install Lala Companion on Linux, Windows or Mac.
• Resizable transparent overlay frame, always on-top AI interface for desktops.

Join Lalaland

Discord | X

Examples:

demo.mp4

Dev Local Setup

Copy .env.example to .env and fill in the OPENAI_API_KEY.

npm i

NODE_OPTIONS="--no-experimental-strip-types" npm run start

Dev Notes

This Electron React app consists of 2 main renderers or "views" that talk to eachother through IPC. The reason being, is its hard to build nice GUIs in an also fully transparent, click through, focus-less threejs 3D scene.

As a rule, no buttons or 2D layers go in the 3D overlay. Only threejs 3D. Vanilla UIs go in "main". It ruins the vibe to mix.