Latent Self

An interactive face-morphing mirror installation.

Overview

Latent Self is an interactive art installation that uses a webcam to capture a user's face and then applies a series of transformations to it in real-time. The transformed image is then displayed on a screen, creating a "latent self" of the user.

Features

• Real-time face morphing
• Multiple transformation axes (age, gender, smile, species, beauty)
• Emotion bank with six presets (happy, angry, sad, fear, disgust, surprise)
• Adjustable blend weights for each axis
• Fullscreen kiosk mode (--kiosk)
• Admin panel for on-site configuration
• MQTT heartbeat for remote monitoring
• Typed configuration via pydantic-settings
• Demo mode with prerecorded media (--demo)
• Optional live memory usage readout in the admin panel
• Periodic logging of average FPS and latency metrics

Installation

1. Clone the repository:

   git clone https://github.com/your-username/latent-self.git

2. Install the dependencies:

   pip install -r requirements.lock

3. Download the model weights (see below).

Model Weights

This project requires the following model weights:

• ffhq-1024-stylegan2.pkl – StyleGAN‑ADA generator
• e4e_ffhq_encode.pt – e4e encoder
• latent_directions.npz – W⁺ latent directions (age, gender, smile)

Place these files in a models directory in the project root.

To speed up inference you can convert the generator and encoder to ONNX or TensorRT:

python scripts/convert_models.py --weights models --out models --tensorrt

ModelManager will automatically load *.onnx or *.engine files if present.

Admin Password

Generate a hashed password for the admin panel:

python scripts/generate_password_hash.py mysecret

Copy the printed hash into your config.yaml under the admin_password_hash field. You can also set an admin_api_token value for header-based access.

Remote Admin API

Start the lightweight HTTP server with --web-admin to manage configuration over the local network:

python latent_self.py --web-admin

The server listens on port 8001 by default and exposes three endpoints:

• GET /config – return the current configuration as JSON
• POST /config – update configuration fields (JSON body)
• POST /reload – reload configuration from disk

If admin_password_hash is set in config.yaml, Basic Auth credentials are required to access these routes. You may also set admin_api_token and send it in the X-Admin-Token header with each request.

OSC Control

Enable the OSC server by setting osc.enabled: true in config.yaml. It listens on port 9000 by default and accepts the following messages:

• /direction – switch to a specific direction (e.g. AGE, SMILE)
• /blend/<name> – set blend weight for a direction (0.0–1.0)
• /cycle_duration – update morph cycle length in seconds

Example using oscsend:

oscsend localhost 9000 /direction s "SMILE"
oscsend localhost 9000 /blend/age f 0.5

Usage

python latent_self.py
python latent_self.py --ui qt --kiosk  # Qt fullscreen
python latent_self.py --demo           # Use prerecorded media

Controls

Keyboard shortcuts when running with the default OpenCV UI:

q - quit
y - age
g - gender
h - smile
e - ethnicity
s - species
u - beauty
1 - happy
2 - angry
3 - sad
4 - fear
5 - disgust
6 - surprise
b - blended morph

Demo

(If the image fails to load, run python scripts/decode_images.py to regenerate the PNG/GIF files.)

To try the application without a webcam, place a demo.mp4 file or a folder of images inside the data/ directory and run with --demo.

For additional options run:

python latent_self.py -h

See the User Manual for detailed setup and the Troubleshooting Guide for common issues.

Metrics Output

The application logs average FPS and frame latency every few seconds. Adjust metrics_interval in config.yaml to control how often these statistics are emitted.

(If the image is missing, run python scripts/decode_images.py or python scripts/capture_screenshots.py.)

Building the Documentation

The docs site is built with MkDocs. To preview it locally:

python scripts/decode_images.py  # restore PNG/GIF assets
mkdocs serve

The site is localized using the mkdocs-static-i18n plugin. A language switcher appears in the navigation when multiple translations are present.

Deploying Versioned Docs

Tag your release and run:

scripts/deploy_docs.sh

This publishes the documentation for the tagged version and updates the latest and stable aliases using the mike plugin.

Continuous Deployment

Documentation changes merged to main automatically trigger a GitHub Actions workflow that builds all docs sites and publishes them to GitHub Pages.