Bayesian Inverse Graphics (BIG)

This repository contains the code for the paper "Bayesian Inverse Graphics for Few-Shot Concept Learning"

TLDR: probabilistic programming + differentiable rendering = minimal-data learning

Citation

Here are the links for the preprint version https://arxiv.org/abs/2409.08351 and the NeSy springer version.

@inproceedings{arriaga2024bayesian,
  title={Bayesian Inverse Graphics for Few-Shot Concept Learning},
  author={Arriaga, Octavio and Guo, Jichen and Adam, Rebecca and Houben, Sebastian and Kirchner, Frank},
  booktitle={International Conference on Neural-Symbolic Learning and Reasoning},
  pages={141--165},
  year={2024},
  organization={Springer}
}

Poster

Modules

All modules are implemented in jax

• jaynes Probabilistic Programming Library (Automatic Bayesian Inference).
• tamayo Differentiable Rendering Library.
• lecun Convnets.

Run

Setup

0. Install requirements e.g. pip install -r requirements.txt
1. Download the datasets (fscvlr.zip) and weights (VGG16.eqx) from here.
2. Move fsclvr.zip inside repository bayesian-inverse-graphics/.
3. Move VGG16.eqx inside repository bayesian-inverse-graphics/.
4. Extract datasets unzip fsclvr.zip

Training

5. Run python optimize_scene.py
6. Run python extract_features.py
7. Run python optimize_bijectors.py

Test

8. Run python learn_concept.py --concept 0

Funding

This project was developed in the Robotics Group of the University of Bremen, together with the Robotics Innovation Center of the German Research Center for Artificial Intelligence (DFKI) in Bremen. It has been funded by the German Federal Ministry for Economic Affairs and Energy and the German Aerospace Center (DLR), in the PhysWM project.