This repository provides the code for the Master's Thesis "Gaussian Splatting Regularization for Improved Out-of-Distribution View Synthesis". A link to the paper will be provided soon!
OOD-Gaussian-Splatting-Demo.mp4
First, install all the dependencies and install the python package.
# First, create conda enviroment
conda create -n oodgs python=3.11 -y
conda activate oodgs
# Assumes CUDA 12.6
./setup.sh
To generate the error map using Met3r loss, use:
python render_met3r.py
To generate the error map using Depth MVC loss, use:
python render_depth_mvc.py
For benchmarking the inference speed (avg forward pass time), use:
python benchmark_inference_speed.py {met3r | depth_mvc} --runs <N> --image-size <H>
For benchmarking the VRAM usage, use:
python benchmark_vram_usage.py {met3r | depth_mvc} --image-size <H>
This project tries to improve the quality of Out-Of-Distribution (OOD) rendering in Gaussian Splatting models. The approach we chose is to regularize the Gaussian optimization by adding OOD regularization in the training pipeline. The illustration of our approach is shown in the image:
Two OOD Regularization methods were proposed: Met3r Loss and Depth MVC Loss. Met3r loss is a variation of the Met3r metric proposed in the https://arxiv.org/html/2501.06336v1, adapted and improved for this application. It tries to capture the discontinuity in features, while the Depth MVC loss tries to capture differences in the geometry between train and OOD views.