FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions

CVPR 2024

Zhen Liu^1,*, Hao Zhu^1,*, Qi Zhang², Jingde Fu¹, Weibing Deng¹, Zhan Ma¹, Yanwen Guo¹, Xun Cao¹,

¹Nanjing University, ²Tencent AI Lab, ^*Equal contibution

Project Page | Paper

We introduce a novel implicit neural representation that allows for flexible tuning of the spectral bias, enhancing signal representation and optimization. 🚀

This repository provides the code for several applications:

• Image Fitting: Demonstrates the model's ability to represent 2D images.
• SDF Fitting & Evaluation: Includes code for fitting signed distance functions and evaluating the results, based on the Bacon repository.
• NeRF Implementation: Our NeRF experiments are built upon the torch-ngp codebase.
• NTK Visualization: We utilize code from inr_dictionaries to visualize the neural tangent kernel, offering insights into the model's behavior.

Setup

conda create -n finer python=3.8
conda activate finer
pip install -r requirements.txt

Training

Image Fitting

bash run_finer.sh 
# run_siren.sh; run_pemlp.sh; run_gauss.sh; run_wire.sh

SDF Fitting & Evaluation

Setup a conda environment based on Bacon and run download_datasets.py to download datasets.

cd sdf/bacon/experiments
conda activate bacon

## train 
bash run_paper_finer.sh # siren, wire, guass, wire-finer, guass-finer

## evaluation
python eval.py

NTK Visualization

Setup a conda environment based on inr_dictionaries.

cd ntk
run ntk.ipynb

Neurons Visualizations

• The outputs of the first layer neurons

Citation

@inproceedings{liu2024finer,
    title = {FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions},
    author = {Liu, Zhen and Zhu, Hao and Zhang, Qi and Fu, Jingde and Deng, Weibing and Ma, Zhan and Guo, Yanwen and Cao, Xun},
    booktitle = {Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)},
    year = {2024}
}