A deep learning based tool for generating high-resolution PBR materials from flat diffuse textures. This generates an albedo, DirectX and OpenGL normal maps, a roughness map, and a displacement map. The tool consists of a custom hybrid transformer-convnet architecture based on SCUNet, trained on public domain and CC0 licensed PBR materials.
Paper detailing the design will be released in the future. Designed and developed in partial fulfillment of a Computer Science Integrated Masters degree.
Comparisons between 256x256 input textures and PBR materials generated from them and rendered in Blender Cycles.
comp.mp4
On an RTX 3070, the network achieves an average runtime of 5.1 seconds for 512x512 input textures, compared to ~150 seconds for NVIDIA's AI Texture tools included with RTX Remix.
Developed for Python 3.11, PyTorch 2.2, CUDA 11.6+/12.x. Tested only on Windows.
Ensure that one of the either CUDA versions and the latest NVIDIA drivers are installed. All other dependencies can be installed using the corresponding requirements_cudaXX.txt.
The GUI can be started via either python test_gui.py or test_gui.bat.
Input files can be selected from a file dialog in the GUI or drag & dropped into the GUI. All common image formats and the DDS texture format are supported, and with and without an alpha channel. Generated material can be saved using a folder dialogue in PNG format.
An additional network can be specified to upscale the diffuse texture alongside our PBR generation network to potentially further enhance the result, as long as the model is support by Spandrel. Community texture super-resolution models can be found at OpenModelDB.
- AmbientCG: CC0 PBR materials for training
- cgbookcase: CC0 PBR materials for training
- Poly Haven: CC0 PBR materials for training
- SCUNet: Main basis for the PBR generator network architecture
- Swin Transformer V2: SwinV2 modification applied to SCUNet
- ESRGAN+: Basis for the PBR super-resolution component
- Dear PyGui: GUI framework
- Crash Course in BRDF Implementation: BRDF implementation for material renderer
- Spandrel: Support for a wide range of customer upscalers
- DISTS: Perceptual loss
- Textile: Tiling loss
- A-ESRGAN: Discriminator architecture
- VSGAN: Tiled inference