Testing BoxDiff for my master thesis

BoxDiff 🎨 (ICCV 2023)

BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion

Jinheng Xie¹  Yuexiang Li²  Yawen Huang²  Haozhe Liu^2,3  Wentian Zhang² Yefeng Zheng²  Mike Zheng Shou¹

¹ National University of Singapore  ² Tencent Jarvis Lab  ³ KAUST

Quick start

conda create --name boxdiff python=3.10
conda activate boxdiff
pip install -r requirements.txt

Image generation

The .csv file containing the prompts should be inside a folder named prompts that is posiotioned in the root of the project.

The .csv file used is expected to have the following structure (no limits in the number of objects): id,prompt,obj1,bbox1,obj2,bbox2,obj3,bbox3,obj4,bbox4

Citation

@InProceedings{Xie_2023_ICCV,
    author    = {Xie, Jinheng and Li, Yuexiang and Huang, Yawen and Liu, Haozhe and Zhang, Wentian and Zheng, Yefeng and Shou, Mike Zheng},
    title     = {BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2023},
    pages     = {7452-7461}
}

Acknowledgment - the code is highly based on the repository of diffusers, google, and yuval-alaluf.