Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion

Anle Ke¹   ·   Xu Zhang¹   ·   Tong Chen¹   ·   Ming Lu¹  ·   Chao Zhou²  ·   Jiawen Gu²  ·   Zhan Ma¹   

¹ Nanjing University   ²Kuaishou Technology 

🌐 Project Page   📃 Paper  

📖 Table Of Contents

• ✨ Visual Results
• ⏳ Train
• 😀 Inference
• 🌊 TODO
• ❤ Acknowledgement
• 🙇‍ Citation

⚙️ Environment Setup

- conda create -n ResULIC python=3.10
- conda activate ResULIC
- pip install -r requirements.txt

✨ Visual Results

⏳ Train

Note: The numbers in the yaml filenames (e.g., 1_1_1) represent $\lambda_{\text{diffusion}}$, $\lambda_{\text{mse}}$, and $\lambda_{\text{bpp}}$ respectively.

Stage 1: Initial Training

1. Download Pretrained Model
   Download the pretrained Stable Diffusion v2.1 model into the ./weight directory:

   wget https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt --no-check-certificate -P ./weight
   

2. Modify the configuration file./configs/train_zc_eps.yaml and ./configs/model/stage1/xx.yaml accordingly.

3. Start training.

   bash stage1.sh 
   

Stage 2:

1. Modify the configuration file ./configs/train_stage2.yaml and ./configs/model/stage2/xx.yaml accordingly.

2. Start training.

   bash stage2.py 
   

😀 Inference

Note: It is recommended to set "ddim_steps" to a number that is divisible by "add_steps". For example, when add_steps=600, ddim_steps could be 2, 3, 5...

1. W/o Srr, W/o Pfo.

   CUDA_VISIBLE_DEVICES=2 python inference_win.py \
    --ckpt xx \
    --config /xx/xx.yaml \
    --output xx/ \
    --ddim_steps 3 \
    --ddim_eta 0 \
    --Q x.0 \
    --add_steps x00
   

2. W/ Srr, W/o Pfo.

    CUDA_VISIBLE_DEVICES=2 python inference_res.py \
    --ckpt xx \
    --config /xx/xx.yaml \
    --output xx/ \
    --ddim_steps 3 \
    --ddim_eta 0 \
    --Q x.0 \
    --add_steps x00
   

🌊 TODO

• Release code
• Release quantitative metrics （👾The quantitative metrics for ResULIC presented in our paper can be found in indicator.）
• Release pretrained models (Coming soon)

❤ Acknowledgement

This work is based on ControlNet, ControlNet-XS, DiffEIC, and ELIC, thanks to their invaluable contributions.

🙇‍ Citation

If you find our work useful, please consider citing:

@inproceedings{Ke2025resulic,
               author = {Ke, Anle and Zhang, Xu and Chen, Tong and Lu, Ming and Zhou, Chao and Gu, Jiawen and Ma, Zhan},
               title = {Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion},
               booktitle = {International Conference on Machine Learning},
               year = {2025}
               }