SUSHI-GGUF

A minimalist framework to convert and quantize SDXL models to GGUF format

Overview

SUSHI-GGUF is a tool that helps you:

• Extract UNet, CLIP, and VAE components from SDXL models
• Convert UNet to GGUF format for optimized inference
• Create quantized versions with reduced precision

Setup

git clone --recursive https://github.com/nerdlab53/sushi-gguf.git
cd sushi-gguf
conda create -n sushi-gguf python=3.10
conda activate sushi-gguf
pip install -r requirements.txt

Usage

Convert local model

python main.py --model_path /path/to/your/model.safetensors --output_dir ./output

Download and convert from CivitAI

python main.py --civitai --model_name "my_model" --model_version_id "12345" --civitai_token "YOUR_TOKEN"

Specify quantization types

python main.py --model_path /path/to/model.safetensors --quant_types Q5_K_S Q8_0

Features

• Extract model components
• Convert to GGUF format
• Quantize to various precision levels (Q4_K_S, Q5_K_S, Q8_0)
• Download models directly from CivitAI

Requirements

• Python 3.8+
• safetensors
• gguf
• rich (for terminal UI)
• wget (for downloads)

Credits

• LLama.cpp
• CivitAI
• Rich
• Wget
• old_fisherman at civitai
• bluepencil-XL at civitai