It downloads all the images from a provided Username, Model ID or Model TAG from CivitAI. Should the API not spit out all the data for all images then I'm sorry. The script can only download where data is provided.
The images are Downloaded into a folder with the name of the user, ModelID or the TAG
Second Level is the Model Name with which the image was generated.
- Install Python 3 Ensure you have Python 3.8 or newer installed.
install Python3
- Install Dependencies NEW requirements for users who already use the script
pip install -r requirements.txt- Migrate Existing Data
- If you are using the previous version with a
downloaded_images.jsonfile and want to continue using your data for tracking, you must use the my toolmigrate_json_to_sqlite.pyto transfer your old data to the new database. - Make sure both
migrate_json_to_sqlite.pyand your olddownloaded_images.jsonare in the same directory. - Run the migration tool from your terminal:
python migrate_json_to_sqlite.py- Follow the prompts. It will create
tracking_database.sqliteand offer to rename your old JSON file.
Run the script without any command-line arguments:
python civit_image_downloader.pythe script will ask you to:
Enter timeout value (seconds) [default: 60]:Choose image quality (1=SD, 2=HD) [default: 1]:Allow re-downloading tracked items? (1=Yes, 2=No) [default: 2]:Choose mode (1=user, 2=model ID, 3=tag search, 4=model version ID):Enter max concurrent downloads [default: 5]:- (Mode-specific prompts):
- Mode 1:
Enter username(s) (, separated): - Mode 2:
Enter model ID(s) (numeric, , separated): - Mode 3:
Enter tags (, separated): - Mode 3:
Disable prompt check? (y/n) [default: n]:(Check if tag words must be in the image prompt) - Mode 4:
Enter model version ID(s) (numeric, , separated):
- Mode 1:
If you just hit enter it will use the Default values of that Option if it has a default value.
Provide arguments directly on the command line. Unspecified arguments will use their defaults. --mode is required.
Available Arguments
--timeout INT(Default: 60)--quality {1,2}(1=SD, 2=HD, Default: SD)--redownload {1,2}(1=Yes, 2=No, Default: 2)--mode {1,2,3,4}(Required)--tags TAGS(Comma-separated, required for Mode 3)--disable_prompt_check {y,n}(Default: n)--username USERNAMES(Comma-separated, required for Mode 1)--model_id IDS(Comma-separated, numeric, required for Mode 2)--model_version_id IDS(Comma-separated, numeric, required for Mode 4)--output_dir PATH(Default: "image_downloads")--semaphore_limit INT(Default: 5)--no_sort(Disables model subfolder sorting, Default: False/Sorting enabled)--max_path INT(Default: 240)--retries INT(Default: 2)
- Download HD images for user "artist1", allowing redownloads, higher concurrency:
python civit_image_downloader.py --mode 1 --username "artist1" --quality 2 --redownload 1 --semaphore_limit 10 - Download SD images for models 123 and 456, using defaults for other options:
python civit_image_downloader.py --mode 2 --model_id "123, 456" - Download SD images for tag "sci-fi", disabling prompt check, no redownloads:
python civit_image_downloader.py --mode 3 --tags "sci-fi" --disable_prompt_check y --redownload 2
If only some arguments are provided (e.g., only --mode), the script will use the provided options and prompt the user for any missing inputs.
The downloaded files will be organized within the specified --output_dir (default: image_downloads). Sorting (--no_sort flag) affects the structure inside the identifier folder.
With Sorting Enabled (Default)
image_downloads/
├── Username_Search/
│ └── [Username]/
│ ├── [Model Name Subfolder]/ # Based on image metadata 'Model' field
│ │ ├── [ImageID].jpeg # or .png, .webp, .mp4, .webm
│ │ └── [ImageID]_meta.txt
│ ├── invalid_metadata/ # For images with meta but no parsable 'Model' field
│ │ ├── [ImageID].jpeg
│ │ └── [ImageID]_meta.txt
│ └── no_metadata/ # For images with no metadata found
│ ├── [ImageID].jpeg
│ └── [ImageID]_no_meta.txt
├── Model_ID_Search/
│ └── model_[ModelID]/
│ ├── [Model Name Subfolder]/
│ │ └── [ImageID].jpeg
│ │ └── [ImageID]_meta.txt
│ ├── invalid_metadata/
│ │ └── [ImageID].jpeg
│ │ └── [ImageID]_meta.txt
│ └── no_metadata/
│ └── [ImageID].jpeg
│ └── [ImageID]_no_meta.txt
│
├── Model_Version_ID_Search/
│ └── modelVersion_[VersionID]/
│ ├── [Model Name Subfolder]/
│ │ └── [ImageID].jpeg
│ │ └── [ImageID]_meta.txt
│ ├── invalid_metadata/
│ │ └── [ImageID].jpeg
│ │ └── [ImageID]_meta.txt
│ └── no_metadata/
│ └── [ImageID].jpeg
│ └── [ImageID]_no_meta.txt
└── Model_Tag_Search/
└── [Sanitized_Tag_Name]/ # e.g., sci_fi_vehicle
├── model_[ModelID]/ # Folder for each model found under the tag
│ ├── [Model Name Subfolder]/ # Sorting within model folder
│ └── [ImageID].jpeg
│ └── [ImageID]_meta.txt
│ ├── invalid_metadata/
│ └── [ImageID].jpeg
│ └── [ImageID]_meta.txt
│ └── no_metadata/
│ └── [ImageID].jpeg
│ └── [ImageID]_no_meta.txt
└── summary_[Sanitized_Tag_Name]_[YYYYMMDD].csv
With Sorting Disabled (--no_sort)
All images and metadata files for a given identifier (username, model ID, model version ID, or model ID within a tag) are placed directly within that identifier's folder, without the [Model Name Subfolder], invalid_metadata, or no_metadata subdirectories.
image_downloads/
├── Username_Search/
│ └── [Username]/
│ ├── [ImageID].jpeg
│ ├── [ImageID]_meta.txt
│ └── [ImageID]_no_meta.txt
├── Model_ID_Search/
│ └── model_[ModelID]/
│ ├── [ImageID].jpeg
│ └── ...
├── Model_Version_ID_Search/
│ └── modelVersion_[VersionID]/
│ ├── [ImageID].jpeg
│ └── ...
└── Model_Tag_Search/
└── [Sanitized_Tag_Name]/
├── model_[ModelID]/
│ ├── [ImageID].jpeg
│ ├── [ImageID]_meta.txt
│ └── [ImageID]_no_meta.txt
└── summary_[Sanitized_Tag_Name]_[YYYYMMDD].csv # CSV still in tag folder
This file replaces the old JSON file. It stores a record of each downloaded image/video, including its path, quality, download date, associated tags (from Mode 3), original URL, and extracted checkpoint name (from metadata). You can explore this file using tools like "DB Browser for SQLite".
Migration Tool (migrate_json_to_sqlite.py)
If you are updating from a version using downloaded_images.json, run this separate Python script once in the same directory as your JSON file before using the main downloader. It will read the JSON and populate the new tracking_database.sqlite file.
python migrate_json_to_sqlite.py-
Code Structure (Major Refactoring):
The entire script has been refactored into an object-oriented structure using theCivitaiDownloaderclass. This encapsulates state (configuration,tracking data, statistics) and logic (downloading, sorting, API interaction) within the class, eliminating reliance on global variables. -
Scalable Tracking (SQLite Migration):
Replaced JSON: The previousdownloaded_images.jsontracking file has been replaced with an SQLite database (tracking_database.sqlite).
Relational Tags: Image tags (for Mode 3 summaries) are now stored relationally in a separateimage_tagstable, linked to the maintracked_imagestable. This allows for efficient querying.
Migration: A separatemigrate_json_to_sqlite.pyscript is provided for users to perform a one-time migration of their existingdownloaded_images.jsondata into the new SQLite database format. -
Robust Error Handling & Retries:
Automatic Retries: Integrated thetenacitylibrary to automatically retry failed network operations (image downloads, API page fetches, model searches) caused by common transient issues like timeouts, connection errors, or specific server-side errors (500, 502, 503, 504).
File Operations: Implemented a_safe_movefunction with retries to handle potential file locking issues during sorting (especially on Windows). Added checks to verify move operations. -
Improved Tag Search (Mode 3) Validation:
Invalid Tag Detection: When searching by tag, the script now fetches the first page of results and checks if any of the returned models actually contain the searched tag in their own metadata tags. -
Detailed Per-Identifier Statistics:
Granular Reporting: The final statistics summary now provides a detailed breakdown for each identifier (username, model ID, tag, version ID) processed during the run. -
Improved User Interface & Experience:
Input Validation: Added/improved validation loops for interactive inputs (e.g., ensuring numeric IDs, positive numbers). Handles invalid CLI arguments more gracefully (logs errors, exits).
Clearer Output: Refined console and log messages. Added specific warnings for invalid tags or identifiers that yield no results. Reduced console noise by logging successful per-file downloads only at the DEBUG level. Added a final summary note listing identifiers that resulted in no downloads.
This update introduces support for three different startup modes.
Fully Interactive Mode: If no command-line arguments are provided, the script will prompt the user for all required inputs interactively, as before.
Fully Command-Line Mode: If all necessary arguments are supplied via the command line, the script will execute without any prompts, offering a streamlined experience for advanced users.
Mixed Mode: If only some arguments are provided, the script will use the provided options and prompt the user for any missing inputs. This allows for a flexible combination of both modes.
The new Feature includes a check for mismatched arguments. If you provide arguments that don't match the selected mode, you will receive a warning message, but the script will continue to run,
ignoring the mismatched arguments and prompting for the required information if necessary.
Warning: --Argument is not used in ... mode. This argument will be ignored.
All images with no_meta_data are now moved to their own folder named no_meta_data.
They also have a text file containing the URL of the image, rather than any metadata.
No metadata available for this image.
URL: https://civitai.com/images/ID?period=AllTime&periodMode=published&sort=Newest&view=feed&username=Username&withTags=false
A bug was fixed where the script sometimes did not download all the images provided by the API.
The logging function was also enhanced. You can now see how many image links the API provided and what the script has downloaded.
A short version is displayed in your terminal.
Number of downloaded images: 2
Number of skipped images: 0
While more detailed information is available in the log file.
Date Time - INFO - Running in interactive mode
Date Time - WARNING - Invalid timeout value. Using default value of 60 seconds.
Date Time - WARNING - Invalid quality choice. Using default quality SD.
Date Time - INFO - Received 2 items from API for username Example
Date Time - INFO - Attempting to download: https://image.civitai.com/247f/width=896/b7354672247f.jpeg
Date Time - INFO - Attempting to download: https://image.civitai.com/db84/width=1024/45757467b84.jpeg
Date Time - INFO - Successfully downloaded: image_downloads/Username_Search/Example/2108516.jpeg
Date Time - INFO - Successfully downloaded: image_downloads/Username_Search/Example/2116132.jpeg
Date Time - INFO - Marked as downloaded: 21808516 at image_downloads/Username_Search/Example/2108516.jpeg
Date Time - INFO - Marked as downloaded: 21516132 at image_downloads/Username_Search/Example/2116132.jpeg
Date Time - INFO - Total items from API: 2, Total downloaded: 2
Date Time - INFO - 2 images have no meta data.
Date Time - INFO - Total API items: 2, Total downloaded: 2
Date Time - INFO - Image download completed.
The script can now selectively download images that belong to a specific model version ID. Option 4
This saves disk space and in addition, the Civit AI Server API is used less, which leads to a more efficient use of resources.
The Script will download the Images to this new Folder --> Model_Version_ID_Search
Updated the Folder Structure
i have noticed that the timeout of 20 seconds is too short for model ID and version ID and that i get more network errors than downloads,
so i have set it to 60 seconds for now.
But if you want to be on the safe side, then enter the following: 120 for the option: Enter timeout value (in seconds):
this has always worked up to now
Updated Folder Structure.
The script creates a Folder for each Option you can choose.
This new structure ensures better organization based on the search type, making image management more efficient.
New Feature
Redownload of images. The new option allows the tracking file to be switched off. So that already downloaded images can be downloaded again.
Allow re-downloading of images already tracked (1 for Yes, 2 for No) [default: 2]:
If you choose 2 or just hit enter the Script will run with Tracking as Default like always.
New Update
When the script is finished, a summary of the usernames or Model IDs that could not be found is displayed.
Failed identifiers:
username: 19wer244rew
Failed identifiers:
ModelID: 493533
With this Script you can search locally in txt a file if your TAG is searchable.
Just launch tagnames.py and it creates a txt File with all the Tags that the API gives out for the Model TAG search Option 3
But there are some entrys that are cleary not working. I dont kow why they are in the API Answer.
It has an function to add only new TAGS to he txt File if you run it again.
Features:
Model Tag Based Image download in SD or HD with Prompt Check Yes or NO
Prompt Check YES means when the TAG is also present in the Prompt, then the image will be Downloaded. Otherwise it will be skipped.
Prompt Check NO all Images with the searched TAG will be Downloaded. But the chance for unrelated Images is higher.
CSV File creation within Option 3 TAG Seach
The csv file will contain the image data that, according to the JSON file, has already been downloaded under a different TAG in this format:
"Current Tag,Previously Downloaded Tag,Image Path,Download URL"
Litte Statistc how many images have just been downloaded and skipped with a why reasons.
Updates:
Use of Multiple Entrys in all 3 Options comma-separated
New Folder Structure for Downloaded Images in all Options First Folder is named after what you searched Username, ModelID, TAG. Second is the Model that was used to generate the image
Performance:
Code optimizations now the script runs smoother and faster.
Better Error Handling for some Cases
Rate Limiting set to 20 simultaneous connections. Download Date Format changend in the JSON Tracking File
Option for Downloading SD (jpeg) Low Quality or HD (PNG) High Quality Version of Images
Better Tracking of images that already downloaded, with a JSON File called downloaded_images.json in the same Folder as the script. The Scripts writes for SD Images with jpeg Ending
"ImageID_SD":
"path": "image_downloads/civitAIuser/image.jpeg",
"quality": "SD",
"download_date": "YYYY-MM-DD - H:M"
For HD Images with PNG Ending
"ImageID_HD": {
"path": "image_downloads/civitAIuser/Image.png",
"quality": "HD",
"download_date": "YYYY-MM-DD- H:M"
into it and checks before Downloading a Image. For Both Option, Model ID or Username
Image Download with Model ID. Idea for it came from bbdbby The outcome looks sometimes chaotic a lot of folders with Modelnames you cant find on CivitAI. Because of renaming or Deleting the Models. But older Images have the old Model Names in the META data.
Sort function to put the images and meta txt files into the right Model Folder. The sort Function relies on the Meta Data from the API for the images. Sometimes Chaos. Especially for models that have a lot of images.
Tracking of images that already downloaded with a text file called downloaded_images.txt in the same Folder as the script. The Scripts writes the Image ID into it and checks before Downloading a Image. For Both Option, Model ID or Username
Increased the timeout to 20
It is writing the Meta Data for every image into a separate text file with the ID of the image: ID_meta.txt. If no Meta Data is available, the text file will have the URL to the image to check on the website.
Increased the timeout to 10
Added a delay between requests