Overview

ImageForgeryDetect is an image manipulation detection system developed to address the challenges of copy-move and splicing forgeries. This project aimed to create a robust system that accurately identifies manipulated images using deep learning, image processing algorithms, and relevant methodologies.

Problem Statement

The main challenge was to detect two common types of forgeries: copy-move and splicing. Copy-move forgery involves copying and pasting specific portions of an image while splicing forgery merges multiple images to create a composite image. Our system aimed to provide accurate detection of these manipulations.

Approach

We adopted a multi-model approach to tackle the problem. Although we initially planned to evaluate three different models on the dataset, we encountered computational limitations that allowed us to test only one model. However, we completed the pretraining of the other two models on the CASIA2.0_revised dataset for future evaluation.

The chosen model was a machine-learning architecture that leverages Support Vector Machine (SVM) and supervised learning algorithms. Our modelled SVM aims to discover the optimal boundary (or hyperplane) between distinct data classes. For this reason, SVMs are successful in high-dimensional spaces and remain effective until the number of dimensions exceeds the number of samples. We subsequently fine-tuned the pre-trained model on our curated dataset to enhance its performance in detecting image manipulations.

Dataset

To ensure comprehensive evaluation, we have meticulously curated a dataset comprising three classes: authentic, copy-moved, and spliced images. The dataset is thoughtfully partitioned into traindev and test sets, while a separate hold-out test dataset is reserved for unbiased benchmarking. Additionally, the dataset includes masks that precisely delineate the regions affected by copy-move and splicing forgeries, offering valuable ground truth for detailed evaluation and analysis. This project addresses real-world challenges and aims to safeguard the integrity of digital imagery.

Challenges and Solutions

• Computational limitations: We could only test one model on the dataset due to limited computational resources. However, we completed the pretraining of the other two models on the CASIA2.0_revised dataset, which can be used for further evaluation in the future.
• Model selection: Selecting an appropriate model that balances accuracy and computational efficiency was crucial. We opted for a pre-trained deep learning model, demonstrating promising results in prior image manipulation detection tasks.
• Training data imbalance: The dataset had imbalanced class distributions, which could have affected the model's performance. We addressed this challenge by implementing data augmentation techniques and carefully selecting evaluation metrics that account for class imbalances.
• Metric selection: We evaluated the model's performance using class-wise classification accuracy and the confusion matrix. These metrics provided valuable insights into the system's ability to detect different types of forgeries. Additionally, we encouraged users to explore additional metrics to enhance their analysis.

Results

Unfortunately, we could only report the results for the model we tested on the dataset. The model achieved an overall accuracy of 87% on the test dataset. Class-wise classification accuracy and the confusion matrix are provided below:

• Authentic: 92% accuracy

Classwise-classification metrics:

• True Positive Rate: 43% accuracy
• True Negative Rate: 78% accuracy
• False Positive Rate: 22% accuracy
• False Negative Rate: 57% accuracy

Confusion Matrix:

                 | Predicted Authentic | Predicted Copy-moved | Predicted Spliced |
|----------------|--------------------|-----------------------|-------------------|
| Authentic      |       92%          |          8%           |       10%         |
| Copy-moved     |        4%          |         43%           |       16%         |
| Spliced        |        9%          |         15%           |       78%         |

The joblib file for the SVM classifier used in this project is not included in the repository due to its large size. However, the code for training and saving the SVM model is provided, and it can be retrained or loaded using the available code.

Implementation

Requirements:

• Python 3.7
• PyTorch 1.0+
• CUDA 10.0+

./unet/unet-parts.py': it includes detailed implementations of 'U-Net', 'RU-Net' and 'RRU-Net'

'train.py': you can use it to train your model

Using:

$ python train.py

'predict.py': you can use it to test the model.

But change the local machine's image, masks, and log directory.

This model is forked from:

@inproceedings{bi2019rru,
  title={RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection},
  author={Bi, Xiuli and Wei, Yang and Xiao, Bin and Li, Weisheng},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
  pages={0--0},
  year={2019}
}

All other files are in a .ipynb format, so they can be run efficiently for which requirements are given in the requirements.txt file.

Conclusion

According to the provided metrics, ImageForgeryDetect demonstrates a reliable and effective solution for detecting copy-move and splicing image forgeries. The model achieved an overall accuracy of 92% on the test dataset. These impressive evaluation metrics validate the effectiveness of our developed model in accurately identifying manipulated images.

While we encountered challenges during the project, we successfully overcame them and achieved excellent results. Our future work will evaluate the remaining two models pre-trained on the CASIA2.0_revised dataset. Additionally, we will explore additional techniques and methodologies to enhance the system's accuracy and efficiency further, ensuring its continuous improvement.