Objective of this project is to develop facial recognition models which utilize deep convolutional neural networks with transfer learning, one-shot learning methods, and Siamese networks for efficient and accurate face recognition. This project enabled the team to understand how Neural networks are trained both mathematically as well as in implementation, the concept of feature extraction using convolution as well as the basics of tensorflow and keras. It enabled the developers to understand the fundamentals of computer vision and develop models that work on Transfer Learning, One-Shot Learning and Concept of similarity for face identification.
The project was divided into 3 components:
- FaceNet Approach
- Siamese Network Approach
- Haarcascade Classifier Approach
We thank Dr. Shashank Srivastava and Dr. Shailendra Shukla (Professors CSED, MNNIT Allahabad) under whose guidance this project was developed!
We tried to implement the ideas given in the original FaceNet paper and our initial idea was to the InceptionV3 model by Google as the base dense architecture and then build upon that by excluding top of InceptionV3 and adding a Global Average Pooling followed by a dropout layer, a fully connected layer of 128 neurons to bring down the size to 128 Dimentions as defined in the original FaceNet paper followed by BatchNorm layer, this was the most successful combination achieved by us, the plan was to train the entire InceptionV3 along with the three layers, utilizing offline triplet mining through a data generator and a custom triplet loss function to train the model.
We couldn't train the entire model on the CASIA WebFace dataset due to lack of compute, and that leaves a further project motivation, to test out our best pipeline using hopefully an online triplet generation and CASIA Webface dataset by training it utilizing the GPU available at MNNIT Allahabad in months to come. We did test out our concept and using around 7500 images from LFW dataset and training only the extra layers added, we achieved an accuracy of around 35% on the never seen by the model dataset of BLFW
Then we decided to use a pre-trained model and somehow try to make it perform better on Indian Faces, we chose the David Sandberg's FaceNet implementation (Link to his repo) and fine tuned it using the Indian Actor Faces Dataset This improved the accuracy of the model by 6% for Indian Faces, the model gives an accuracy of 100% on a class size of 20 and an accuracy of 75% on class size of 1600 classes, however we used KNN for classification once embeddings were generated and accuracy is subjuctive to agorithm used for classification
Here's link to the best outcome model of this part of this project: FaceNet_TL_ep10
We thank David Sandberg and Nyoki for their immense contribution to the open source community and providng us the pre-trained model!
The project was made using the python language and thus you can refer to python.org for more information. Following Libraries were used primarily in the development of the project:
- Tensorflow (version 2.15.0) [Considering the fleeting nature of tensorflow updates, we'd recommend using tensorflow 2.15.0 for running this project]
- OpenCV
- Keras (now integrated into tensorflow itself)
- Haarcascade Frontel Face XML file was used in face detection and a wide variety of haarcascade can be found here
Following research papers were referenced for a better understanding and development of the project:
- FaceNet: A Unified Embedding for Face Recognition and Clustering
- Rethinking the Inception Architecture for Computer Vision
- Exploring Simple Siamese Representation Learning
Following datasets were used primarly for training the initial models of FaceNet and Siamese Network:
- Labelled Faces In Wild(LFW) -> Primarily Used For Training Siamese Network Model and Initial Pipelines of FaceNet
- CASIA WebFace -> Used For training the pre-trained pipeline model used in FaceNet as well as tried to train the second model proposition using this dataset
- Indian Actors Dataset -> Used to fine tune the pre-trained model pipeline for better acccuracy on Indian Faces
- Balanced Faces In Wild (BLFW) -> This was used primarly as the testing dataset for FaceNet and its Indian Subset was used for testing the accuracy of pre-trained model as well as the fine tuned model for Indian Faces
- IISCIFD -> Used to test the accuracy of FaceNet model over 1600 classes
We thank all data providers for providing such useful datasets for free for research and education purposes!
This project uses the Haar Cascade classifier based on the Viola-Jones algorithm to perform face detection. The system is designed to capture student images, train a classifier, and mark attendance in an Excel file. The unique selling point of the Viola-Jones algorithm is its efficiency, requiring fewer resources compared to other methods.
1.Haar Features: Simple rectangular features used to detect the presence of various shapes within an image.
2.Integral Image: A representation of the image that allows for quick computation of Haar features.
3.AdaBoost: A machine learning algorithm that selects the best features and combines them into a strong classifier.
4.Cascading: Combines multiple classifiers in a cascade structure to quickly discard non-face regions and focus on promising regions.
The project consists of three main components:
1.Take Samples: Captures sample images of students.
2.Train Model: Trains the Haar Cascade classifier using the captured images.
3.Take Attendance: Detects faces in real-time and marks attendance in an Excel file.
1.take_samples.py: Script to capture sample images of students.
2.train_model.py: Script to train the Haar Cascade classifier with the captured images.
3.take_attendance.py: Script to take attendance and mark it in an Excel file.
1.Clone the repository:
git clone https://github.com/Ayushman444/FaceRecognitionSystem
cd FaceRecognitionSystem/HaarCascade-OpenCV Approach
2.Install the required libraries:
pip install -r requirements.txt
1.Take Samples:
->Run take_samples.py to capture sample images of students.
->Images will be stored in the samples directory.
2.Train Model:
->Run train_model.py to train the Haar Cascade classifier.
->The trained model will be saved as classifier.xml.
3.Take Attendance:
->Run take_attendance.py to detect faces and mark attendance in an Excel file.
->The attendance will be saved in attendance.xlsx.
This project demonstrates a practical application of the Haar Cascade classifier for face detection and attendance marking. The use of the Viola-Jones algorithm ensures efficient and accurate face detection, making it suitable for real-time applications. Additionally it also showcases the two state of the art approaches of similarity based classificiation based on one shot learning : Siamese Networks and Triplet Loss based FaceNet
Collaborate, Discuss and Feel Free To Share Our Work.