Welcome to the Network Security Threat Detection project — an end-to-end implementation of a production-grade ML pipeline integrating modern MLOps tools including MLflow, DVC, Docker, GitHub Actions, CI/CD, and Dagshub.
💡 Designed for real-world use cases like intrusion detection using machine learning — deployable, trackable, reproducible, and scalable!
In this project, we tackle the problem of network intrusion detection using ML algorithms. Our goal is to build a pipeline that:
- Ingests network traffic data
- Preprocesses and extracts meaningful features
- Trains a model to classify normal vs suspicious activity
- Deploys the model with full CI/CD and MLOps support
- ✅ Tracks raw and processed datasets
- ✅ Pipeline defined in
dvc.yaml - ✅ Remote storage: DagsHub
- 🎯 Log metrics, parameters, and artifacts
- 📊 Compare multiple models & runs visually
- 🧠 Integrate with sklearn pipeline easily
- Reproducible environments
- Easy deployment across dev/stage/prod
- Every
pushorPRtriggers:- ✅ Code linting
- ✅ Unit tests
- ✅ DVC pipeline execution
- ✅ MLflow tracking
- ✅ Docker build & push
- Containerized inference API (FastAPI or Flask)
- Ready for deployment on AWS/GCP/Azure or Heroku
| Category | Tools |
|---|---|
| 👨💻 Programming | Python, Pandas, NumPy, Scikit-learn |
| 📦 ML Lifecycle | MLflow, DVC, Dagshub, Hydra |
| 🧱 MLOps/CI-CD | GitHub Actions, Docker, YAML, Pytest |
| 🌐 Visualization | Matplotlib, Seaborn, MLflow UI |
| 🔁 Version Control | Git, GitHub |
# 1. Clone the repo
git clone https://github.com/yourusername/network-security-mlops.git
cd network-security-mlops
# 2. Set up DVC
dvc pull # Pull data from remote storage
dvc repro # Reproduce the entire pipeline
# 3. Track experiments
mlflow ui # Launch MLflow dashboard at http://127.0.0.1:5000
# 4. Build Docker container
docker build -t network-mlops-app .
docker run -p 8000:8000 network-mlops-app
This is the dataset used to build and train the machine learning model in this project.
Download it from the link below:
- 🔢 Accuracy, Precision, Recall, ROC Curve
- 📁 Model artifact: RandomForestClassifier.pkl
- 📊 Visual run comparison dashboard
For seamless monitoring of the project’s DVC pipelines and MLflow experiments, kindly visit the Dagshub repository linked below:
Here's a preview of how the experiments and metrics are logged in MLflow:
Below is the Models metrics That have used in this model traning
Below is a visualization of metrics like accuracy, precision, recall over different runs:
DVC pipeline stages and tracking interface hosted on Dagshub:
.github/workflows/mlops.yml
Runs the full pipeline on every push:
- ✅ Lint + Tests
- ✅ DVC repro
- ✅ MLflow log
- ✅ Docker image push
- ✨ Add LSTM or Deep Learning models
- 📦 Convert to FastAPI microservice
- 🔐 Add role-based access control for logs
- 📊 Grafana + Prometheus monitoring
- ☁️ Deploy to AWS Sagemaker or GCP AI Platform
- ✨ CICD Pipeline
If you liked this project, please consider giving it a ⭐️ and sharing with others!
🛡️ Empowering network security with reproducible ML pipelines!
Choose either of the following options to serve your trained model:
- An interactive web interface to test your model
- Upload CSV or manually enter input features
- See predictions live, with metrics and visualizations
streamlit run src/app_streamlit.py
- A production-grade REST API endpoint to serve predictions
- Send POST requests with JSON payload
- Easily integrate with any web or mobile frontend
python src/app_flask.py
💡 Whether you're demoing the model in a web dashboard (Streamlit) or deploying via API (Flask), this project gives you flexibility to move fast in any real-world environment.
This project structure and workflow was inspired by Krish Naik sir’s teachings on practical MLOps and end-to-end ML deployment.
💡 I’ve extended and personalized it with full MLOps integration, CI/CD, and deployment options for real-world use cases in Network Security.
You can also explore Krish Naik’s YouTube tutorials on:
- DVC + MLflow
- MLOps project pipelines
- Dockerized ML apps
- Streamlit deployment
- GitHub Actions for CI/CD
Contributions are welcome to enhance this open-source solution further!
💬 “Learning by doing is the fastest path to mastery.” — Thanks to Krish Naik sir for being a constant inspiration to ML learners.