E2E Machine Learning Framework

This framework allows an E2E machine learning framework from data ingestion, analysis, and deployments over cloud interfaces.

Technologies and Frameworks

ZenML

We use ZenML as our ML pipiline. For more ZenML.

Where your data alts

your data files needs to go in the /data directory

Run your experiments locally with MLFlow

1. Install ZenML

pip install zenml
pip install "zenml[server]==0.80.1"
zenml integration install mlflow -y

2. Run ZenML Server. zenml login --local zenml init

3. Register your tracker in zenml. <tracker_name> is the tracker name zenml experiment-tracker register <tracker_name> -f <flavor name>

4. Register artifact store zenml artifact-store register <store_name> --flavor <flavor name>

5. Register model deployer zenml model-deployer register <deployer_name> --flavor=<flavor name>

6. Register ZenML stack. zenml stack register <stack_name> -e <tracker_name> -a <store_name> -o default

7. Set your global stack zenml stack set <tracker_name>

8. execute demo executor python run_my_pipeline.py

Test Coverage

We use coverage and unittest packages for unit testing

1. Install coverage package by pip install coverage
2. Run tests and get overage by python -m coverage run -m unittest tests/*.py
3. Coverage report using coverage report and html report can be generated by coverage html