Overview

A Github repo is built from scratch along with a scaffolding to achieve Continuous Integration and Continuous Delivery. Project scaffolding includes Makefile, requirements.txt, Python virtual environment , project scripts and test files. To explore the CI/CD pipeline, A python based machine learning app using the flask web framework has been deployed as the Azure web app. This ML application uses a pre trained sklearn model to predict house prices in Boston based on certain defined features.

The Project intends to use

• Github Actions
• Azure Pipelines.

Project Plan

• A link to a Trello board for the project
• A link to a spreadsheet that includes the original and final project plan

Instructions

The top level architecture of the project is presented below

Whenever new code is pushed into github, the Gihub Actions is launched by deafult to run the tests listed in Makefile, thus ensuring Continuous Integration. The Azure piplines are connected to Github to ensure Continuous Delivery, thus Azure Web App is deployed on every successfull code commit.

1. Prerequisites

   • Azure Account
   • Azure DevOp Account
   • Azure Pipelines

2. Azure CLI Setup

   • Login to Azure CLI
   • Setup Azure Shell with Github Integration
     • Generate ssh keygen using the command ssh-keygen -t rsa
     • Use the command cat your-keyfile-path and copy the public key
     • Add the public key as new SSH Key in Github
   • Clone the project using the command git clone git@github.com:arunprakashpj/Deploying-CICD-Pipeline-in-Azure.git

3. Flask ML Service Launch

   • Setup a virtual environment using the following command -python3 -m venv ~/.demovenv -source ~/.demovenv/bin/activate
   • Run make install
   • Run az webapp up -n <your-appservice>

4. Configure Continuous Integration

   • Workflow automation can be achieved by Github Actions
   • You can find the Action file inside the .github/workflows
   • If you prefer to create Github Action on your own,
     • Click on Actions on the top of your repo
     • Click New Workflow
     • Replace the content of the yml file with the contents of .github/workflows/pythonapp.yml
     • Commit with appropriate commit message

5. Configure Continuous Delivery

   • Login Azure DevOp
   • Create new project
   • Setup new service connection (Preferrably of type Azure Resource Manager)
   • Choose your subscription scope level, subscription, resource group, service connection name and grand access to all pipelines
   • Select pipelines and create a new one
   • Connect to Github
   • Configure Python to Linux Web App on Azure
   • Save and Run

Now the pipeline is created. The pipeline will pull the new version of code during the each commit. You can access the commit history, build summary, deployment summary can be accessed easily.

Screenshots :

• Project cloned into Azure Cloud Shell

• Execute makeall command

• Command to deploy the webapp

• Project running on Azure App Service

• Continuous Integration

• Running Azure App Service from Azure Pipelines automatic deployment

• Deployed the project successfully in the Azure Pipelines. Note the official documentation should be referred to and double checked as you setup CI/CD.

• Successful prediction from deployed flask app in Azure Cloud Shell is observed. The output looks similar to this:

udacity@Azure:~$ ./make_predict_azure_app.sh
Port: 443
{"prediction":[20.35373177134412]}

• Output of streamed log files from deployed application

• Output of load testing on the deployed application

Enhancements

There is a lot of scope to improve this basic CI/CD project.

Potential improvements can be

• Addition of more quality gateways
• Role based restrictions
• Distinguished dev and prod environment.

Demo