This project implements continous integration and continous delivery pipeline for a Python machine learning API built with the Flask web framework. The application is deployed into Azure.
The pre-trained sklearn model used predicts housing prices in Boston according to several features, such as average rooms in a home, data about highway access, teacher-to-pupil ratio, and so on. If you're curious, you can read more about the data, which was initially taken from Kaggle, on the data source site. This project also demonstrates operationalization of the serverless machine learning app (app.py) that serves out predictions (inference) about housing prices through API calls. This project can be extended to any pre-trained machine learning model, such as those used for image recognition and data labeling.
- Link to Trello board for this project.
- Link to spreadsheet that includes the initial and future plans for this project.
The quickest way to create an App Service instance is to use the Azure command-line
interface (CLI) through the interactive Azure Cloud Shell. In the following steps,
you use az webapp up to both provision the App Service and perform your first
deployment of the microservice API.
-
Sign in to the Azure portal at https://portal.azure.com.
-
Open the Azure CLI by selecting the Cloud Shell button on the portal's toolbar:
-
The Cloud Shell appears along the bottom of the browser. Select Bash from the dropdown:
-
On GitHub, fork the repository https://github.com/mudathirlawal/cicd-for-ml-microservice-api-on-azure.git into your GitHub account.
-
In the Cloud Shell, clone the repository using
git clone. As in:git clone https://github.com/<your-alias>/cicd-for-ml-microservice-api-on-azure
Replace with the name of the GitHub account you used to fork the repository.
To paste into the Cloud Shell, use
Ctrl+Shift+V, or right-click and select Paste from the context menu.The Cloud Shell is backed by an Azure Storage account in a resource group called cloud-shell-storage-[your-region]. That storage account contains an image of the Cloud Shell's file system, which stores the cloned repository. There is a small cost for this storage. You can delete the storage account at the end of this article, along with other resources you create.
-
In the Cloud Shell, change directories into the repository folder that has the Flask app, so the
az webapp upcommand will recognize the app as Python. Then execute:cd cicd-for-ml-microservice-api-on-azure -
In the Cloud Shell, run
az webapp upto create an App Service and initially deploy the API.az webapp up -n [your-appservice]
Change [your-appservice] to a name for your app service that's unique across Azure. Typically, you use a personal or company name along with an app identifier, such as [your-name]-flaskpipelines. The app URL becomes [your-appservice].azurewebsites.net.
When the command completes, it shows JSON output in the Cloud Shell.
If you encounter a "Permission denied" error with a .zip file, you may have tried to run the command from a folder that doesn't contain a Python app. The az webapp up command then tries to create a Windows app service plan, and fails.
From the first line of output from the previous
az webapp upcommand, copy the name of your resource group, which is similar to:{your-name}_rg_Linux_{your-region}Enter the following command, using your resource group name, your app service name, and your startup file or command:
az webapp config set -g <your-resource-group> -n <your-appservice>
Again, when the command completes, it shows JSON output in the Cloud Shell.
To see the running app, open a browser and go to
http://[your-appservice].azurewebsites.net. If you see a generic page, wait a few seconds for the App Service to start, and refresh the page.
cd ~
sudo apt-get install python3-venv # If needed; not needed in Azure Cloud Shell
git clone https://github.com/<your-alias>/cicd-for-ml-microservice-api-on-azure
cd cicd-for-ml-microservice-api-on-azure
python3 -m venv ~/.cicd-for-ml-microservice-api-on-azure
source ~/.cicd-for-ml-microservice-api-on-azure/bin/activate
make all
az webapp up -n <your-desired-name-for-the-appservice>
az webapp config set -g <your-resource-group> -n <your-appservice-name>
export set FLASK_APP=app.py
python3 -m flask run
./make_prediction_on_azure.sh # Make a prediction by running this script.
# To see the running app, open a browser and go to:
# http://<your-appservice-name>.azurewebsites.netpy -3 -m venv .cicd-for-ml-microservice-api-on-azure
.cicd-for-ml-microservice-api-on-azure\scripts\activate
pip install -r requirements.txt
$env:FLASK_APP = "app.py"
python -m flask runOpen a browser and navigate to http://localhost:5000 to view the app. When you are finished, close the browser, and stop the Flask server with Ctrl+C.
To deploy to Azure App Service from Azure Pipelines, you need to establish a service connection between the two services.
In a browser, go to dev.azure.com. If you don't yet have an account on Azure DevOps,
select Start free and get a free account. If you have an account already, select
Sign in to Azure DevOps.
To simplify the service connection, use the same email address for Azure DevOps as you use for Azure.
Once you sign in, the browser displays your Azure DevOps dashboard, at the URL
https://dev.azure.com/<your-organization-name>. An Azure DevOps account can belong to
one or more organizations, which are listed on the left side of the Azure DevOps dashboard.
If more than one organization is listed, select the one you want to use for this walkthrough.
By default, Azure DevOps creates a new organization using the email alias you used to sign in.
A project is a grouping for boards, repositories, pipelines, and other aspects of Azure DevOps. If your organization doesn't have any projects, enter the project name Flask Pipelines under Create a project to get started, and then select Create project.
If your organization already has projects, select New project on the organization page. In the Create new project dialog box, enter the project name Flask Pipelines, and select Create.
From the new project page, select Project settings from the left navigation.
On the Project Settings page, select Pipelines > Service connections, then select New service connection, and then select Azure Resource Manager from the dropdown.
In the Add an Azure Resource Manager service connection dialog box:
Give the connection a name. Make note of the name to use later in the pipeline. For Scope level, select Subscription. Select the subscription for your App Service from the Subscription drop-down list. Under Resource Group, select your resource group from the dropdown. Make sure the option Allow all pipelines to use this connection is selected, and then select OK.
The new connection appears in the Service connections list, and is ready for Azure Pipelines to use from the project.
If you need to use an Azure subscription from a different email account, follow the instructions on Create an Azure Resource Manager service connection with an existing service principal.
-
From your project page left navigation, select Pipelines.
-
Select New pipeline.
-
On the Where is your code screen, select GitHub. You may be prompted to sign into GitHub.
-
On the Select a repository screen, select the repository that contains your app, such as your fork of the Flask app.
-
You may be prompted to enter your GitHub password again as a confirmation, and then GitHub prompts you to install the Azure Pipelines extension. On this screen, scroll down to the Repository access section, choose whether to install the extension on all repositories or only selected ones, and then select Approve and install.
-
On the Configure your pipeline screen, select Python to Linux Web App on Azure. Your new pipeline appears. When prompted, select the Azure subscription in which you created your Web App.
- Select the Web App
- Select Validate and configure
Azure Pipelines creates an
azure-pipelines.ymlfile that defines your CI/CD pipeline as a series of stages, Jobs, and steps, where each step contains the details for different tasks and scripts. Take a look at the pipeline to see what it does. Make sure all the default inputs are appropriate for your code.
-
Select Save at upper right in the editor, and in the pop-up window, add a commit message and select Save.
-
Select Run on the pipeline editor, and select Run again in the Run pipeline dialog box. Azure Pipelines queues another pipeline run, acquires an available build agent, and has that build agent run the pipeline. The pipeline takes a few minutes to complete, especially the deployment steps. You should see green checkmarks next to each of the steps.
If there's an error, you can quickly return to the
YAMLeditor by selecting the vertical dots at upper right and selecting Edit pipeline. -
From the build page, select the Azure Web App task to display its output. To visit the deployed site, hold down the
Ctrlkey and select the URL after App Service Application URL.
If your app fails because of a missing dependency, then your requirements.txt file was not processed during deployment. This behavior happens if you created the web app directly on the portal rather than using the
az webapp upcommand as directed in this guide.
The
az webapp upcommand specifically sets the build action SCM_DO_BUILD_DURING_DEPLOYMENT to true. If you provisioned the app service through the portal, however, this action is not automatically set.
The following steps set the action:
Open the Azure portal, select your App Service, then select Configuration. Under the Application Settings tab, select New Application Setting. In the popup that appears, set Name to SCM_DO_BUILD_DURING_DEPLOYMENT, set Value to true, and select OK. Select Save at the top of the Configuration page. Run the pipeline again. Your dependencies should be installed during deployment.
To avoid incurring ongoing charges for any Azure resources you created in this walkthrough, such as a B1 App
Service Plan, delete the resource group that contains the App Service and the App Service Plan. To delete the
resource group from the Azure portal, select Resource groups in the left navigation. In the resource group
list, select the ... to the right of the resource group you want to delete, select Delete resource group,
and follow the prompts.
You can also use az group delete in the Cloud Shell to delete resource groups.
To delete the storage account that maintains the file system for Cloud Shell, which incurs a small monthly charge, delete the resource group that begins with cloud-shell-storage-.
The Machine Learning model used in building this API is pre-trained. Therefore, to further generalize the implementation of this Continous Integration & Continous Deployment process, we could integrate the dataset training stage in the pipeline by employing the Azure Kubernetes Service (AKS) while leveraging relevant tools such as TensorFlow and KubeFlow to simplify the operation. In this architecture, the models will run in AKS clusters backed by GPU-enabled virtual machines.
For further information on this, see Machine Learning model training with AKS by Microsoft.
- Microsoft 2020, accessed 2020-10-07,
https://azure.microsoft.com/en-us/services/kubernetes-service/#solution-architectures - Microsoft 2020, accessed 2020-10-07,
https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python-webapp?view=azure-devops