ArcGIS Enterprise on Kubernetes in AKS
This walkthrough will guide you through the process of initial deployment and disaster recovery of ArcGIS Enterprise on Kubernetes 11.4 site in Microsoft Azure public cloud using GitHub Actions.
The walkthrough uses azure/arcgis-site-core and azure/arcgis-enterprise-k8s templates.
Duration: about 2 hours
Before you begin, you need to have the following resources and accounts:
- GitHub.com user account
- Service principal in Microsoft Azure account with Owner role
- Microsoft Azure user account to administer Azure Managed Grafana
- Docker Hub account that has access to the private repositories with ArcGIS Enterprise on Kubernetes container images
- Authorization file for ArcGIS Enterprise on Kubernetes 11.4
- Helm charts for ArcGIS Enterprise on Kubernetes 11.4 (ArcGIS_Enterprise_on_Kubernetes_Helm_Charts_1.4.0_193010.tgz)
- Domain name for the ArcGIS Enterprise site
- SSL certificate files (certificate, private key, and chain files in PEM format) for the ArcGIS Enterprise site domain name
The authorization file and Helm charts for ArcGIS Enterprise on Kubernetes can be downloaded from My Esri.
This step creates new private GitHub repository for ArcGIS Enterprise site from the template repository.
1.2 Create new private repository from https://github.com/esri/arcgis-gitops template
Open https://github.com/esri/arcgis-gitops URL in a web browser, click on Use this template green button, and select "Create a new repository" from the dropdown menu.
Select the repository owner, enter "arcgis-aks" as repository name and "ArcGIS Enterprise on Kubernetes in AKS" as description, change the visibility to "Private", and click "Create repository" button.
 |
|---|
This step installs Azure CLI in a GitHub codespace workspace and uses it to create an Azure storage account and a blob container for Terraform backend that will be used by the workflows.
In the new repository, click the green "<> Code" button, switch the tab to "Codespaces", and click the "Create codespace on main" button. The codespace will be created and opened in a new browser tab.
 |
|---|
In the codespace, click on the "Terminal" tab in the bottom panel and run the following commands:
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bashRun the following commands, replacing <client id>, <client secret>, and <tenant id> by the service principal's credentials:
az login --service-principal --username <client id> --password <client secret> --tenant <tenant id>Get all subscriptions for the tenant:
az account subscription listOutput:
[
{
"authorizationSource": "RoleBased",
"displayName": "Azure subscription",
"id": "/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
"state": "Enabled",
"subscriptionId": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
"subscriptionPolicies": {
"locationPlacementId": "Public_2014-09-01",
"quotaId": "MSDNDevTest_2014-09-01",
"spendingLimit": "Off"
}
}
]Note "subscriptionId" value of the subscription you want to use.
Check that the service principal is in the Owner role in the subscription:
az role assignment list --all --assignee <client id> --query "[].{roleDefinitionName:roleDefinitionName scope:scope}" --output tsvOutput:
Owner /subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
Set the current subscription:
az account set --subscription <subscription id>Create a resource group for the Azure storage account in East US region by running the following command:
az group create --name arcgis-gitops --location eastusNames and display names of the available Azure locations can be listed by running
az account list-locations --query "[].{name:name displayName:displayName}" --output tsvcommand.
Run the following command replacing <storage account name> with a globally unique name:
az storage account create --name <storage account name> --resource-group arcgis-gitopsNOTE: Storage account name must be between 3 and 24 characters in length and use numbers and lower-case letters only.
Replace <storage account name> with the storage account name and run the following command to create blob container "tfbackend" in the storage account:
az storage container create --name tfbackend --account-name <storage account name> --auth-mode loginReplace <client id>, <subscription id>, and <storage account name> with the service principal client ID, subscription ID, and the storage account name in the following command and run the command to grant the service principal required access to the blob container:
az role assignment create --role "Storage Blob Data Owner" --assignee <client id> --scope /subscriptions/<subscription id>/resourceGroups/arcgis-gitops/providers/Microsoft.Storage/storageAccounts/<storage account name>/blobServices/default/containers/tfbackendThis step configures and validates secrets and variables in the repository settings used by the GitHub Actions workflows to retrieve Azure and Docker Hub credentials and other properties.
Open the new repository settings by clicking on the Settings tab in the repository, then click on "Secrets and Variables" menu item from "Security" section of the left sidebar, and select "Actions" from the dropdown menu.
- Click on "New repository secret" button, enter
AZURE_CLIENT_IDinto the "Name" field, enter the service principal client ID into the "Secret" field, and click "Add secret" button. - Add
AZURE_CLIENT_SECRETsecret with the service principal client secret. - Add
AZURE_TENANT_IDsecret with the Microsoft Entra tenant ID. - Add
CONTAINER_REGISTRY_USERsecret with the Docker Hub username. - Add
CONTAINER_REGISTRY_PASSWORDsecret with the Docker Hub user password.
Switch the "Secrets"/"Variables" tab to "Variables".
- Click on "New repository variable" button, enter
AZURE_DEFAULT_REGIONinto the "Name" field, enter "East US" into the "Value" field, and click "Add variable" button. - Add
AZURE_SUBSCRIPTION_IDvariable with the Azure subscription ID. - Add
TERRAFORM_BACKEND_STORAGE_ACCOUNT_NAMEvariable with the Azure storage account name of Terraform backend. - Add
TERRAFORM_BACKEND_CONTAINER_NAMEvariable set to "tfbackend".
If you need to use an Azure region other that East US, check if the region is supported by Application Gateway for Containers.
 |
|---|
Click on "Actions" tab in the repository, select "validate-settings-azure" workflow on the left sidebar, click "Run workflow" on the right, and click "Run workflow" button to run the workflow. Reload the web page and wait until the workflow run is completed. The workflow will succeed if the credentials are valid.
 |
|---|
This step copies the required workflows to .github/workflows directory and lists the available workflows in the repository.
Switch back to the codespace browser tab and run the following commands:
cp -r azure/arcgis-site-core/workflows/* .github/workflows/cp -r azure/arcgis-enterprise-k8s/workflows/* .github/workflows/git add --allgit commit -m "Enable workflows"git push origin mainList the available workflows in the repository:
gh workflow listOutput:
NAME STATE ID
enterprise-k8s-azure-backup active 138163280
enterprise-k8s-azure-destroy active 138163281
enterprise-k8s-azure-image active 138163282
enterprise-k8s-azure-ingress active 138163283
enterprise-k8s-azure-organization active 138163284
enterprise-k8s-azure-restore active 138163285
enterprise-k8s-azure-test active 138163286
site-core-azure-destroy active 138163287
site-core-azure active 138163288
site-k8s-cluster-azure-destroy active 138163289
site-k8s-cluster-azure active 138163290
validate-settings-aws active 137938401
validate-settings-azure active 137938402
This step updates the configuration files used by the workflows.
In the EXPLORER sidebar of the codespace window, click on "config" folder, then right-click on "certificates" folder, and select "Upload..." from the context menu. In the file selector dialog window navigate to the SSL certificate files location, select the certificate, private key, and chain files, and click "Open" button to upload the files to the workspace.
Click again on "config" folder, then on "azure" subfolder, and then on "arcgis-enterprise-k8s" subfolder. Double-click on "ingress.tfvars.json" file to open it in the editor.
Change "deployment_fqdn" value to the fully qualified domain name of the ArcGIS Enterprise site.
Replace "tls_certificate_path", "tls_private_key_path", and "ca_certificate_path" properties with the uploaded SSL certificate file paths.
In the EXPLORER sidebar of the codespace window, navigate to "azure/arcgis-enterprise-k8s/organization/helm-charts/arcgis-enterprise" folder and select "Upload..." from the context menu. In the file selector dialog window navigate to the Helm charts location, select ArcGIS_Enterprise_on_Kubernetes_Helm_Charts_1.4.0_193010.tgz file and click "Open" button to upload the file to the workspace.
Change the current folder to the Helm charts subfolder:
cd azure/arcgis-enterprise-k8s/organization/helm-charts/arcgis-enterprise/Extract the Helm charts from the archive:
tar -xzf ArcGIS_Enterprise_on_Kubernetes_Helm_Charts_1.4.0_193010.tgzRename the extracted subfolder to the Helm chart version:
mv arcgis-enterprise 1.4.0Delete the Helm charts archive:
rm ArcGIS_Enterprise_on_Kubernetes_Helm_Charts_1.4.0_193010.tgzGo back to the workspace root folder:
cd /workspaces/arcgis-aksIn the EXPLORER sidebar of the codespace window, click on "config" folder, then right-click on "authorization" folder, select "New Folder..." from the context menu, and enter "11.4" as the folder name. Right-click on the new "11.4" folder, and select "Upload..." from the context menu. In the file selector dialog window navigate to the ArcGIS Enterprise on Kubernetes authorization file location, select the file and click "Open" button to upload the file to the workspace.
Open config/azure/arcgis-enterprise-k8s/organization.tfvars.json file in the editor and update the following properties:
-
Replace "authorization_file_path" property to the uploaded authorization file paths
"~/config/authorization/11.4/<authorization file name>". -
Set "deployment_fqdn" property to the ArcGIS Enterprise site fully qualified domain name.
-
Set "admin_first_name", "admin_last_name", "admin_username", "admin_password", "admin_email", "security_question_index", and "security_question_answer" to the ArcGIS Enterprise primary administrator account properties.
The primary administrator user name must be at least six characters in length. The only special characters allowed are the at sign (@), dash (-), dot (.), and underscore (_). The primary administrator password must be at least eight characters in length. It must contain at least one alphabet letter (uppercase or lowercase), at least one digit, and at least one special character. All special characters are allowed.
Change the "passcode" property value in backup.vars.json and restore.vars.json config files to a passcode that will be used when restoring the backup.
Replace the "deployments" property value in config/azure/site-index.json file by ["arcgis-enterprise-k8s"].
git add --allgit commit -m "Update config files"git push origin mainClick on "Actions" tab in the repository, select "verify-site-config-azure" workflow on the left sidebar, click "Run workflow" on the right, and click "Run workflow" button to run the workflow. Reload the web page and wait until the workflow run is completed. The workflow will succeed if no errors are found in the config files.
This step runs the workflows to provision the core Microsoft Azure infrastructure and create an Azure Kubernetes Service (AKS) cluster.
If a workflow run fails, correct the error and run the workflow again.
Click on "Actions" tab in the repository, refresh the web page, select "site-core-azure" workflow on the left sidebar, click "Run workflow" on the right, and click "Run workflow" button.
Once the workflow run is complete, run the following command in the codespace terminal to list the created Azure resources:
az resource list --resource-group arcgis-infrastructure-core --query "[].{type:type name:name}" --output tsvOutput:
Microsoft.Network/publicIPAddresses arcgis-nat
Microsoft.Network/networkSecurityGroups arcgis-internal
Microsoft.KeyVault/vaults arcgis
Microsoft.Storage/storageAccounts siteced309d2da222641
Microsoft.Network/networkSecurityGroups arcgis-bastion
Microsoft.Network/virtualNetworks arcgis
Microsoft.Network/natGateways arcgis
Microsoft.Network/networkSecurityGroups arcgis-private
Microsoft.Network/privateDnsZones privatelink.blob.core.windows.net
Microsoft.Network/publicIPAddresses arcgis-bastion
Microsoft.Network/privateDnsZones/virtualNetworkLinks privatelink.blob.core.windows.net/blob-private-dns-zone-vnet-link
Microsoft.Network/bastionHosts arcgis-bastion
Microsoft.Network/privateEndpoints siteced309d2da222641-private-endpoint
Microsoft.Network/networkInterfaces siteced309d2da222641-private-endpoint.nic.418e4e78-1bdf-48e9-a892-15bf61f46e1d
Run "site-k8s-cluster-azure" workflow to create an AKS cluster that meets ArcGIS Enterprise on Kubernetes system requirements.
Once the workflow run is complete, run the following command in the codespace terminal to list the created Azure resources:
az resource list --resource-group arcgis-k8s-cluster --query "[].{type:type name:name}" --output tsvOutput:
Microsoft.ContainerService/managedClusters arcgis
Microsoft.ManagedIdentity/userAssignedIdentities azure-alb-identity
Microsoft.Monitor/accounts arcgis-enterprise
Microsoft.ServiceNetworking/trafficControllers arcgis
Microsoft.Network/privateDnsZones privatelink.azurecr.io
Microsoft.ContainerRegistry/registries arcgis214c33f19674fa7f
Microsoft.AlertsManagement/prometheusRuleGroups UXRecordingRulesRuleGroup - arcgis-enterprise
Microsoft.Dashboard/grafana arcgis
Microsoft.AlertsManagement/prometheusRuleGroups NodeRecordingRulesRuleGroup - arcgis
Microsoft.AlertsManagement/prometheusRuleGroups KubernetesRecordingRulesRuleGroup - arcgis
Microsoft.Network/privateEndpoints arcgis214c33f19674fa7f-private-endpoint
Microsoft.Network/privateDnsZones/virtualNetworkLinks privatelink.azurecr.io/acr-private-dns-zone-vnet-link
Microsoft.Network/networkInterfaces arcgis214c33f19674fa7f-private-en.nic.43d62ffc-619a-4dff-85af-0806094bd618
Microsoft.ServiceNetworking/trafficControllers/associations arcgis/arcgis
Retrieve the AKS cluster access credentials into the .kube/config file so kubectl can use them:
az aks get-credentials --resource-group arcgis-k8s-cluster --name arcgis --overwrite-existingTest connection to the AKS cluster:
kubectl get nodesOutput:
NAME STATUS ROLES AGE VERSION
aks-default-21074325-vmss000000 Ready <none> 13m v1.30.7
aks-default-21074325-vmss000001 Ready <none> 13m v1.30.7
aks-default-21074325-vmss000002 Ready <none> 13m v1.30.7
aks-default-21074325-vmss000003 Ready <none> 13m v1.30.7
Replace <user object id with the Azure user account object ID and <subscription id> with the Azure subscription ID and run the following command to assign the user "Grafana Admin" role in "arcgis" Azure Managed Grafana instance:
az role assignment create --assignee-object-id "<user object id>" --assignee-principal-type user --role "Grafana Admin" --scope "/subscriptions/<subscription id>/resourceGroups/arcgis-k8s-cluster/providers/Microsoft.Dashboard/grafana/arcgis"If the service principal has permissions to query Microsoft Entra ID, you also can use the user's sign-in name instead of client ID with
--assigneeparameter.
Retrieve the Grafana endpoint URL from the workflow run log or from the output of the following command:
az grafana show --resource-group arcgis-k8s-cluster --name arcgisOpen the URL in a web browser and login to Grafana with the Azure Entra ID user.
This step runs the workflows to build Enterprise Admin CLI container image, create Kubernetes namespace for the deployment and configure ingress, create ArcGIS Enterprise organization, test and backup the deployment.
Run "enterprise-k8s-azure-image" workflow that builds Enterprise Admin CLI container image and pushes it to the private ACR repository of the AKS cluster.
Run "enterprise-k8s-azure-ingress" workflow that creates the ingress resources for the ArcGIS Enterprise on Kubernetes deployment.
After the workflow run is complete, retrieve DNS name of the load balancer created by the workflow from the workflow run log and create a CNAME record for it within the DNS server of the ArcGIS Enterprise domain name.
 |
|---|
Run "enterprise-k8s-azure-organization" workflow to deploy ArcGIS Enterprise on Kubernetes in the AKS cluster and create an ArcGIS Enterprise organization.
After the workflows run is complete, the ArcGIS Enterprise site will be accessible at the deployment FQDN https://<deployment FQDN>/arcgis/manager.
The first run of the workflow may fail with BackoffLimitExceeded error. Run the workflow again if that happened.
Run "enterprise-k8s-azure-test" workflow.
Run "enterprise-k8s-azure-backup" workflow.
 |
|---|
This step destroys the Azure resources created by the walkthrough.
Run "enterprise-k8s-azure-destroy" workflow with "Delete namespace" option checked.
Run "site-k8s-cluster-azure-destroy" workflow.
Run "site-core-azure-destroy" workflow.
Run the following command in the codespace terminal to delete "arcgis-gitops" resource group:
az group delete --name arcgis-gitopsThe walkthrough demonstrates only the basic capabilities of ArcGIS Automation using GitHub Actions. Check instructions of the templates and workflows for more advanced features and options.