In this blog post, I'll guide you through the step-by-step process of setting up a Continuous Integration and Continuous Deployment (CI/CD) pipeline for microservices using Azure DevOps and ArgoCD. This comprehensive guide will help you automate the building, testing, and deployment of your applications, ensuring a seamless and efficient development workflow.
Introduction
Modern software development demands rapid iterations and seamless deployments. Continuous Integration and Continuous Deployment (CI/CD) practices enable developers to merge code frequently, automate tests, and deploy applications efficiently. In this blog, we'll leverage Azure DevOps for Continuous Integration (CI) and ArgoCD for Continuous Deployment (CD) to build a robust pipeline that handles everything from code commits to deploying applications in Azure Kubernetes cluster(AKS).
Architecture
Prerequisites
Before we dive in, make sure you have the following:
An Azure account
An SSH client (e.g., terminal for Mac/Linux, PuTTY for Windows)
Git
Basic knowledge of Kubernetes
Getting Started with Azure
1. Sign Up for an Azure Account
If you don't already have an Azure account, sign up at Azure.
2. Sign In to Azure Portal
Once you have an account, sign in to the Azure Portal.
3. Provision Azure Resources
You can either use Terraform scripts to automate the provisioning of resources or manually create the necessary resources on the Azure portal. For simplicity, we’ll outline the manual steps:
Create a Linux VM: This VM will serve as your agent pool for Azure DevOps.
Create an Azure Container Registry (ACR): This will store your Docker images.
Create an Azure Kubernetes Service (AKS) Cluster: This will host your microservices.
Detailed Steps for Provisioning Resources
Approach. 1: Manual creation
Create a Linux VM:
Go to the Azure Portal, click on "Create a resource," and select "Virtual Machine."
Follow the wizard to set up the VM.
Create an Azure Container Registry (ACR):
Create an Azure Kubernetes Service (AKS) Cluster:
Go to "Create a resource" and select "Kubernetes Service."
Follow the wizard to set up the cluster. Note that if you are using a free tier, you may need to choose a different region to avoid usage quota issues.
Note: Make sure all the resources are in the same resource group it will be easy to delete them.
Approach. 1: Using IaC [terraform] creation
Install Azure CLI
You can visit the following blog and install Az CLI it will hardly take 5 mins.
%[https://blogs.vijaysingh.cloud/mastering-azure-cli]
Login to Azure:
Open your terminal and type the following command
az login
This will open a new browser window for you to sign in to your Azure account. If the CLI can open your default browser, it will do so and load an Azure sign-in page. Otherwise, you need to open a browser page and follow the instructions on the command line to enter an authorization code.
Set your subscription (optional):
If you have multiple Azure subscriptions, and the one you want to use isn’t your default, you can set the subscription you want to use with this command:
az account set --subscription "your-subscription-id"
Replace "your-subscription-id" with your actual subscription ID.
Install Terraform:
If you haven’t installed Terraform, you can download it from the official Terraform website. Unzip the package and move the binary to your PATH.
Terraform Download
Provision Azure Resources:
git clone https://github.com/vsingh55/AzureDevOps-CI-CD.git
cd AzureDevOps-CI-CD/Terraform
terraform init
terraform plan
terraform apply
Setting Up Azure DevOps
1. Create an Azure DevOps Project
2. Export the Repository to Azure
3. Obtain Personal Access Tokens (PAT)
You'll need two personal access tokens: one for the Azure agent and one for ArgoCD. Follow these steps to generate a PAT:
Go to your profile in Azure DevOps.
Navigate to "Personal Access Tokens (PAT)."
Generate new tokens with the required scopes.
Configuring the CI Pipeline
1. Set Up the Agent Pool
Adding VM to Agent Pool
Add the Created VM to the Agent Pool:
In Azure DevOps, go to "Organization settings" and select "Agent pools."
Create a new agent pool and register your Linux VM.
Install the Azure DevOps Agent: SSH into your Linux VM and run the following commands:
wget https://vstsagentpackage.azureedge.net/agent/3.239.1/vsts-agent-linux-x64-3.239.1.tar.gz
sudo apt update
sudo apt install docker.io
mkdir myagent && cd myagent
tar zxvf vsts-agent-linux-x64-3.239.1.tar.gz
./config.sh
During configuration, provide your Azure DevOps server URL (https://dev.azure.com/{your-organization}) and the personal access token.
Start the Agent:
./run.sh
Ensure the agent is running and listed as online in the Azure DevOps portal.
Creating and Configuring Pipelines
Create a New Pipeline:
Add YAML Configuration: Use the provided YAML files for each microservice. These files define the steps to build and push Docker images to the ACR. For example, the voting-service.yml file might look like this:
trigger:
paths:
include:
- vote/*
resources:
- repo: self
variables:
dockerRegistryServiceConnection: 'this Will be automatically generated'
imageRepository: 'voteapp'
containerRegistry: 'vijayazurecicd.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/vote/Dockerfile'
tag: '$(Build.BuildId)'
pool:
name: 'azureagent'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
steps:
- task: Docker@2
displayName: Build an image to Azure container registry
inputs:
containerRegistry: '$(dockerRegistryServiceConnection)'
repository: '$(imageRepository)'
command: 'build'
Dockerfile: 'vote/Dockerfile'
tags: '$(tag)'
- stage: Push
displayName: Push stage
jobs:
- job: Push
displayName: Push
steps:
- task: Docker@2
displayName: Push an image to Azure container registry
inputs:
containerRegistry: '$(dockerRegistryServiceConnection)'
repository: '$(imageRepository)'
command: 'push'
tags: '$(tag)'
Run the Pipeline: Trigger the pipeline to ensure everything is set up correctly. The pipeline should build the Docker image and push it to the ACR.
💡
For all the services please visit Pipelines folder in
Git Repo.
3. Continuous Integration with Azure DevOps
Azure DevOps will handle the building and pushing of Docker images to the Azure Container Registry. Every time a change is pushed to the repository, the pipeline will automatically run, ensuring that the latest version of the code is built and ready for deployment.
Setting up Continuous Delivery:
Azure Kubernetes Services (AKS)
If you have provisioned resources using terraform then skip the step and run the cmd to connect from terminal. It is for those who wants to do it manually.
Run cmd on terminal
az aks get-credentials --resource-group <resource-group-name> --name <K8's cluster name> --overwrite-existing
Deploying with ArgoCD
1. Install ArgoCD
Installing ArgoCD on Kubernetes Cluster
Create a Namespace:
kubectl create namespace argocd
Install ArgoCD:
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl get pods -n argocd
Ensure all ArgoCD pods are running.
Access ArgoCD:
kubectl get svc -n argocd
kubectl edit svc argocd-server -n argocd
Change the type from ClusterIP to NodePort to expose the ArgoCD server.
Configuring ArgoCD for Continuous Deployment
Retrieve Initial Admin Password:
kubectl get secrets -n argocd
kubectl edit secret argocd-initial-admin-secret -n argocd
Within the file find password and copy it.
echo <password> | base64 --decode
This will give you the password for the ArgoCD admin user.
Access ArgoCD UI: In your browser, navigate to http://<node-external_ip>:<nodeport>. Use admin as the username and the decoded password to log in.
Allow inbound rule for ArgoCD: Add inbound rule for argoCD nodeport in AKS networking settings.
Connect to Azure Git Repo:
In the ArgoCD UI, go to Settings and connect your Azure Git repository.
Use the repository URL format:
https://<personal_access_token>@dev.azure.com/<organization_name>/<project_name>/_git/<project_name>
Create an Application in ArgoCD:
In ArgoCD, create a new application.
Fill in the details such as application name, project, sync policy, repository URL, and path.
Automate Image Updates:
Enable AKS to Pull Images from ACR:
kubectl create secret docker-registry <secret-name> \
--namespace <namespace> \
--docker-server=<container-registry-name>.azurecr.io \
--docker-username=<service-principal-ID> \
--docker-password=<service-principal-password>
Get the service principal ID and password from the Azure portal:
Go to the Azure portal, navigate to your ACR, and enable admin access.
Copy the service principal ID and password.
Verifying the Deployment
Verify ArgoCD Sync:
Access Deployed Applications:
Get the external IP and node ports for your applications:
kubectl get nodes -o wide //To know node external ip
kubectl get svc -n <namespace> //
Access the applications using the external IP and ports.
Add Inbound rule for htttp port.
Here are some screenshots before and after the implementing CI/CD
Conclusion
Congratulations! You've successfully set up a CI/CD pipeline using Azure DevOps, AKS and ArgoCD. This pipeline automates the building, testing, and deployment of your microservices, ensuring a seamless and efficient workflow. By leveraging these powerful tools, you can focus on writing code and delivering features while the pipeline takes care of the rest.
Acknowledgements
Thanks to Azure, Azure DevOps , ArgoCD for providing the platform and tools to build this CI/CD pipeline.
💡
Feel free to comment if you have any questions or suggestions!
