In this blog post, I'll guide you through the step-by-step process of setting up a Continuous Integration and Continuous Deployment (CI/CD) pipeline for microservices using Azure DevOps and ArgoCD. This comprehensive guide will help you automate the building, testing, and deployment of your applications, ensuring a seamless and efficient development workflow.
Introduction
Modern software development demands rapid iterations and seamless deployments. Continuous Integration and Continuous Deployment (CI/CD) practices enable developers to merge code frequently, automate tests, and deploy applications efficiently. In this blog, we'll leverage Azure DevOps for Continuous Integration (CI) and ArgoCD for Continuous Deployment (CD) to build a robust pipeline that handles everything from code commits to deploying applications in Azure Kubernetes cluster(AKS).
Architecture
Prerequisites
Before we dive in, make sure you have the following:
An Azure account
An SSH client (e.g., terminal for Mac/Linux, PuTTY for Windows)
Git
Basic knowledge of Kubernetes
Getting Started with Azure
1. Sign Up for an Azure Account
If you don't already have an Azure account, sign up at Azure.
2. Sign In to Azure Portal
Once you have an account, sign in to the Azure Portal.
3. Provision Azure Resources
You can either use Terraform scripts to automate the provisioning of resources or manually create the necessary resources on the Azure portal. For simplicity, we’ll outline the manual steps:
Create a Linux VM: This VM will serve as your agent pool for Azure DevOps.
Create an Azure Container Registry (ACR): This will store your Docker images.
Create an Azure Kubernetes Service (AKS) Cluster: This will host your microservices.
Detailed Steps for Provisioning Resources
Approach. 1: Manual creation
Create a Linux VM:
Go to the Azure Portal, click on "Create a resource," and select "Virtual Machine."
Follow the wizard to set up the VM.
Create an Azure Container Registry (ACR):
Navigate to "Create a resource" and select "Container Registry."
Fill in the required details and create the registry.
Create an Azure Kubernetes Service (AKS) Cluster:
Go to "Create a resource" and select "Kubernetes Service."
Follow the wizard to set up the cluster. Note that if you are using a free tier, you may need to choose a different region to avoid usage quota issues.
Note: Make sure all the resources are in the same resource group it will be easy to delete them.
Approach. 1: Using IaC [terraform] creation
Install Azure CLI
You can visit the following blog and install Az CLI it will hardly take 5 mins.
Login to Azure:
Open your terminal and type the following command
az login
This will open a new browser window for you to sign in to your Azure account. If the CLI can open your default browser, it will do so and load an Azure sign-in page. Otherwise, you need to open a browser page and follow the instructions on the command line to enter an authorization code.
Set your subscription (optional):
If you have multiple Azure subscriptions, and the one you want to use isn’t your default, you can set the subscription you want to use with this command:
az account set --subscription "your-subscription-id"
Replace
"your-subscription-id"
with your actual subscription ID.Install Terraform:
If you haven’t installed Terraform, you can download it from the official Terraform website. Unzip the package and move the binary to your PATH.
Provision Azure Resources:
git clone https://github.com/vsingh55/AzureDevOps-CI-CD.git cd AzureDevOps-CI-CD/Terraform terraform init terraform plan terraform apply
Setting Up Azure DevOps
1. Create an Azure DevOps Project
Sign in to Azure DevOps.
Create a new project by clicking on "New Project."
Give your project a name and description and select the visibility (private or public).
2. Export the Repository to Azure
Export the voting-app repository to your azure devops portal because we are going to use microservices of voting-app.
https://github.com/dockersamples/example-voting-app.git
3. Obtain Personal Access Tokens (PAT)
You'll need two personal access tokens: one for the Azure agent and one for ArgoCD. Follow these steps to generate a PAT:
Go to your profile in Azure DevOps.
Navigate to "Personal Access Tokens (PAT)."
Generate new tokens with the required scopes.
Configuring the CI Pipeline
1. Set Up the Agent Pool
Adding VM to Agent Pool
Add the Created VM to the Agent Pool:
In Azure DevOps, go to "Organization settings" and select "Agent pools."
Create a new agent pool and register your Linux VM.
Install the Azure DevOps Agent: SSH into your Linux VM and run the following commands:
wget https://vstsagentpackage.azureedge.net/agent/3.239.1/vsts-agent-linux-x64-3.239.1.tar.gz sudo apt update sudo apt install docker.io mkdir myagent && cd myagent tar zxvf vsts-agent-linux-x64-3.239.1.tar.gz ./config.sh
During configuration, provide your Azure DevOps server URL (
https://dev.azure.com/{your-organization}
) and the personal access token.Start the Agent:
./run.sh
Ensure the agent is running and listed as online in the Azure DevOps portal.
2. Configure Pipelines
Creating and Configuring Pipelines
Create a New Pipeline:
Go to the Pipelines section in Azure DevOps.
Create a new pipeline and select "Azure Repos Git" as the source.
Add YAML Configuration: Use the provided YAML files for each microservice. These files define the steps to build and push Docker images to the ACR. For example, the
voting-service.yml
file might look like this:# Docker # Build and push an image to Azure Container Registry # https://docs.microsoft.com/azure/devops/pipelines/languages/docker trigger: paths: include: - vote/* resources: - repo: self variables: # Container registry service connection established during pipeline creation dockerRegistryServiceConnection: 'this Will be automatically generated' imageRepository: 'voteapp' containerRegistry: 'vijayazurecicd.azurecr.io' dockerfilePath: '$(Build.SourcesDirectory)/vote/Dockerfile' tag: '$(Build.BuildId)' pool: name: 'azureagent' stages: - stage: Build displayName: Build stage jobs: - job: Build displayName: Build steps: - task: Docker@2 displayName: Build an image to Azure container registry inputs: containerRegistry: '$(dockerRegistryServiceConnection)' repository: '$(imageRepository)' command: 'build' Dockerfile: 'vote/Dockerfile' tags: '$(tag)' - stage: Push displayName: Push stage jobs: - job: Push displayName: Push steps: - task: Docker@2 displayName: Push an image to Azure container registry inputs: containerRegistry: '$(dockerRegistryServiceConnection)' repository: '$(imageRepository)' command: 'push' tags: '$(tag)'
Run the Pipeline: Trigger the pipeline to ensure everything is set up correctly. The pipeline should build the Docker image and push it to the ACR.
3. Continuous Integration with Azure DevOps
Azure DevOps will handle the building and pushing of Docker images to the Azure Container Registry. Every time a change is pushed to the repository, the pipeline will automatically run, ensuring that the latest version of the code is built and ready for deployment.
Setting up Continuous Delivery:
Azure Kubernetes Services (AKS)
If you have provisioned resources using terraform then skip the step and run the cmd to connect from terminal. It is for those who wants to do it manually.
Run cmd on terminal
az aks get-credentials --resource-group <resource-group-name> --name <K8's cluster name> --overwrite-existing
Deploying with ArgoCD
1. Install ArgoCD
Installing ArgoCD on Kubernetes Cluster
Create a Namespace:
kubectl create namespace argocd
Install ArgoCD:
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml kubectl get pods -n argocd
Ensure all ArgoCD pods are running.
Access ArgoCD:
kubectl get svc -n argocd kubectl edit svc argocd-server -n argocd
Change the
type
fromClusterIP
toNodePort
to expose the ArgoCD server.
2. Configure ArgoCD
Configuring ArgoCD for Continuous Deployment
Retrieve Initial Admin Password:
kubectl get secrets -n argocd kubectl edit secret argocd-initial-admin-secret -n argocd
Within the file find password and copy it.
echo <password> | base64 --decode
This will give you the password for the ArgoCD admin user.
Access ArgoCD UI: In your browser, navigate to
http://<node-external_ip>:<nodeport>
. Useadmin
as the username and the decoded password to log in.Allow inbound rule for ArgoCD: Add inbound rule for argoCD nodeport in AKS networking settings.
Connect to Azure Git Repo:
In the ArgoCD UI, go to Settings and connect your Azure Git repository.
Use the repository URL format:
https://<personal_access_token>@dev.azure.com/<organization_name>/<project_name>/_git/<project_name>
Create an Application in ArgoCD:
In ArgoCD, create a new application.
Fill in the details such as application name, project, sync policy, repository URL, and path.
Automate Image Updates:
Write a script to update Kubernetes manifests with the new image name from the ACR.
Create a folder in your Azure repo for the scripts and integrate them into your pipelines lets call it update stage.
Enable AKS to Pull Images from ACR:
kubectl create secret docker-registry <secret-name> \ --namespace <namespace> \ --docker-server=<container-registry-name>.azurecr.io \ --docker-username=<service-principal-ID> \ --docker-password=<service-principal-password>
Get the service principal ID and password from the Azure portal:
Go to the Azure portal, navigate to your ACR, and enable admin access.
Copy the service principal ID and password.
Verifying the Deployment
Verify ArgoCD Sync:
Make changes to the application code and push to the repository.
ArgoCD will detect the changes and sync the application automatically.
Access Deployed Applications:
Get the external IP and node ports for your applications:
kubectl get nodes -o wide //To know node external ip kubectl get svc -n <namespace> //
Access the applications using the external IP and ports.
Add Inbound rule for htttp port.
Here are some screenshots before and after the implementing CI/CD
Conclusion
Congratulations! You've successfully set up a CI/CD pipeline using Azure DevOps, AKS and ArgoCD. This pipeline automates the building, testing, and deployment of your microservices, ensuring a seamless and efficient workflow. By leveraging these powerful tools, you can focus on writing code and delivering features while the pipeline takes care of the rest.
Acknowledgements
Thanks to Azure, Azure DevOps , ArgoCD for providing the platform and tools to build this CI/CD pipeline.