GitOpsification of Cloud Infrastructure using OpenShift GitOps(Argo CD) and Cross Plane

Introduction

The goal of this article is to achieve the declarative continuous deployment of cloud infrastructure using Git as a single source of truth or in the other words “GitOpsfication of cloud infrastructure”.

There is a lot of talk about GitOps for continuous delivery of applications or kubernetes infrastructure but in this article we will explore the integration of GitOps with Cloud Infrastructure automation.

Problem Statement

Automating the cloud infrastructure using Infrastructure as Code(IaC) is one of the most popular things in the DevOps community and there are many tools like Terraform, Cloud Formation Templates, Azure Resource Manager e.t.c.,. to achieve this. However, most of these tools have their own limitations.

Tools like Azure Resource Manager and AWS CFT are not universal tools and would not work in the world of Hybrid cloud. Whereas on the other side, tools like Terraform that support most of the popular cloud providers are not intelligent enough to determine and fix the manual changes made to the cloud providers.

Also, Terraform which completely relies on the state file cannot be stored in the git repositories due to the following reasons.

1. It may contain sensitive Information.
2. Incorrect or Stale state. When Terraform is executed, it executes against the state file. If a state file is checked into the Git, there is a risk of running Terraform against an old state file. For example, if you forget to pull the latest changes the state file on your machine could be different to other team members.

While terraform aims to solve the above issues using remote backends like s3 buckets, our focus of interest for this article is Git. So, this is a blocker to implement GitOps for cloud infrastructure automation using Git as source of truth.

Crossplane over Terraform

Crossplane extends your Kubernetes cluster to support orchestrating any infrastructure or managed service. The major advantage of Crossplane is it can be used in the GitOps approach as

1. Crossplane can be installed on a kubernetes cluster and crossplane definitions are Kubernetes custom resources.
2. Crossplane manifests can be stored in Git.

The other important aspect of crossplane is that it maintains a state between the deployed kubernetes manifests and actual Infrastructure of the cloud provider.

This is how a simple Crossplane manifest for creating s3 bucket would look like:

You can refer to crossplane-contrib which is their official github organisation for more examples.

OpenShift-GitOps with CrossPlane comes to the Rescue

Integrating Crossplane with OpenShift-GitOps builds a Robust and an Auto-Healing system. Apart from this, Gitopsification also brings in the advantages of Git, like version control, auditing, reviews and rollbacks.

OpenShift-GitOps takes care of continuously monitoring the Git repository for any changes and deploying them without any manual intervention. Also, OpenShift-GitOps maintains a state between the manifests stored in Git repository and those which are deployed into kubernetes.

To summarise the advantages:

• Infrastructure tracking, reviewing and auditing using Git.
• Git is a single source of truth.
• Rollbacks are easy.
• Auto healing, any unintentional or unwanted changes made to the infrastructure are corrected by the system.
• Reliability.

Installation and configuration

Step 1: On your OpenShift cluster, Install CrossPlane. The below commands will install crossplane in the crossplane-system namespace.

Due to the nature of OpenShift’s enterprise strength security model, when we install Crossplane we need to set various security contexts. By setting it to null we pass through the user credentials authority of the logged in OpenShift user through to the Crossplane components.

# In OpenShift this creates the Service Account ‘crossplane’ during the # chart install
$ oc create ns crossplane-system
$ helm install crossplane — namespace crossplane-system crossplane-stable/crossplane — version 1.9.0 — set securityContextCrossplane.runAsUser=null — set securityContextCrossplane.runAsGroup=null — set securityContextRBACManager.runAsUser=null — set securityContextRBACManager.runAsGroup=null — set alpha.oam.enabled=true
# Install the Crossplane CLI
$ curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh
$ sudo mv kubectl-crossplane /usr/local/bin
# check the Crossplane CLI installed and working
$ kubectl crossplane — version
$ kubectl crossplane — help

Step 2: On your OpenShift cluster, Install GitOps operator. Proceed with the creation of a gitops instance in the dedicated namespace or you the default instance that is available in openshift-gitops namespace. Please note that this is a cluster scoped resource, so it can deploy cluster scoped resources without any additional configuration.

Step 3: Configure AWS provider for crossplane. It is expected that you already have aws cli configured with your AWS account. If not please follow the instructions mentioned here.

Once you have the AWS CLI configured, Install and Configure crossplane AWS provider.

kubectl crossplane install provider crossplane/provider-aws:v0.21.2 aws-sample
AWS_PROFILE=default && echo -e “[default]\naws_access_key_id = $(aws configure get aws_access_key_id — profile $AWS_PROFILE)\naws_secret_access_key = $(aws configure get aws_secret_access_key — profile $AWS_PROFILE)” > creds.conf
kubectl create secret generic aws-creds -n crossplane-system — from-file=creds=./creds.conf

And finally configure the provider …

kubectl apply -f https://raw.githubusercontent.com/crossplane/crossplane/release-1.9/docs/snippets/configure/aws/providerconfig.yaml

Step 4: Create a GitOps application(as shown below) that points to the Git repository with crossplane manifests as source.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: example
  namespace: openshift-gitops
spec:
 destination:
   server: 'https://kubernetes.default.svc'
 project: default
 source:
   path: s3
   repoURL: https://github.com/iam-veeramalla/crossplane-gitops-poc

Step 5: Allow a couple of minutes for the system to pull the changes from Git and create the infrastructure on your favourite cloud provider.

Step 6: Make some changes to the Git repository to see if the changes are updated.

Step 7: Make some unwanted changes to the cloud provider to see if the changes are auto corrected by the system.

Step 8: Make some unwanted changes to the crossplane manifests on the kubernetes cluster to see if the changes are auto corrected and replaced with the manifests in Git by the system.

Here is the video edition of the blog.
https://youtu.be/6qLbgPl8u1U