In this tutorial, we are going to do learn to implement CI/CD on AWS. First, let's have a quick overview of CI/CD on AWS.
What is CI/CD on AWS?
- As we know that one of the main parts of DevOps is adopting the culture of continuous integration and continuous delivery/deployment (CI/CD). CI/CD on AWS can be pictured as a pipeline, where new code is submitted on one end, tested over a series of stages, and then published as production-ready code.
- CI/CD on AWS is implemented using AWS CodePipeline where it utilizes a continuous delivery service that models, visualizes, and automates the steps required to release software by defining stages in a pipeline. The code that successfully passes through all the stages defined will be deployed. We can also add other requirements to our pipeline, such as manual approvals, to help ensure that only approved changes are deployed to production. We will be learning more about CodePipeline in the next tutorial.
Implementing CI/CD on AWS
In this tutorial, we are going to discuss the following two services that can be used while implementing CI/CD on AWS. Our goal is to promote a code commit or change to pass through various automated stage gates all the way from development to production environments.
- AWS Code Commit
- AWS Code Build
We are going to discuss both of them one by one along with practical examples. Let's have a quick overview of AWS CodeCommit that will be used while implementing CI/CD on AWS.
AWS Code Commit overview
- AWS CodeCommit is a fully-managed source control service that makes it easy for companies to host secure and highly scalable private Git repositories.
- With AWS CodeCommit, there is no need to operate our own source control system or worry about scaling its infrastructure.
- It has a highly scalable, redundant, and durable architecture. The service is designed to keep our repositories highly available and accessible.
- CodeCommit is integrated with AWS IAM allowing us to assign user-specific permissions to our repositories. Note that it also automatically encrypts our files in transit and at rest.
- With AWS Code Commit, we can store any type of file and there are no repository size limits as well.
- With AWS CodeCommit, we can keep our repositories close to our other production resources in the AWS Cloud, which helps increase the speed and frequency of our development lifecycle.
- We can migrate to CodeCommit from any Git-based repository.
- It supports Git commands as well as its own AWS CLI commands and APIs.
AWS Code Commit Hands-On
First, we will log in to our AWS account and type CodeCommit under the Services tab.
Create Repository
Here we will click on Create repository.
Next, we will provide the Repository name i.e. go-linux-cloud-app. We can also provide the description and add tags to the repository. Then we will click on Create.
Upload File
The repository is currently empty. We can add files to it directly from the console or by cloning the repository to our computer. We will see both in the tutorial. Here in Add file, we will click on Upload file.
After uploading the file. We will provide Author name, Email address, and Commit message and then click on Commit changes.
This is our index.html uploaded to go-linux-cloud-app repository. We can also edit it here but it is not considered a good practice.
For the repository just created, we have other options as well such as
- Pull requests where we can create pull requests.
- Commits where we can view a list of commits made to a branch in a CodeCommit repository, view a graph of the commits made to our repository, and compare different commits as well.
- Branches where we can create other branches to store code changes that we don't want to include in the default branch i.e. main.
- Git tags where we can view Git tags for a repository. Note that we can use Git tags to mark a commit with an identifier that helps other repository users understand its importance.
- Settings where we can do General, Notifications, Triggers, Repository tags, and Amazon Cloud Guru Reviewer-related configurations.
Create Notification rule
Now we will create a Notification rule for go-linux-cloud-app repository. From the left pane, we will click on Settings. Then we will click on the Notifications tab. After that, we will click on Create notification rule. Here we will set the Notification name to my-demo-notification-rule, then we will choose Detail type i.e. the level of detail we want in notifications, and set it to Full. Then we will select the events that will trigger notifications. Here we will click on Select all. We can select events according to our requirements.
Now we will create a target to use specifically for this notification rule. Here we have two options to choose from SNS Topic or AWS Chatbot(Slack). We will choose SNS topic. Note that SNS topics created as targets have no subscribers but have all policies applied to act as a target for notifications. We will set our SNS topic name to codecommit-demo. Then we will click on Create.
Here we will click on Submit to create our Notification rule.
Create Trigger
Now we will switch to the Triggers tab and click on Create trigger.
Here we will provide the Trigger name i.e. my-demo-trigger. Then we will select the Events for the trigger. Here we will select Push to existing branch. There are other options available as well such as All repository events, Create branch or tag and Delete branch or tag. Then we will specify the Branch name on which the trigger should be applied. Here we will select main. Then under Service details, we will choose the service to use. Here we will select Amazon SNS. Then we will choose a topic name that we created earlier. And in the end, we will click on Create Trigger.
Connect to CodeCommit using Git client
We can also connect to CodeCommit from our computer as we discussed earlier. For that, we will go to our IAM user and generate credentials for the user. Following is the IAM user that needs to connect to CodeCommit from a local computer.
First, we will click on the username i.e. golinuxcloud. Then we will switch to the Security Credentials tab. Here we have two options to connect to CodeCommit.
- SSH keys for AWS CodeCommit.
- HTTPS Git credentials for AWS CodeCommit.
We will be using HTTPS Git credentials. Here we can generate a user name and password that can be used to authenticate HTTPS connections to AWS CodeCommit repositories. We will click on Generate credentials.
Our Credentials are available. We can download them or even copy them.
Now we will get back to our Repository's Code section and then click on Clone HTTPS under Clone URL. On the top, we can see the URL that we will be using to access our repository in CodeCommit.
Now we will enter the following command in cmd of our computer.
git clone https://git-codecommit.us-east-2.amazonaws.com/v1/repos/go-linux-cloud-app
Next, we will enter the credentials that we have just downloaded above.
Now we are all set to use the regular git commands with our repository.
Next, we will have a quick overview of AWS CodeBuild that will be used while implementing CI/CD on AWS as it belongs to a family of AWS Code Services which can be used to create complete, automated software release workflows for continuous integration and delivery (CI/CD).
AWS Code Build overview
- AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
- With CodeBuild, we don’t need to provision, manage, patch, update, and scale our own build servers.
- CodeBuild scales continuously and processes multiple builds concurrently, so our builds are not left waiting in a queue.
- With AWS CodeBuild, we no longer have to worry about paying for idle build server capacity as we are charged based on the number of minutes it takes to complete our build.
- With AWS CodeBuild, we can use our own build tools and programming runtimes by creating customized build environments in addition to the prepackaged build tools and runtimes supported by CodeBuild.
AWS Code Build Hands-on
Now from the left pane, we will click on Getting Started under Build. Then we will click on Create project.
Create build project
Here we will provide Project name i.e. MyGoLinuxCloudBuildProject. We can also provide the description. In our build project, we will be testing whether our index.html residing in the CodeCommit repository contains GoLinuxCloud text or not.
Then we will select the Source provider for our build project which in our case will be CodeCommit. Then we will select the repository i.e. go-linux-cloud-app. Next, we will select the Reference type i.e Branch. For Branch, we will select main that contains our code to build.
For the Environment image, we will select Managed image which is an image managed by AWS CodeBuild. For Operating System, we will select Ubuntu. For Runtime, we will select Standard. Then for Image, we will select aws/codebuild/standard:5.0 (select latest). For the Image version, we will select Always use the lastest image for this runtime version. For the Environment type, we will select Linux.
Now we will create a new service role for our code build project. We can also do Additional configurations related to Timeout, certificate, VPC, compute type, environment variables, file systems but we will leave them to default for now. For Build specifications, we will select Use a buildspec file. Note that if we do not provide a file name here then CodeBuild looks for a file named buildspec.yml in the source code root directory. Next, we will leave everything else to default for now and click on Create build project.
buildspec.yml
Now let's upload a buildspec.yml file in the root directory of our repository. Here we have added commands for the install, pre_build, build, and post_build phases. In the build phase, we can see that if GoLinuxGloud exists in index.html the test is going to pass otherwise it's going to fail.
This is what our index.html
looks like for GoLinuxCloud.
Start build
After clicking on Start build, we can see the build status was set to Succeeded which means that GoLinuxCloud was there in index.html.
Now we see under Phase details all the phases after the build was started.
When we edit index.html and remove GoLinuxCloud from there the output is as expected i.e. the build failed.
If we look at the phase details we can clearly see that the error occurred while executing the grep command in the BUILD phase.
Summary
With this, we have come to the end of the first part of getting started with CI/CD on AWS. In this tutorial, we learned about two AWS services named AWS CodeCommit and AWS CodeBuild that can be used while implementing CI/CD on AWS. We also did a hands-on on AWS CodeCommit and AWS CodeBuild that we will be using in the next tutorial while implementing CI/CD on AWS.
In the next tutorial, we will be discussing CodeDeploy and CodePipeline along with practical examples to implement CI/CD on AWS.
Feel free to leave your valuable feedback in the comments section.
Happy learning!
References