Lambda CI/CD Pipeline using AWS CodeBuild

Most of us, AWS and DevOps Architects, have been there, creating a Jenkins Pipeline and in the process try to find appropriate Jenkins plugin(s), deal with slow/stopped slaves by spinning up additional Jenkins slaves and create Cloudformations and Ansibles to install the code. I certainly use Jenkins a lot and in some cases it gets the job done quite well, however recently I was preparing for a demo on serverless security for a convention and created this Lambda Code Pipeline using only AWS Services which seems to run faster and has less moving parts, hence decided to share with you guys. The advantages are below:

1. No overhead of creating and maintaining Jenkins Master and Slaves (EC2s) saving you $$$
2. AWS Service ensures scalability and high availability (and in case of issues you take it to AWS directly rather than 3rd parties)
3. Simpler learning curve and faster to test
4. Let's admit it guys, it is super meta to use Lambda to deploy Lambda!

How does it work

Quite straightforward, above is the design diagram, below are steps with the github components, don't forget to check out the demo video (link in next section) to see the whole thing in action:

1. Git repository - Code repository where your Lambda resides along with buildspec.yml (more on this on #2 below). https://github.com/saha-rajdeep/github-codebuild
2. AWS Codebuild - This is where your code gets built and packaged. This is a python code build example. There are three parts to it - what to build, how to build and where to store outputs

buildspec.yml file on your git tells CodeBuild what libraries to install. You can run any pip install command for your code. Check the builsdpec.yml file from my git for reference - https://github.com/saha-rajdeep/github-codebuild/blob/master/buildspec.yml.

3. Once the zip package goes the S3 bucket, trigger a deployment lambda to grab that zip and deploy the lambda from the zip. Deployment lambda - https://github.com/saha-rajdeep/Lambda/blob/master/lambda_deployer.js.

Please note - this is quite simple version for demo, currently it just creates the Lambda, you can enhance it to update the lambda as well if lambda already exists. I grabbed the original Lambda from this blog: https://aws.amazon.com/blogs/compute/new-deployment-options-for-aws-lambda/ and tweaked a little.

Demo in Action

Please check out the video below to see the whole thing in action.

Hopefully this gives you ideas about moving some parts of your DevOps toolchain to AWS. As I mentioned before, this is for a bigger Serverless Security talk (reason why the video ends kind of abruptly), once the actual conference is done, I'd share the slides and full video highlighting some interesting aspects of Serverless Security. Till then, Happy Experimenting!