Up and Running with Lacework and AWS Code Build (Update)

In this installment of Up and Running with Lacework we demonstrate how to build a continuous integration pipeline with AWS Code Build that builds Amazon Machine Images (AMIs) using Hashicorp Packer that are patched for vulnerabilities and automatically tested by Lacework’s new Host Vulnerability scanning service. Additionally, we are going to cover how you can store secrets in AWS System Manager Parameter Store.

We covered this topic live on Twitch.tv back in October on the AWS “Howdy Partner” show with AM Grobelny


Continuous integration and continuous delivery pipelines (CI/CD) are considered core to the foundation of high performing cloud companies. Companies that invest in automation are reaping the benefits by delivering more value to their customers through continuous release of new features and capabilities.

While shipping products is the highest priority, the next highest priority of digital companies without a doubt is security.  Adopting CI/CD pipelines brings a ton of opportunity for collaboration amongst security, developers, and operations teams to inject security early in the software development lifecycle (SDLC).

Requirements

Before you begin you will need the following to complete this walkthrough:

  • GitHub Account
  • An AWS Account with administrative access to IAM
  • A Lacework account with administrative access to create API keys

Side Note: We’ll make a number of IAM permission changes in this guide. If lots of people on your team making IAM changes makes you nervous, you’ll want to investigate Lacework’s AWS Cloudtrail Activity Monitoring to let innovation flourish with full visibility of change and risks.

Summary Of Steps:

  1. Fork the Example Code Repo
  2. Setup AWS CodeBuild
  3. Update IAM Role policy for Packer
  4. First AMI Build with Packer and AWS CodeBuild
  5. Configure Secrets in AWS System Manager Parameter Store
  6. Add Lacework Vulnerability Scanning to the Build

Fork the Example Repo

Fork the repo on github

To make this integration work we have created a GitHub repo with all of the code you will need, but first you will need to login to your GitHub account, and fork the example repo to your own GitHub account.

  1. Login to Github
  2. Head to https://github.com/lacework/up-and-running-aws-codebuild
  3. Fork the repository to your own Github account

Setup AWS CodeBuild

Next up we need to set up Code Build. You will need your AWS credentials, and the link to the repository you cloned in the previous step. The following sections will use the us-east-1 region of AWS, but should work in any region.

 

  1. Login to the AWS Console and navigate to AWS CodeBuild
  2. Click Create Build Project
  3. For the Project Name use “AMI_Builder
  4. For the Source section choose “Github” as the source. Connect to Github using OAuth then authorize CodeBuild to access your personal GitHub org (You can revoke this at any time). For the Repository URL paste link to the example you forked in the previous step.
    Github OAuth setup
  5. For the Environment section use the default selection of Managed Image and choose “Amazon Linux 2” for the operating system. For Runtime(s) choose “Standard”, and for Image choose “aws/codebuild/amazonlinux2-x86_64-standard:3.0“. The Role Name should default to “codebuild-AMI_Builder-service-role“. Expand the Additional configuration section and scroll down to Environment variables and add a “Plaintext” variable of “AWS_REGION” with a value of “us-east-1
    CodeBuild environment setup
  6. For the remaining sections leave everything default for now and then click Create Build Project.

Update IAM Role Policy for Packer

In the previous step we created an IAM role for AWS CodeBuild called “codebuild-AMI_Builder-service-role“. That Role has a basic policy that provides the CodeBuild project to required resources. Packer however needs certain permissions to be able to make API calls, so we need to update the IAM Policy before we can build.

  1. Open IAM in the AWS Console
  2. On the right hand side Click on Roles and search for “codebuild-AMI_Builder-service-role
  3. Click on the Policy (should be “CodeBuildBasePolicy-AMI_Builder-us-east-1“)
  4. Click Edit Policy then choose the JSON editor
  5. Scroll down to line 50 and add a “,” after the “}” then paste the following snippet of JSON from the example repo
  6. Click Review Policy
  7. Click Save Changes

First AMI Build with Packer and AWS CodeBuild

In this section we are going to kick off the first build of a base image with CodeBuild just to test that images are publishing successfully to AWS.

Tail Logs

  1. Open the AWS Console and Navigate to CodeBuild
  2. Click on Build projects then click on the AMI_Builder project
  3. Click Start Build
  4. You can leave everything default and click on Start Build again
  5. Click Tail Logs to watch the build (Build may take between 5-10min)

Adding Lacework Vulnerability Scanning to the build

Now that AMIs are publishing successfully we can add some security to the build process and validate that our images are free of vulnerabilities.

In the build process we are already using the shell provisioner of Packer to run a yum update -y which will update all of the packages on the host.

We are also going to use the Lacework CLI to generate a package manifest (a list of installed packages and their version) and submit that to Lacework to scan for vulnerabilities. After returning the results we will remove the Lacework CLI from the build.

Create Lacework API Key

Animated gif of the Lacework console showing how to generate an api key

In order to authenticate with Lacework’s APIs for a vulnerability scan, you will need to create an API key and secret.

  1. Log in to the Lacework Console.
  2. Click Settings -> API Keys.
  3. Click CREATE NEW API KEY.
  4. Give the API key a Name and optional Description.
  5. Click SAVE.
  6. Click DOWNLOAD to save the API key file locally.

The contents of your API key contain a keyId, secret, subAccount, and account:

    
    {
      "keyId": "ACCOUNT_ABCEF01234559B9B07114E834D8570F567C824039756E03",
      "secret": "_abc1234e243a645bcf173ef55b837c19",
      "subAccount": "my-sub-account",
      "account": "my-account.lacework.net"
    }
  

Store API Key in AWS System Manager Parameter Store

The API Key for Lacework should be treated securely and never shared out. AWS CodeBuild supports the ability to pull secrets from the AWS Systems Manager Parameter store! We will use that to store the parameters for integrating with our build Lacework.

We need to configure the following parameters:

      
      LW_ACCOUNT (Plaintext): "Your Lacework Account"
      LW_API_KEY (Plaintext): "Your Lacework API Key ID"
      LW_API_SECRET (SecureString): "Your Lacework API secret"
      
  1. Login to the AWS Console and navigate to AWS Systems Manager
  2. On the left hand side click on Parameter Store
  3. Click Create Parameter
  4. For the Name use LW_ACCOUNT, for the Type use “String” and for the Value use the name of your Lacework account
  5. Click Create Parameter to store the parameter
  6. Click Create Parameter
  7. For the Name use LW_API_KEY, for the Type use “String” and for the Value use the API_KEY_ID from the JSON you downloaded in the previous step
  8. Click Create Parameter to store the parameter
  9. Click Create Parameter
  10. For the Name use LW_API_SECRET, for the Type use “SecureString” and for the Value use the API_KEY_SECRET from the JSON you downloaded in the previous step
  11. Click Create Parameter to store the parameter

AWS SSM parameter store setup

Update IAM Policy for SSM Access

In order for CodeBuild to pull parameters from the Parameter store we need to update the IAM Policy to provide access.

  1. On the Right hand side click on Roles and search for “codebuild-AMI_Builder-service-role”
  2. Click on the Policy (should be “CodeBuildBasePolicy-AMI_Builder-us-east-1”)
  3. Click Edit Policy then choose the JSON editor

Update the policy to include “ssm:GetParameters” The example repo contains the complete updated policy for reference.

Update the buildspec.yml to pull parameters

The buildspec.yml configuration file needs to be updated to pull in parameters and set environment variables.

  1. On your local workstation clone the example git repo you forked in the first step:
            
            $ git clone https://github.com/<your_user>/up-and-running-aws-codebuild.git
            

     

  2. Change directories into up-and-running-aws-codebuild

The env section at the top will set ENVIRONMENT VARIABLES that we need in the build process to scan the host for vulnerabilities

Commit and Push changes

Yes, pushing to master is not a good idea but for this exercise and in the interest of time we are going to do just that..

      
      $ git add .
      $ git commit 'updating buildspec with param store'
      $ git push origin master
      

New buildspec.yml should be ready to go to pull in params.

Update Packer Template to test for vulnerabilities

In order to test our build for vulnerabilities, we are going to use the Lacework CLI which will generate a package manifest and submit it to Lacework to scan the list of packages for vulnerabilities and return the results to AWS CodeBuild.

We are going to update our Packer template to leverage the ENV variables from the previous step so the Lacework CLI can authenticate with your Lacework account. The packer build will place a script on the host at build that installs the Lacework CLI, and then executes a lacework vulnerability host scan-pkg-manifest --local command which generates the package manifest, and submits it to Lacework to scan.

Once the results are returned we will uninstall the Lacework CLI from the host

  1. On your local workstation, open the amazon_linux_packer_template.json Packer template in an EDITOR of choice
  2. Update the packer template with the following contents from the amazon_linux_packer_template.json.example_with_lacework included in the example repo

Commit and Push changes

Yes, pushing to master is not a good idea but for this exercise and in the interest of time we are going to do just that again..

      
      $ git add .
      $ git commit 'updating packer template to test for vulnerabilities'
      $ git push origin master 
      

New packer templates should be ready to go!

 

AMI Build with Packer and AWS CodeBuild and vulnerability scanning with Lacework!

We are now ready to add vulnerability scanning to our build with Lacework

  1. Open the AWS Console and Navigate to CodeBuild
  2. Click on Build projects then click on the AMI_Builder project
  3. Click Start Build
  4. You can leave everything default and click on Start Build again
  5. Click Tail Logs to watch the build (Build may take between 5-10min)
  6. This time around you should see the results of the scan…

Packer output with NO VULNERABILITIES!

Nice work! The build has no vulnerabilities!!! Time to celebrate!

Conclusion

Integrating security testing into the SDLC is not a nice to have anymore, it is a must! If you are launching VMs in the any of the cloud providers then you absolutely should be concerned about the images you are launching, and ensure they are patch for vulnerabilities. AWS CodeBuild and Hashicorp Packer are certainly one way to start creating trusted builds for your organization, but the patterns shown here can certainly be achieved with other tools. Hopefully this gives you some inspiration to get this pattern going in your environment!

Happy automating!