In this edition of Up and Running with Lacework, we dive into HashiCorp’s Packer to show you how to build base images from code that are free of vulnerabilities and validated with Lacework’s new Host Vulnerability Scanning.
The move to immutable infrastructure has been gaining momentum over the last few years, with more and more companies adopting a pattern where servers are treated as cattle rather than pets. Do you have a new version of your software that you want to deploy on an instance? Build a new virtual machine with that software already installed. Got vulnerabilities? Build a new base image with those vulnerabilities patched! There should be no argument that if you want to do immutable infrastructure, then you are going to need AUTOMATION, and a lot of it!
When it comes to automating base images for your cloud environment, one tool often rises to the top of the list, and that is Packer from HashiCorp. In their own words…
“HashiCorp Packer automates the creation of any type of machine image. It embraces modern configuration management by encouraging you to use automated scripts to install and configure the software within your Packer-made images. Packer brings machine images into the modern age, unlocking untapped potential and opening new opportunities.”
The TL;DR is that you can codify the steps to create base images for your public and private cloud environments, and automate the delivery to those images using a unified workflow.
Security and Packer
There is a ton of security that comes from using Packer just in the fact that you have an opportunity to set up a canonical source of truth for how your base images are created. By storing their configuration in source control, you can define who has access to those configurations, who can contribute, how contributions are reviewed both through peer review and through the use of automated testing, and who has the permissions to merge new changes. This in itself also provides auditability because you have a record of who changed what and when. You can see who approved those changes, and what tests were run. And speaking of tests, how about automated security testing for things like host or container vulnerabilities?
Lacework and Packer
Lacework recently announced a new Host Vulnerability feature in the platform which provides unparalleled visibility into the vulnerabilities of your cloud environments. That visibility includes not just the ability to find vulnerabilities, but to also filter down whether the specific packages with vulnerabilities are active within your environment, and if those hosts are online or offline.
Additionally, the new Host Vulnerability feature has an API that allows you to scan a package manifest (a list of installed packages on a host and their versions), and Lacework will return the vulnerabilities found. We recently updated the Lacework command-line interface (CLI) to provide a simple workflow for generating a package manifest on any of the Linux distributions we support. You can now also submit that package manifest directly to Lacework to return a vulnerability assessment.
This use case just happens to align perfectly with Packer in that we can build images for any cloud, generate a package manifest and test that image for vulnerabilities before deploying the image. From there, your teams can have the data to decide whether to fail a build, or provide a waiver or exception to let the build proceed.
In this tutorial we will show you how easy it is to get Lacework integrated into a Packer build so you can start scanning your images for vulnerabilities earlier in the build process.
For this tutorial you will need the following:
- A Lacework Account
- Packer (v.1.6.x)
- AWS Account – To publish AMIs
- AWS CLI – Packer leverages AWS APIs so you should have the CLI installed and configured to your account
Create Lacework API Key
Before we can even begin building hosts with Packer and requesting scans of those hosts with Lacework, we are going to need an API key and secret from Lacework.
- Login to your Lacework account
- Click on Settings → API Keys
- Click CREATE NEW API KEY
- Give your new API Key a Name and an optional Description and then click SAVE
- Click the Download button and save the JSON file somewhere as we will need it to configure Packer
The contents of your API key contain a “keyId” and “secret” and will look something like this…
Clone Example Source Code
We have supplied a git repository with an example Packer template to get you started.
- Open a Terminal of choice and run the command:
git clone https://github.com/lacework/up-and-running-packer.git
- Change directories into that folder (
Overview of Packer Templates
The repository contains two Packer JSON templates that can be used to test integrating Lacework with Packer. One template intentionally does not patch vulnerabilities to help you better understand the data that is returned when vulnerabilities are found. Another template does patch the vulnerabilities test result when no vulnerabilities are discovered.
- aws_rhel8_base_no_vulns.json – This template builds a Red Hat Enterprise Linux 8 base image from Red Hat’s latest RHEL8 image from the Amazon Marketplace. As part of the build process, there is a step to run a
yum update -yfollowed by
yum upgrade -ywhich upgrades all of the packages to the latest versions
- aws_ubuntu1804_base_vulns.json – This template builds an Ubuntu 1804 base image from Canonical Ubuntu1804 image from the Amazon Marketplace. This image purposefully does not upgrade any of the base packages to illustrate what the data looks like
Both templates use Packer’s shell provisioner to execute commands on the instance. The following is a snippet from the aws_rhel8_base_no_vulns.json:
The example above configures environment variables to set the Lacework CLI, runs an upgrade of the packages on the host, installs the Lacework CLI, runs a vulnerability assessment from a generated package manifest from the instance and then uninstalls the Lacework CLI before publishing the image.
Configure Environment Variables
Both Lacework and AWS API keys are secrets and should be treated as such. Rather than hard-coding secrets into Packer templates, Packer supports the usage of environment variables. The reference Packer templates use environment variables as follows:
AWS stores API keys and secrets configuration in your home directory in a
~/.aws directory in a file called ‘credentials.’ You may have multiple profiles configured depending on the number of accounts in your environment, and can choose the appropriate profile you want to publish the image to.
Linux and macOS
$ export AWS_PROFILE='<YOUR AWS PROFILE>' # ~/.aws/credentials
$env:AWS_PROFILE='<YOUR AWS PROFILE>’
With the environment variables configured, it is time to kick off a build. Let’s first look at a build of the Ubuntu 18.04 image where vulnerabilities are detected:
- Open a Terminal and cd into up-and-running-packer directory
- Run the following command:
packer build aws_ubuntu1804_base_vulns.json
The results return all of the vulnerabilities found on the base image, which shows numerous vulnerabilities of varying degrees of criticality, some with fixes and some without. Determining when/if to break a build is something that must be assessed by each organization. Luckily Lacework provides a ton of context about where your services run, what they communicate with, and whether those vulnerabilities are actually active in your environment, which we touched on in the Up and Running with the Lacework CLI.
If you are interested in seeing a Packer build with no vulnerabilities, then you can just run a build of the aws_rhel8_base_no_vulns.json with the command:
packer build aws_rhel8_base_no_vulns.json
Immutable infrastructure provides profound benefits for teams looking to move fast and efficiently, but doing so does not need to come at the expense of security. Integrating Lacework with Packer gives teams the opportunity to catch vulnerabilities before images are published or deployed. Hopefully this tutorial inspired you to get Lacework and Packer up and running together in your environment!
Stay tuned for more from Up and Running with Lacework!