Lacework introduces new Kubernetes Audit Logs monitoring
Julien Sobrier and Joe Goldberg
17. Mai 2022
Lacework has always offered strong protection for your Kubernetes environment in the cloud and it just got better with a new feature, Kubernetes Audit Logs Monitoring. This new feature integrates with the Kubernetes API and greatly increases your insight into Kubernetes user activity and actions that lead to better detection of cyberthreats, misconfigurations, and areas of non-compliance in your deployment.
Lacework for Kubernetes 101
Kubernetes, or K8s, is a critical infrastructure layer in most cloud deployments as the leading container orchestration and management technology. While Kubernetes offers substantial operational benefits, it also needs to be properly protected and secured to prevent a cyber threat from compromising containers and the workloads within them. This compromise could lead to nefarious activities such as cryptomining, or finding and exfiltrating sensitive data. To prevent this, robust Kubernetes security is needed so well-intentioned insiders do not introduce misconfigurations into containers, and also to detect external threats accessing containers and the data within them.
Traditional security solutions struggle with Kubernetes visibility as they were not built to handle its complexity, the temporal nature of containers and their rapid scaling up and down, or the sheer volume of events they generate. They also often cover limited stages of the overall Kubernetes deployment process so they have coverage gaps. These legacy solutions also are not embedded into developer or DevOps workflows, so Security teams cannot “shift-left” security too early in the Kubernetes build process. Lastly, they often rely on manual rules to detect threats which results in too much time spent writing and maintaining rules, excessive false positives, and an inability to detect anomalous, or unknown, threats manual rules cannot detect.
Enter Lacework, built day one for cloud security with the ability to handle the complex and temporary nature of containers and cloud resources at unlimited scale. We offer comprehensive visibility, threat detection and alerts, and compliance checks on your entire Kubernetes environment. This protection spans all the stages of Kubernetes cloud infrastructure deployment, from build-time to runtime. We also provide automated, embedded capabilities into developer workflows, enabling security to “shift-left” while not getting in the way of development efforts. Lastly, Lacework offers automated anomaly and threat detection with our patented Polygraph unsupervised machine learning, so little to no time is spent writing rules and false positives are rare.
Our overall capabilities and features include:
- Runtime monitoring of containers, clusters, nodes, and OSes via our agent which is supported in many Kubernetes environments.
- Visibility on Kubernetes-related cloud provider activity such as encryption, container registries, load balancers, and more. This is done via our agent-less ability to ingest cloud provider configuration and audit logs.
- Checking image configuration and scanning for vulnerabilities on Kubernetes containers before they are pushed to production, and optionally blocking insecure containers from being deployed. This is done via our Kubernetes Admission Controller
- Review and fix Infrastructure as Code (IaC), including Helm charts used to manage K8s applications and services, as it is written by developers to ensure it does not introduce a downstream misconfiguration into the Kubernetes environment. This is done via Lacework Infrastructure as Code (IaC) Security.
- NEW! Kubernetes Audit Log Monitoring which is shown in the red box in the diagram and is detailed in the next section.
FIGURE 1: Lacework protects across all stages of Kubernetes usage
New! Kubernetes Audit Logs Monitoring
We are excited to announce our robust Kubernetes protection just got better! We now have in limited availability New! Kubernetes Audit Logs monitoring, which allows our customers to ingest Kubernetes audit logs from the Kubernetes API so all users, events, authentications, and API activities in the Kubenetes environment can be ingested and used by Lacework for more visibility, better automated threat detection by our patented Polygraph machine-learning technology, additional compliance checks, and more robust reporting.
Complementing Polygraph and its anomaly detection are over 40 out-of-the-box Lacework policies specific to Kubernetes and powered by the Lacework Query Language (LQL) to detect misconfigurations and create an audit trail of important events. Custom policies can be optionally created as well. And by ingesting and monitoring your Kubernetes audit logs, you will align to audit logging guidance from many compliance and best-practice frameworks including the NSA & CISA Kubernetes Hardening Guide lists, NIST 800-53, CIS Benchmark for Kubernetes, PCI DSS, SOC 2, and ISO 27001.
Also important is that this new feature is easy to deploy with no pod or agent needed. It is a simple, UI-based process to make the Kubernetes audit logs available to Lacework, resulting in a time-to-value measured in minutes.
Previously, it was difficult for organizations to get value from Kubernetes audit logs. These logs were typically stored on cloud storage and then sent to a SIEM, where manual detection rules had to be built which were time-consuming and resulted in a high false-positive rate. Also, the sheer volume of events and expensive SIEM storage costs made it cost-prohibitive to store these events in a SIEM. With Lacework it is the opposite. Kubernetes threat detection requires no rules to build and is highly-accurate. Additionally, audit logs can remain on cloud provider storage buckets where only one copy is needed and storage costs are significantly lower.
Lacework Kubernetes Audit Log monitoring is in limited availability today for all customers for Amazon Elastic Kubernetes Services (EKS). General availability of this feature for all customers should come within the next few months. Support for the similar offerings of Azure Kubernetes Service (AKS) and Google Kubernetes Engine (GKE) are coming later. Contact Sales to learn more, see a demo, or get access to Kubernetes Audit Log monitoring for EKS.
FIGURE 2: Example of the Lacework Polygraph combining Kubernetes audit log events and events from other data sources to automatically identify anomalous activity related to Kubernetes.
FIGURE 3: Some of the pre-built policies to detect misconfigurations or suspicious activity specific to Kubernetes
Information in the EKS Audit Logs and why it is critical for security and compliance
The audit logs contain all the activities happening in Kubernetes, including user activities, background activities such as health checks, deployment of workloads or Kubernetes RBAC, authentication and authorization checks, etc. There are three main types of activities that are of special interest.
The first type of activities is around manual user activities. These activities are usually ephemeral and impossible to track without the audit logs:
- Manual access to containers using kubectl exec/debug/attach/port-forward
- Manual creation of pods, especially ephemeral pods
- Manual deletion of resources, especially accidental deletion of a namespace and all its resources
FIGURE 4: The Lacework Polygraph shows a user accessing 4 containers with the kubectl exec command.
Lacework allows you to track who did what at any time, including showing the AWS identity behind a Kubernetes user.
The second type of activities that are of special interest is the deployment or update of Kubernetes resources: workloads, RBAC, configmaps, network services, etc. Since the audit logs can include the full definition of resources created, a security platform can check for misconfiguration or unsafe definitions of workloads and RBAC for Kubernetes, such as:
- Workloads running as root, privileged or with dangerous capabilities
- Workloads with broad access to the underlying node – sensitive file system mounts, shared IPC, shared PID, shared network
- New container registries being used
- New workloads deployed to the kube-system or kube-public namespace
- New Cluster role with access to all resources and/or all actions
New bindings to the groups cluster-admin, admin, etc.
FIGURE 5: Lacework event for the creation of a new Cluster Role that uses wildcards (*) to give access to all resources
The third type of activities are errors that might be caused by wrong permissions, lateral movement by a threat, or discovery in a cluster:
- Access to secrets denied
- Creation of a pod denied
- Invalid API call
- Activities from unauthenticated users
- An increase of the error rate
- API calls blocked by an admission controller
FIGURE 5: List of API call errors
Conclusion and Next Steps
Kubernetes audit logs are a great source of data to give you the insight needed to properly detect threats, suspicious behavior, misconfigurations, and areas of non-compliance in your Kubernetes environment. With our new ability to easily leverage Kubernetes audit logs from cloud providers, starting with Amazon EKS, you can rest easier with the insight and detection you need to secure your Kubernetes environment!