If You Can See It, You Can Secure It: Anomaly Detection in the Cloud

No matter what you sell to customers, you are in the data business. Data is used to help your people make better decisions, deliver better products and services, and maintain competitive advantages. The trove of data you’re sitting on includes all kinds of private information, including payment card details, employee records, health data, and various forms of intellectual property. For obvious reasons, it’s highly sought by attackers but, unfortunately, data leaks happen far too often and organizations are not always sure how to protect against them.

There are countless prescriptive lists that define best practices for protecting data in the cloud but breach prevention can’t be managed simply through a checklist. Data leaks have been around as long as people have transcribed and stored information, and because it’s valuable, bad actors will continue to find new ways to infiltrate.

Attackers don’t always use complex approaches to ply their trade; they rely on a combination of human engineering with knowledge of how IT environments are structured. The problem is that the goals of the attacker are different from those of the protector. The hacker is organized and structured, but only up to a point. Protectors, as organized as they are, just can’t keep up with the volume and randomness of attacks unless they apply automated anomaly detection to the task.

Some cloud security solutions use rules to define what is and what isn’t considered acceptable behavior within the cloud environment. The attacks that occur in a rules-based system, however, are hard to define because effective breaches are often based on impersonation of something legitimate. Take unauthorized access, for example. Employees leave their jobs, but organizations don’t follow a discipline to remove them, so dead accounts are actually active. Since those accounts have access to cloud resources, someone impersonating credentials is able to operate freely within the environment while looking like they’re legitimate.

With automated anomaly detection, however, you will always be notified of events that are not normal to your environment. This ensures that you are notified of the activity that you want to know about without having to sift through all the noise. In the case of dead accounts, anomaly detection will recognize and report on accounts that are suddenly active after periods of inactivity. They can identify when abnormally large amounts of data are extracted from databases, or if users are accessing from unusual IP addresses.

Security teams are overwhelmed by the magnitude of the environments they manage so they have to rely on shortcuts like dashboards and logs to make sense of activity and, specifically, if that activity is threatening. Security dashboards are typically fueled by data that are generated from a rules-based approach where the activity that runs counter to structured rules is flagged through alerts. This approach limits visibility, and you can’t secure what you can’t see.

Human activity doesn’t scale to meet these demands, nor can it adapt to the complexity required to continuously update rules, and organizations need to know their security posture is aligned with how fast they need to move. This may be the time to go with the new approach of using automated anomaly detection to identify bad actors within your environment.

The Lacework approach removes the rule-writing element because of our unsupervised machine learning that performs automated anomaly detection. Once the product is deployed, it begins to learn and understand your environment by analyzing data from your cloud accounts and workloads. From here it creates a baseline and automatically alerts you of any anomalous behavior. You’re getting value almost immediately, and you don’t need to wait to determine if your rules are working. You’ll just have to identify the resources that you would like to monitor and allow the product to do the rest.

Lacework’s foundation is Polygraph, a deep temporal baseline, which we present to users as a set of behavioral maps or “Polygraphs”. With the complexity and volume of a modern data center, Polygraph employs a baseline, zero-touch approach without leaving any blank spaces where attackers can hide.

Figure 1. Lacework’s Polygraph analyzes events and processes to detect behavioral anomalies

Lacework decomposes the data into the lowest possible isolation unit that an OS supports: a process. Everything that happens to and with data happens through a process. Every app has a different process and processes are not mixed between apps. When it comes to baselining a complex set of data, they are the ideal unit for detecting vulnerabilities because:

  • They can be validated – every process is associated with a specific binary that has a particular SHA-256 hash.
  • They are traceable – processes are launched by users, applications, or other processes and we can keep track of how they started.
  • They are predictable – a process has a particular command line, purpose, and a life cycle.
  • They are responsible for all communications – processes, and only processes, communicate with each other and with external hosts on the Internet.

The solution to preventing data leaks requires continuous monitoring of inter-process activities, even those occurring inside the same file. Enterprises need a host-based intrusion detection system designed to monitor process hierarchy, process and machine communications, any changes in user privileges, internal and external data transfers, and all other cloud activity. An effective system looks across all layers and it analyzes activity based on normalized behavior, which gives a continuous real-time view even across short-lived services that may only exist for a few minutes. Having that process-to-process visibility is a critical factor in having strong, effective security built into any cloud environment.

Photo by Dylan Freedom on Unsplash