Visibility: A Technical Chauffeur of Data, Part III
January 23, 2018
This is part 3 of 4 in a blog series on key trends in securing the public cloud.
Gaining visibility into the operational data you need is often underestimated when organizations deploy to the public cloud. It’s not like you can simply deploy a layer 2 span port and see all traffic in a single location for East-West and North-South. Firewall-logs will simply not get you enough information to make meaning of the data.
The over-promised land
Even more difficult is the ability to query the data and make meaning of it. The promise of the SIEM was to take disparate pieces of information and gain insights through query and correlation. The reality, however in my experience, is that organizations rarely get to said promised-land. The very mention of a SIEM project typically results in a smile, wink, and sometimes a frown in a project that never got completed or is “currently being looked into.”
Collect and query
In security, it’s basically understood that the more you collect the more you know. The unfortunate reality is that it should be more like: the more you can query the more you know. While collecting the data is critically important you have to be able to query it in a reasonable amount of time. Read, near real-time.
With respect to AWS and collection, this typically starts with basic monitoring of access and API information for compliance, goes to a more sophisticated collection of commonly used APIs, network collecting, and then host. I made a low fidelity chart to highlight the breadth and depth of insights, collection, and efficacy below. We believe that if you want to truly be secure you need to be as close as possible to the top right in the curve.
Now that you have the data you need to be able to query it. This is typically where things get more difficult. What type of data store do you deploy? Do you need to index the data? How do you know what to index? Do you need an API for the data? Do you build or buy technology to store and query? The good news is that there is an increasingly good set of options to accomplish this. The bad news is that there is an increasing number of options.
Security as a query
We are strong believers in abstracting all of this complexity behind an easy to use query layer with APIs and a UI so users can access the data with near-real-time query response. Additionally, the data must be formatted and stored in a way that accounts for the unknown. This is a LOT harder than it sounds and hard to account for.
In the last part of this series, I will examine the final key trend around context. Getting the right data with a queryable interface is key but context is king.