Two Sides to the Coin

Sometimes, it seems that talk of “big data”, “security analytics”, and “big data security analytics” can dominate discourse within the information security profession.  This tends to produce a confusing and somewhat overwhelming environment for the enterprise buyer, where all of the words and ideas can begin to blend together.  Since I spent over a decade on the enterprise/operational side before moving to the vendor side, I can sympathize with the confusion this can bring to the enterprise audience.  Leaders in the enterprise have many responsibilities, and it is difficult for them to keep track of the large number of vendors and what each vendor's specialty is.

Many enterprises see the need and share a desire to be doing "big data" and "security analytics", and thus, it's not particularly surprising that many vendors are offering "big data" and "security analytics" solutions.  But what does it actually mean to do "big data" and "security analytics"?  I think it's helpful to take a step back and think a level deeper about this in order to better understand it.

At a high level, "big data" and "security analytics" are about the two very different, but equally important concepts of collection and analysis.  Allow me to explain.  Before it is possible to run analytics, one needs the right data upon which to run those analytics.  Before "big data" emerged as a buzzword, this was called "collection" or "instrumentation of the network and endpoint".  Further, in order to run analytics, one also needs a high performance platform upon which to issue the precise, targeted, incisive queries required by analytics.  Before "security analytics" emerged as a buzzword, this was sometimes called analysis or forensics, among other terms.

Collection and analysis, at enterprise speeds, are both equally important. If you think about it, you can't really have one without the other.  Or, to put it another way, what good does the greatest collection capability provide without a way to analyze that data in a timely and accurate manner?  Similarly, what good does the greatest analytical capability provide without the underlying data to support it?

In addition to being the elements of big data, collection and analysis form the cornerstone of a strong security program.  Collection and analysis provide an organization with the visibility required to practice Continuous Security Monitoring (CSM).  Although a detailed discussion of CSM is beyond the scope of this post, the topic has been discussed at length by NIST, SANS, Gartner, and others.  The goal of CSM is to allow an organization to move rapidly from Detection to Analysis and on to Containment and Remediation.  In other words, CSM facilitates and enables the incident response process and life cycle.  An organization’s ultimate goal, when prevention efforts fail, is to detect and respond to intrusions before they cause damage to the organization.

Continuous Security Monitoring involves many details.  Here are some thoughts on high level guidelines around strategic steps organizations can take in the area of CSM to improve their information security postures:

• Identification of business risks and concerns to be addressed through Continuous Security Monitoring
• Creation of goals and priorities based on business risks and concerns
• Identification of the least number of data sources of highest value that provide the required visibility across the enterprise
• Collection of relevant data sources
• Exposure of the collected data with sufficient performance to facilitate Detection, Analysis, Containment, and Remediation
• Development of content and logic leveraging the collected data to supply the work queue with high fidelity alerting
• Development of process for investigation and response

While it is tempting to collect all of the available data within the enterprise, this actually works against the interests of the security organization.  It is prudent to ensure that the minimal data that provides sufficient context and coverage is collected, but not more than that.  Collecting more data than required creates two issues:

• Analytical (query) performance degrades rapidly, making timely incident response nearly impossible
• Retention periods shorten, producing historical blind spots that impede response for long present intrusions

Big data is an interesting topic with the potential to be an incident response enabler.  It’s important to remember that big data involves two equally important but somewhat diametrically opposed interests – collection and analysis.  Both aspects are important, but they have a tendency to work against each other if left unchecked.  It’s important to remember the ultimate goal of collection and analysis, which is the enablement of timely incident response.  It is in this spirit that we aim to gain the most information from the smallest subset of data.  All the data in the world does you no good if you cannot leverage it in a timely manner when you need it most.  In incident response, less is more.