A Business Take on Heartbleed

I spent some time today reviewing blogs and articles from the past week. Not surprisingly, much of the content was dominated by Heartbleed, the OpenSSL vulnerability dating from 2011. There were many different perspectives taken on the Heartbleed vulnerability across a variety of different forums. From my perspective, one vantage point that seemed to be missing was the business take.

I thought it would be a useful contribution to the blogosphere to take the reader through the business perspective on Heartbleed. For the sake of this blog, let's put ourselves in the shoes of an organization that has recently learned of the vulnerability and seeks to assess its risk, damage, and liability. Though individual organizations may differ in their exact steps, the following process may help give the reader a general understanding of the approach taken in response to the Heartbleed vulnerability at a high level:

1. The first step after learning of the Heartbleed vulnerability is to identify vulnerable servers within the organization. In other words, before we can respond, we must understand what we are responding to. If accurate and detailed server inventory information is available, an organization could identify vulnerable servers from this information. More likely, the organization will need to perform a vulnerability scan across all server networks to identify any vulnerable servers. This is predicated on the assumption that the organization knows where all of its servers are located, including any located in outsourced, hosting, or cloud providers. In practice, unfortunately, this turns out to be a fairly large assumption.
2. Once vulnerable servers have been identified, the organization will need to determine for how long each server has been vulnerable. In the case of Heartbleed, each server may have been vulnerable for two years or longer. The length of time that each server was vulnerable provides us with important time bounds around our investigation.
3. For each vulnerable server, a damage assessment will need to be done. In other words, the organization will need to understand if the vulnerability was actively exploited, and if so, what information was compromised. Further, the organization will need to understand if that information was used to gain access to additional information. For example, if passwords were stolen and subsequently used to log in to users’ accounts, that could potentially compromise all information available through that specific application for those users. In order to perform a damage assessment, network forensics will need to be performed. Network forensics allows us to study the network traffic data of record and understand the history of what occurred, when it occurred, and how it occurred. Hopefully the organization has:
1. Its server networks properly instrumented for collection
2. A period of retention long enough to allow the organization to properly assess the damage
3. The capability to analyze all of that data.
4. Once the damage has been assessed, notifications may need to be made per legal or other requirements. Incident responders will need to work with management, executive, legal, and privacy stakeholders to understand what actions need to be taken. An accurate, factual, non-assumptive assessment of the damage will need to be relayed to the stakeholders so that they can determine what subsequent actions need to be taken.  In my experience, in the absence of factual information (e.g., where it is not possible to perform network forensics to obtain ground truth), most organizations take a conservative approach and assume everything that could have been compromised has been compromised.
5. Obviously, patching (remediation) is a large part of the Heartbleed response. Patching can be performed in parallel to the damage assessment. Since we are dealing with live, production servers, the patching process will require a fair bit of coordination.
6. Lessons learned are always an important part of the incident response process, and the Heartbleed response is no different. In my experience, many organizations will learn that:
1. They have server networks that they were not previously aware of
2. Some or all of their server networks were not properly instrumented for collection
3. Their data retention period is not long enough to meet the needs of the Heartbleed response
4. They do not have the analytical capability to properly perform network forensics

If any or all of these are the case, steps can be taken to ensure that the organization is better prepared for the next incident response.

Heartbleed provides us with a unique use case with which to peer through the incident response lens. Some organizations may have been better prepared for Heartbleed than others, but we can all learn from the lessons of Heartbleed. It's only a matter of time until the next Heartbleed comes along, and our goal as a community should be to learn from this experience and improve our preparedness as a whole.