Thoughts on Intelligence Retention

My previous post entitled "Measuring Security Intelligence Value" was quite popular, and I'm glad that I was able to put together a blog post that interested so many people. Recently, I was asked for my thoughts on intelligence retention. This catalyzed me to put together this post.

It is well known that intelligence is an important component of a successful security operations program. Ideally we would like to retain our intelligence forever, but what if that is not possible? In some organizations, it may be necessary to discard or "age-off" intelligence after some amount of time. In this post, I am writing from the perspective of an organization functioning as a consumer of intelligence. I am also assuming that the reader understands the difference between intelligence and information. The intelligence vs. information discussion is beyond the scope of this post, and I will therefore assume that the reader is acutely aware of the difference.

Before I discuss different approaches for aging-off intelligence, I would like to briefly discuss the concept of vetting. As I discussed in the "Measuring Security Intelligence Value" post and in earlier posts, in my experience, quality of intelligence is more important than quantity of intelligence. In other words, properly vetting intelligence before it is added to the organization's security repository has a number of benefits. Aside from the improved signal-to-noise ratio (ratio of true positives to false positives) resulting from improved intelligence, it also helps with the retention issue. When there is less "garbage" consuming retention resources, it allows us to retain our intelligence longer using the same amount of retention resources.

If it becomes necessary to discard intelligence, there are a number of different approaches one could employ. While not an exhaustive list, I have listed a few approaches here:

• Simple time-based: In the simple time-based approach, intelligence is discarded after N days of retention (e.g., 180 days). This is probably the simplest approach to implement, but does not account for any of the other dimensions of each piece or source of intelligence.
• Fidelity-based: In the fidelity-based approach, when it becomes necessary to discard intelligence, it is discarded from lowest fidelity to highest fidelity after the minimal retention period. For example, say we measure fidelity on a 1 to 10 scale, with a value of 1 indicating that the indicator is not very reliable/not of high fidelity, and a value of 10 indicating that the indicator is extremely reliable/of extremely high fidelity. In this approach, after the minimal retention period, we begin by discarding intelligence of fidelity 1, then 2, and so on until it is no longer necessary to discard intelligence. Using this approach enables us to retain our highest fidelity intelligence for longer than our lowest fidelity intelligence.
• Source-based: In the source-based approach, when it becomes necessary to discard intelligence, it is discarded from the least reliable source to the the most reliable source after the minimal retention period. For example, say we measure source reliability on a 1 to 10 scale, with a value of 1 indicating that the source is not very reliable, and a value of 10 indicating that the source is extremely reliable. In this approach, after the minimal retention period, intelligence is discarded from sources of reliability 1, then 2, and so on until it is no longer necessary to discard intelligence. Using this approach enables us to retain intelligence from our best sources for longer than intelligence from our not so great sources.
• Attack stage-based: In the attack stage-based approach, we look at discarding intelligence based on the particular attack stage it is relevant to. For example, we may value intelligence related to command and control (C2) sites more than we value intelligence related to exploit sites. As such, we can build a prioritized list of attack stages per the needs of our organization. After the minimal retention period, we can discard intelligence from the attack stage of least priority, followed by the attack stage of second least priority, and so on, until it is no longer necessary to discard intelligence. Proceeding in this manner allows us to retain intelligence related to the attack stages we are most concerned with for longer than intelligence related to the attack stages we are least concerned with.
• Type-based: In the type-based approach, we look at discarding intelligence based on the type of intelligence it is. For example, we may value URL patterns more than we value IP addresses. As such, we can make a prioritized list of intelligence types. After the minimal retention period, we can discard intelligence from the type of least priority, followed by the type of second least priority, and so on, until it is no longer necessary to discard intelligence. This approach enables us to retain certain types of intelligence longer than others per our organizational needs.
• Flag-based: In the flag-based approach, we "flag" specific intelligence that is of high value to us. After the minimal retention period, we discard intelligence that is not flagged. This allows us to retain specific pieces of intelligence that are of high value to us beyond the minimal retention period.

As I'm sure you've realized by now, it is also possible to use combinations of the above approaches as suits the organization's needs and goals. It is possible to combine various approaches to create a retention approach that leaves an organization with more of the intelligence it values most and less of the intelligence it values least.

Intelligence is not a linear undertaking. So why should intelligence retention be approached only linearly?