IDS

I find it fascinating that although IDS has the potential to be used as a sharpened scalpel to pick out abnormal network activity from various places within the packet, it is mostly used as a packet by packet log collector.  What do I mean by this?  IDS is a technology that can be equipped with a relatively small number of highly potent and actionable signatures designed to look for activity that organizations need to take notice of.  Unfortunately though, most organizations go the complete opposite direction, deploying IDS with thousands of weak/mediocre signatures, perhaps out of fear of "missing something".  The result?  Anything that might have been worth looking at gets buried in 10,000 false positives per day (or more!).  Worse yet, sometimes IDS becomes every analyst's favorite data source to ignore.  It's a shame really -- so much potential in modern IDS devices, yet so underutilized.

BS Filter

It sounds a bit crude, but a good BS filter can be an analyst's best friend.  Analysts are confronted by an overwhelming amount of information on a daily basis.  Whether it be from blogs, mailing lists, management, a vendor, an intelligence feed, a tool, or one or more log sources, it can be overwhelming.  Diving into the wrong lead can tie up precious analyst resources for hours or even days, often taking away from other events or leads that need to be investigated.  So what is an analyst to do?  Follow only the leads most likely to yield fruit!  Easier said than done, of course.  Knowing how to separate out the good leads from those that are a fool's errand is an acquired skill that takes years of false starts to develop.  It can be a skill that is extremely valuable to an organization though and should be respected.

Big Data

There is a lot of buzz lately about big data.  Almost every vendor and most pundits seem to be talking about big data -- and telling us we need it.  While I am seeing a lot of hype and build-up, I'm not seeing a whole lot of useful advice or helpful tips about *how* to actually leverage big data.  Where I work, we've been leveraging big data for over a year now (we just never called it that).  I see this as an opportunity.  We've developed some effective techniques for slicing through big data and producing a reasonable volume of highly actionable alerting from over 4 billion network events a day.  It's time to take this show on the road and share the information.  I've put in some papers for some upcoming conferences -- wish me luck!

To The Endpoint

As time has progressed, monitoring has moved further and further inside the enterprise.  Once upon a time, an enterprise could monitor only its perimeter and still be in decent shape.  Then, attackers started moving up the OSI stack (as has been discussed previously in this blog and elsewhere).  Now, with attackers using legitimate domains/sites, encrypted command and control, and a whole host of other techniques to hide in the noise/legitimate traffic, it is getting increasingly difficult for the defender to keep pace.  It appears to me that the only long term monitoring solution is to go all the way to the endpoint somehow (in tandem with instrumenting the network for monitoring of course).  I'm not sure I see how else we can obtain the visibility on the interior segments of the network that we so desperately need.

A sad reality I fear.

Practical Passive DNS Example

I (along with many others) have blogged in the past about the tremendous value of passive DNS data.  Recently, I have encountered a practical example of an instance where passive DNS data may provide one of the only ways to try and stay on top of a threat.

Recently, some members of the community have encountered/been working with a piece of malicious code that has the following properties:

1) A malicious JAR file is delivered with a content type of text/html.  This JAR file assembles two executables on the victim host and infects it.
2) The infected vicitim host initiates a callback to a randomly generated four character sub-domain (e.g., ..com).  This traffic is encrypted (via SSL) and goes to the same netblock (a /24), regardless of the domain name (which changes regularly).

The purpose of this malware is not yet known, and it is not widely written about or particularly well understood.

Typically, as has been discussed on this blog and elsewhere in the past, it's much easier to look for artifacts of infection, rather than pre-infection activity.  Typically that involves creating monitoring/alerting for one of the following:

1) Looking for command and control URL patterns (which are much more reliable and have lower false positive rates than just domain names or IP addresses).  In this case, the callback traffic is encrypted, and thus, there is no URL pattern that can be monitored for (unless one intercepts the SSL traffic, but that is a separate discussion).
2) Looking for command and control domain names.  Here again, since the domain names change, this is not going to be very effective as a long term monitoring solution.
3) Looking for the malicious code payload delivery.  In this case, since it is a JAR being delivered with a content type of text/html, the noise is overwhelming.  Creating alerting based on this type of logic would inundate incident response personnel with alerts, the overwhelming majority of which would be false positives.  As I've learned, delivering a JAR via content type text/html is actually fairly common, even for legitimate applications.

So what does one do?  Well, since the callbacks all go to the same netblock, one can monitor passive DNS data for any new domains that are requested and resolve to that netblock matching the pattern described above.  That way, if the callback domains change (which they have been), and a host is infected, the incident response team can pick up on the new callback domain(s) and respond accordingly.  True, the incident response team is still one step behind the malware, but that's a lot better than being in the dark.

Pretty neat real world use of passive DNS data in my opinion.

Managing Up

It's always amazed me how a high profile incident seems to turn up management and executives in an incident response environment.  When a serious incident hits, everyone wants to hang out with the incident responders, even people who one might never see day to day.  In these type of situations, managing up is key.  Management and executives have the best intentions, but they don't work with the data day to day, and their technical skills may be a bit dated.  Suggestions on how to proceed, what analysis to do, and how to do it will fly at an incident responder faster than he/she can process them.  Unfortunately, many ideas that seem good in theory are not good ideas in practice.  The volume and variety of data makes finding the right approach tricky, and many behaviors that seem like they would indicate malicious activity don't.  Trust your senior team members and manage up in the best way you can.  Your team will be more productive because of it, and your senior team members will thank you for it.

Focus

When performing incident response, focus is extremely important.  A significant incident can produce innumerable leads and avenues to investigate.  Unfortunately, not all of these leads/avenues are productive ones.  Choosing poorly can have the unintended consequence of locking up resources for days while producing very little value-added analysis.  It is often difficult to know which direction or directions to go in analytically.  In my experience, senior members of the incident response team, who base recommendations on past experiences, lessons learned, and day to day familiarity with the network and its data have good advice to offer here.  That being the case, I've always wondered why management ends up driving high profile incidents.  It's a bit of a wonder if you think about it....