Comments on: Tracking Malicious Activity with Passive DNS Query Monitoring

By: Hash1 (First Name)

Hash1 (First Name) — Wed, 21 Nov 2012 13:52:48 +0000

Hi, Are you releasing your code in near future . Beside ISC db is there any community db exist to query with specially for detection of malware based on DNS history?

I am highly looking forward to for one of my paper of Phd research . actually i was thinking to run a passive DNS on internet

any suggestion ?

0 likes

By: Brandon Enright

Brandon Enright — Tue, 20 Nov 2012 23:00:26 +0000

You’re right that a tool like Bro can parse DNS and a lot of other protocols. I haven’t used ELSA but I assume it’s similar to Splunk in terms of searching and indexing ability.

We really like storing the packet data rather than parsed content because it allows us to go back later and see anything else such as TXIDs or other flags that didn’t necessarily get parsed to text. DNS packets + UDP header are also significantly more compact than a full text representation of the content. Packet data also compresses well. Also, we didn’t want to put a parser in the path from the network to the disk. Some of our capture locations see extremely high DNS packet rates and we’d rather be able to parse after capture.

I’m sure there are things full-text-indexing gets you that our tool does not. For example, we do the filtering and searching for names based on suffix. So http://www.cisco.com turns into {“com”, “cisco.com”, “www.cisco.com”} for the bloom filter. This gives us some search flexibility while maintaining a lot of speed. Searching for all names that match /^www\./ can be done with our tool but it is not fast.

0 likes

By: Doug Burks

Doug Burks — Thu, 08 Nov 2012 10:57:51 +0000

I definitely agree that DNS visibility is a powerful tool for security teams!

For those looking to do this on their own, you can use Bro to log DNS traffic (and many other protocols). Those Bro logs can be sliced and diced using ELSA, a nice web interface for hunting through logs. You can have both Bro and ELSA up and running in a few minutes using Security Onion.

Thanks,
Doug Burks
Security Onion

1 like

By: Jerold Swan

Jerold Swan — Tue, 06 Nov 2012 21:00:14 +0000

Bro-IDS is an open-source tool that includes detailed DNS query logging by default:

http://bro-ids.org/

Two other open-source projects that can be used for analyzing the logs are Brownian and ELSA:

Brownian:
https://github.com/grigorescu/Brownian

ELSA:
https://code.google.com/p/enterprise-log-search-and-archive/

0 likes

By: Brandon Enright

Brandon Enright — Mon, 05 Nov 2012 23:36:03 +0000

We’re using pybloomfiltermmap for the bloom filter. I can’t remember what we set the false-positive rate to but it’s either 5% or 1%.

Since each node only searches the data it has collected we don’t need to do anything fancy like what you’d get with MPI or some other distributed computing / message passing / cluster API. Instead we’ve just built a client/worker model (in Python) where the client issues the search criteria to each node and they each report results back. The client then combines the results.

Progress is tracked by each worker reporting the total files that match for the timerange. Then each time they finish searching a file (either because the file was passed up due to the bloom filter check or actually searched it) they report completing that file.

So if I issue the command:
$ python search --qname ncirc.nato.int --start 2012-11-01

The search status bar looks like:
Search: 100% |#############################| Time: 0:00:49 Files: 100404/100404

Indicating that for the time period there are 100404 1-minute capture files across all nodes.

0 likes

By: David A.

David A. — Mon, 05 Nov 2012 08:18:39 +0000

What library do you use for bloom filters?
What library or functionalities do you use to distribute work to multiple machines? How do you track completion/progress?

0 likes

By: Brandon Enright

Brandon Enright — Wed, 31 Oct 2012 20:56:35 +0000

The sensors are just Linux running on Cisco UCS servers. By passively collecting DNS traffic I meant we’re capturing the packets, not collecting transaction logs from our DNS servers. We use a the VACL feature of our routing/switching gear to selectively capture DNS traffic so that we don’t have to mirror/span all traffic to our collectors.

With the traffic being sent to our Linux machines, we’re using NCAP (ncaptool) to record the traffic into 1-minute capture files. Our code then comes along after the minute is over and builds the bloom filters and rolls the files into our data store for fast searching.

0 likes

By: Rolf Sommerhalder

Rolf Sommerhalder — Wed, 31 Oct 2012 11:50:17 +0000

Can you describe the capture sensors a little bit which you have deployed globally to (passively?) collect the DNS traffic? For example, did you build some appliance that you hook up to switch mirror ports? Or do you collection transaction logs from your DNS servers/resolvers?

0 likes

By: Brandon Enright

Brandon Enright — Tue, 30 Oct 2012 20:52:39 +0000

I’d be asking the same question in your place. I’m a big proponent and user of open-source software and so are several of my colleagues. We’ll definitely be releasing more technical details and we certainly would like to release the code. Although the code is still undergoing development and improvement, we’ve coded it with the possibility of releasing it ind mind. Our biggest hurdle right now is going through the internal approval process.

In short, we really want to but it isn’t something that will happen overnight.

0 likes

By: David A.

David A. — Tue, 30 Oct 2012 19:51:17 +0000

Could please pretty please release some code or more technical details concerning that tool?

I think it would be a tremendous contribution to the infosec toolset for complex infrastructures.

1 like