Cisco Blogs


Cisco Blog > Security

Big Security—Mining Mountains of Log Data to Find Bad Stuff

Your network, servers, and a horde of laptops have been hacked. You might suspect it, or you might think it’s not possible, but it’s happened already. What’s your next move?

The dilemma of the “next move” is that you can only discover an attack either as it’s happening, or after it’s already happened. In most cases, it’s the latter, which justifies the need for a computer security incident response team (CSIRT). Brandon Enright, Matthew Valites, myself, and many other security professionals constitute Cisco’s CSIRT. We’re the team that gets called in to investigate security incidents for Cisco. We help architect monitoring solutions and strategies and enable the rest of our team to discover security incidents as soon as possible. We are responsible for monitoring the network and responding to incidents discovered both internally by our systems or reported to us externally via csirt-notify@cisco.com.

Securing and monitoring a giant multinational high-speed network can be quite a challenge. Volume and diversity, not complexity, are our primary enemies when it comes to incident response. We index close to a terabyte of log data per day across Cisco, along with processing billions of NetFlow records, millions of intrusion detection alarms, and millions of host security log records. This doesn’t even include the much larger data store of authentication and authorization data for thousands of people. Naturally, like all large corporations, dedicated attackers, hacking collectives, hacktivists, and typical malware/crimeware affect Cisco. Combine these threats with internally sourced security issues, and we’ve got plenty of work cut out for us.

Read More »

Tags: , , , , , , , , , ,

Cisco Hosting Amsterdam 2013 FIRST Technical Colloquium

There is still time to register for the upcoming FIRST Technical Colloquium April 2-3 2013. The event has a very exciting program covering, bitsquatting, webthreats, RPZ, Passive DNS, Real-world monitoring examples, Spamhaus, SIE, Cuckoo Sandbox, Malware Analysis and many more current issues facing the incident response community.

The event’s line-up includes notables from Cisco Security Intelligence Operations (SIO), Internet Systems Consortium, Shadowserver foundation, KPN-CERT, NATO, MyCert and ING amongst others. Program details can be found here.
Read More »

Tags: , , , , , , , , , , ,

Tracking Malicious Activity with Passive DNS Query Monitoring

Ask anyone in the information security field they will tell you:

Security is not fair. There is essentially an unlimited supply of attackers that can test your defenses with impunity until they eventually succeed.

As a member of the Cisco Computer Security Incident Response Team (CSIRT) I’ve seen this asymmetry up close, so I can tell you that good security is really hard. Besides the normal security practices like deploying firewalls, IDS sensors, antivirus (AV), and Web Security Appliances, CSIRT is increasingly looking to the network as a data source. We have been collecting NetFlow for years but we have always wanted additional context for the flow data. While it is true that the Internet is built on TCP/IP, Internet services—both good and bad—are found by name using the Domain Name System (DNS). For years infosec has been network-address-centric and the attackers have adapted. Today it is very common to see malware command and control (C&C) use domain generation algorithms (DGAs), Peer-to-Peer (P2P), or even fast-flux DNS to evade IP address-based detection and blocking. It has become absolutely clear that to keep up with the latest attacks and attackers you must have a view into the DNS activity on your network.

CSIRT has been struggling with limited DNS information for a while now, so I am pleased to say we finally have comprehensive visibility into the DNS activity on our network. Before I dive into how we tackled this problem I should back up and explain a bit more about DNS…

Read More »

Tags: , , , , ,

CSIRT Monitoring for Cisco House at the London 2012 Olympic Games

As part of CSIRT’s mobile monitoring offering for special events, we undertook monitoring of the corporate and customer traffic of the Cisco House at the London 2012 Olympics. This engagement presents us with an excellent opportunity to showcase Cisco technology, while keeping a close watch on potential network security threats. CSIRT monitoring for this event will be active for the entire life-span of the Cisco House, from two months before the Olympics, until two months after.

For the London 2012 engagement, we shipped our gear in a 14RU military-grade rack that is containerized: made for shipping. Inside the mobile monitoring rack we have an assortment of Cisco kit and third-party kit that mirrors the monitoring we do internally:

  • Catalyst 3750 to fan out traffic to all the other devices
  • FireEye for advanced malware detection
  • Two Cisco IronPort WSA devices for web traffic filtering based on reputation
  • Cisco UCS box where we run multiple VMs
  • Lancope StealthWatch collector for NetFlow data
  • and a Cisco 4255 IDS for intrusion detection

We mirror the signatures that we have deployed internally at Cisco out to these remote locations. Depending on the environment where the mobile monitoring rack is deployed, we may also do some custom tuning. The kit in the mobile monitoring rack can do intrusion detection, advanced malware detection, and collect and parse NetFlow and log data for investigation purposes. The Cisco UCS rack server also helps us have several VMs,  allowing us to run multiple tools that complement the other devices in the rack. For example, we run a Splunk instance on a VM to collect the logs generated by all the services. The data from the gear in the mobile monitoring rack is analyzed by our team of analysts and investigators, to eliminate false positives, conduct mitigation and remediation, and finally produce an incident report if required.

Read More »

Tags: , , , , , , , , , ,

Cisco CSIRT on Advanced Persistent Threat

For corporations, Advanced Persistent Threat (APT) is a widely publicized yet little understood topic.  Does it exist?  Is it a real threat?  How can an organization tell if it is impacted?

The Cisco Computer Security Incident Response Team (CSIRT) is a global team of information security professionals responsible for the 24/7 monitoring, investigation and response to cyber security incidents for Cisco-owned businesses. CSIRT engages in proactive threat assessment, mitigation planning, incident detection and response, incident trending with analysis, and the development of security architecture. This article will provide the Cisco CSIRT team’s perspective on APT, and is the fifth in a series of blog posts on related issues from CSIRT’s point of view.  As with the other posts, provided here are some real-world examples and techniques that will hopefully help organizations utilize existing tools and processes, or even understand gaps in security infrastructure.  Read on to find out more.

Read More »

Tags: , , , ,