Implementing a Hot Threat Dashboard
Logjam, Freak, Shellshock, BEAST, POODLE, Heartbleed. Each new vulnerability requires a fire-drill to see if you’re vulnerable, if you have protective mechanisms, and to verify that your organization can detect attacks against your corporate network. On top of that, you may also receive bulletins from threat intelligence partners, law enforcement, and other warnings that require heightened vigilance and the ability to detect new attacks.
Cisco’s Active Threat Analytics (ATA) (an offering from Cisco Managed Security Services, which was formerly known as Cisco Managed Threat Defense – MTD) proactively hunts threats on customer networks. As new threats emerge, whether via newly discovered vulnerabilities, intelligence about targeted attacks, or observations regarding malicious anomalies across customers in a target segment, Cisco ATA employs priority 24×7 hunting through customer data to discover active exploitation. Upon discovery, Cisco ATA guides customers to safely mitigate the attacks. Here’s our process for tracking hot threats. I’ve implemented a similar process in CSIRTs, sharing it here in hopes to see my peers implement similar processes.
Why Do You Need a Hot Threat Dashboard?
Of course, the most important thing you can do to prepare for hot threats is to implement a robust security program, complete with the ability to respond to new threats with agility and coordination. Increasingly, new threats emerge for which there are no patches available, and that slip through even well-designed security controls. A hot threat monitoring process is an important tool to vigilantly monitor your network when such threats emerge.
What Do You Do With a Hot Threat Dashboard?
This post will cover how hot threats are evaluated, documented, communicated, hunted, and retired. I’m leaving out a critical part of this: the need to proactively protect your network from the threat. That’s a topic for another post, but this process helps address the difficult waiting period while you’re waiting to employ a security control against the threat, whether that’s a software patch, system rule, or process change.
Identifying Hot Threats
Each day, Cisco ATA investigators monitor several intelligence sources, security mailers, and vulnerability reports searching for emerging threats. Threats are reviewed and discussed during daily shift handover meetings and online via HipChat. When a threat is deemed serious enough for vigilant monitoring, we document the threat, its means of detection, and then begin hunting for it.
Our goal is to maintain an actionable list of hot threats at any one time. At any one time, we expect to focus on a handful of top-priority threats (usually fewer than 15), in order to maximize our efficiency and impact. Expanding our attention beyond these hot threats would minimize the attention we are able to dedicate to detailed forensics and analysis, so we reserve the ‘hot threat’ designation for only the most aggressive, severe, and widespread threats.
When any of the following criteria are met, we record a hot threat in the database.
- Newly disclosed vulnerability with CVSS base score = HIGH (7.0 or greater). The temporal score may change and increase urgency, as does the environmental score depending on how exposed the environment is to the threat.
- A CVSS base score of MEDIUM may be considered if there is active or imminent exploitation (a higher temporal score), and the customer’s environment is vulnerable and exposed to the threat.
- New, reliable intelligence indicating attacks targeted at specific customers or their vertical. This often comes from Talos, open source intel-sharing forums, ISACs, and advisory notices from law enforcement.
- Native intelligence: Growing evidence of a campaign developing on customers’ sites. This may originate from observed anomalies believed to be malicious, a surge in reliable and high fidelity security alerts, or due to an opaque priority request to place vigilance in monitoring specific network zones for a specified period of time.
Here’s a flow diagram of the hot threat process. This highlights the inputs monitored, the process for posting and sharing, and how threats are retired from the dashboard.
Posting a Hot Threat
Any security analyst or investigator can initiate a hot threat. The submitter will log into the portal to register the threat, providing details such as the attack vector, blog and Twitter references about the threat, Indicators of Compromise (IoC), restrictions on intelligence sharing via TLP, a clear description in the title, and full details in the notes.
Title: Use a title that summarizes the threat and is recognizable – with key words and references to standard industry terms.
Description: Make this an overview of the threat. Note the CVSS score and rationale for treating this threat with priority across our monitoring.
Source: Provide your source for this threat intelligence so that others who see the threat can investigate for themselves.
IOCs: This is where you need to log the specific information known about the threat. This is critical because this is how we search for the threat. Each of these fields can contain up to 255 entries.
- File hash (in SHA-1 or MD5 format) that may be seen in an email attachment, web download, or any other network transfer.
- Attacker IP address or range: the command and control (c2) server, exploit kit host, or other IP address materially participating in the attack.
- URL: Most likely where the exploit kit or watering hole is hosted, but could also be the URL used for c2, as we’ve seen when c2 is hosted via YouTube or Pinterest comments.
- Email address: From address used by attackers in spam and phishing campaigns.
- Email subjects: Subject lines from emails used by attackers in spam and phishing campaigns.
- File names: File names that may be seen in email attachments, web downloads, or any other network transfer.
- Geo – the geolocation of the attack origination – only useful if the attack geography is relevant.
- ASN – the BGP Autonomous System Number (ASN) – useful if the network neighborhood of the attacker is relevant to the threat.
Sidebar: Extracting IOCs
While we await the nirvana of all intelligence sharing via STIX/TAXII, we must still cope with the frequent receipt of IOCs embedded in PDF documents. We’ve found a great tool to extract IOCs, and we use it regularly in ATA. You can find the IOC Parser tool on Github.
Reviewing a Hot Threat
Upon posting the hot threat, a senior security investigator (Investigations Manager) will review the threat to validate that it meets the criteria and add further context as necessary. Once vetted, the threat will become active, triggering prioritized monitoring across all shifts.
Sidebar: Prove You’re On the Job
IT leadership often hear about these threats through peers and news outlets. Implementing a hot threat dashboard is a great way to implement proactive monitoring during the crisis. You may want to further consider drafting a hot threat advisory, which I’ll cover in a future blog post.
Notification to Collaborating Teams
Other teams who collaborate on security intelligence and investigations require priority sharing of confirmed threats. Upon confirmation of a new hot threat, the investigator will share the threat with intelligence collaborators just prior to posting on the hot threat board, to ensure coordinated monitoring for all collaborative partners. Notification to other groups is done manually via email, to add further context to the threat, and to honor the TLP restrictions set on the threat indicators.
Monitoring Hot Threats
Cisco’s ATA team will conduct prioritized monitoring via OpenSOC, which places a prioritized, automated hunt through multi-Gbps streams of customer data to match hot threat IOCs or signatures. Upon a match, OpenSOC automatically opens a ticket in RequestTracker (RT), which is then reviewed by a security analyst to determine whether the attack is successful. Just as in all security monitoring, if the attack was stopped by a security control, such as a firewall, Cisco AMP, or anti-virus, the attack was unsuccessful and requires no further investigation.
Once an attack is fully analyzed and investigated, the investigator will notify the customer of the breach, along with detailed mitigating guidance to contain the attack and prevent further harm to their systems. The hot threat is referenced in the customer notification, to enable tracking back to the hot threat board.
This enables the hot threat board to reference successful compromises attributed to the threat, which is posted in each SOC and correlated to the customer via code names. Color indicates severity of the breach.
- Green: 1-5 matched tickets
- Yellow: 6-10
- Red: 10+
In addition to actively hunting threats, ATA looks back in time over past days and weeks for matched indicators. This is made possible by OpenSOC’s indexing of all network metadata and telemetry, and the ability to reference full packet capture collected from each customer network. Further, Sourcefire’s retrospective detection enables us to match file indicators previously judged benign.
Retiring Hot Threats
Hot threats are meant to be hot. So as time passes, the IOCs diminish in their connection to imminent problems. During that period, monitoring threat indicators is absorbed into our monitoring playbook, and detection is enabled via established means represented in OpenSOC stored queries, which point back to network groups, detection signatures, and AMP alerts.
So, once a threat is thirty days old, or if the fidelity of the reports indicates that the threat has substantively diminished in importance, it is retired from the dashboard. Older threats remain searchable for IOCs as needed.
As new, urgent vulnerabilities continue to drop in our laps like hot potatoes, and as you make use of threat intelligence, a process to prioritize urgent threat hunting in your network is a critical way to employ monitoring vigilance. This post introduced how Cisco’s ATA Team researches, hunts, and investigates hot threats, employing a global dashboard to keep the team coordinated 24/7.
Props to Tom Piscitell for building this hot threat portal, and to Chris Riley for designing ATA’s hot threat process.