The Importance of Logs
It’s funny how the world turns. I started off in security working for a bank. The model there was very much build it, break it, fix it with our Operational Security team aligning with platform and application support teams to build the projects the business wanted. Very soon after I joined it became clear that despite our best efforts (and we were scanning our systems nightly for new vulnerabilities even 15 years ago), that sometimes things go wrong and you need logs. As a result, we kicked off a project to implement mandatory access control and auditing across our entire estate. The lessons we learnt then are still being learnt by other organisations today. Notably, in the intervening period of time, having become a security consultant, I lost count of the number of times I tried to encourage organisations for whom we were doing technical assurance work (over 50% of Security Advisory engagements in EMEAR 2017), that they needed logging and auditing and they needed it to monitor their business systems and data. Sometimes they listened but many times they did not.
Fast forward to today and I’ve spent the last 6 weeks supporting Cisco’s IR team in EMEAR with some challenging cases. We’ve had a file system that got corrupted and where the customer couldn’t tell what commands had been executed to get to that point, a logging solution where important application layer logs weren’t deemed valuable enough to store and an endpoint breach where it wasn’t possible to understand who was using a given role account to access the system nor how a set of weak credentials were introduced. What do each of these 3 stories* have in common? The organisations concerned only appreciated the value of logs after the fact when we patiently explained that whilst we could work magic, that magic wouldn’t have been required, if only they’d enabled and configured logging correctly first time out. This is definitely one of the aspects of life at Cisco that I enjoy most – not only do we do the red team, which shows the destructive impact of security weaknesses but we also work with the blue team to give them a get well plan. I am so much more contented delivering Security Advisory work where we get to work with the customer on a solution to their problem than I ever was as a penetration tester landing a 300 page tome on a customer’s desk safe in the knowledge that it would be thick with dust on my return in 12 months time.
Amusingly, in the process of writing this post, I came across a similar article from our US IR folks here, iterating through some of the common log sources you’ll find on a typical enterprise network. It’s a good article and I recommend that you read it, but simply covering these bases will only tell you half the story in a real life scenario. Whilst done well, enterprise logging will tell you what happened when your Active Directory estate gets breached or an endpoint contracts malware but the value of any business is really the data it stores. If you really want to get ahead of the curve from a defensive perspective, then you need to start plumbing your application stacks into your logging infrastructure. SAP audit logs, you should gather them; Exceptions from the application server that hosts your e-commerce platform, yes please; Sudo logs for the robot that makes the beds you sell, never say no. If you don’t know what logs your application stacks can produce, then now is the perfect time to engage with us. We can map your landscape and help you identify the logs you want to onboard. Better still, we can help you build policies and processes to handle the logs that you don’t even have yet.
So why am I telling you this? Firstly, a slice of reality, at some stage there will be a breach and breaches are expensive (half of the incidents that we encountered in 2017 resulted in a cost of over $500K). However, Cisco is not just that sad face that meets you when things go wrong but also the friendly face that helps avoid this scenario. We’ll fly an analyst to Stockholm at the drop of a hat and spend days knee deep in vendor documentation. However, if you think that IR is all about the 4am phone calls, then please come talk to us about the proactive services we offer that will help to prepare your organisation for that day you never want to have.
* And they are made up stories, we would never discuss a specific and sensitive customer scenario.