In a typical week, I spend about 70 percent of my waking hours on work-related matters. Another 50 percent is devoted to my family. Which leaves 20 percent for taking care of the household, and ….
Yes, that adds up to more that 100 percent. But there simply aren’t enough hours in a day for all that needs to be done — not to mention protecting that crucial time with loved ones.
So, thank you, multitasking! I can’t be the only one who has held a child while writing emails, taken conference calls from the supermarket, or had several online meetings running simultaneously.
All of this occurred to me as I struggled to find time for this blog. Writing forces me to shut off everything around me and reflect on the things that really matter — in a world that is rapidly changing, increasingly complex, and in which technology can sometimes seem a mixed blessing. When I do finally carve out an opportunity to write, it is precious time, which I cherish.
But writing is hard. Trust me, I’ve thought about creating a blog for years, and my past is riddled with failed attempts to start. Each time, I hesitated for too long, wondering whether people would really want to hear what I have to say. Like many writers, I have wondered if my compositions were too long, too short, too personal, too corporate, too banal, too deep ….
But as much as I appreciate your attention, dear reader, this time around I realize that I am writing the blog for me, the writer. Like many of us, I navigate a harried, high-pressure life. And this blog is my time, my space, to do something creative and expressive.
Historical data is now an essential tool for businesses as they struggle to meet increasingly stringent regulatory requirements, manage risk and perform predictive analytics that help improve business outcomes. While recent data is readily accessible in operational systems and some summarized historical data available in the data warehouse, the traditional practice of archiving older, detail-level data on tape makes analysis of that data challenging, if not impossible.
Active Archiving Uses Hadoop Instead of Tape
What if the historical data on tape was loaded into a similar low cost, yet accessible, storage option, such as Hadoop? And then data virtualization applied to access and combine this data along with the operational and data warehouse data, in essence intelligently partitioning data access across hot, warm and cold storage options. Would it work?
Yes it would! And in fact does every day at one of our largest global banking customers. Here’s how:
Adding Historical Data Reduces Risk
The bank uses complex analytics to measure risk exposure in their fixed income trading business by industry, region, credit rating and other parameters. To reduce risk, while making more profitable credit and bond derivative trading decisions, the bank wanted to identify risk trends using five years of fixed income market data rather than the one month (400 million records) they currently stored on line. This longer time frame would allow them to better evaluate trends, and use that information to build a solid foundation for smarter, lower-risk trading decisions.
As a first step, the bank installed Hadoop and loaded five years of historical data that had previously been archived using tape. Next they installed Cisco Data Virtualization to integrate the data sets, providing a common SQL access approach that made it easy for the analysts to integrate the data. Third the analysts extended their risk management analytics to cover five years. Up and running in just a few months, the bank was able to use this long term data to better manage fixed income trading risk.
Are your Master Builders free to create? Are your Ordinary Builders helping them to execute? And more to the point, are you acting like the evil President Business, hindering innovation, placing talent in silos, and keeping your organization frozen in the past?
If so, you may find an unlikely role model in Emmet Brickowski.
OK, Emmet may be an animated character made of plastic blocks, but don’t dismiss him so easily. If you are a manager looking to ensure your team is the best it can be, you may want to check out Emmet’s starring role in “The LEGO Movie.” I believe there is deep wisdom in what this little character has to say.
One of the key themes of the film is that many organizations adhere too strongly to their legacy traditions. Though such traditions may have served them well in the past, they can also sow stagnation and put a brake on agility and adaptability. This is especially true in the Internet of Everything (IoE) era, as a massive wave of network connectivity and innovation upends organizations, business models, and entire industries. In the process, longstanding assumptions around strategy and success are falling by the wayside.
Emmet lives in a world run by President Business, the head of a successful corporation that fears any change to the status quo. President Business will even resort to supergluing LEGO pieces to keep them in their rightful places. President Business divides the world into two kinds of people: Ordinary Builders and Master Builders. He rewards Ordinary Builders who follow the rules, building from their LEGO Kits; he disapproves of the “anarchic” creativity of the Master Builders, who like to improvise from a pile of blocks, and he is determined to capture all of them.
This post is co-authored by Martin Lee, Armin Pelkmann, and Preetham Raghunanda.
Cyber security analysts tend to redundantly perform the same attack queries with different input data. Unfortunately, the search for useful meta-data correlation across proprietary and open source data sets may be laborious and time consuming with relational databases as multiple tables are joined, queried, and the results inevitably take too long to return. Enter the graph database, a fundamentally improved database technology for specific threat analysis functions. Representing information as a graph allows the discovery of associations and connection that are otherwise not immediately apparent.
Within basic security analysis, we represent domains, IP addresses, and DNS information as nodes, and represent the relationships between them as edges connecting the nodes. In the following example, domains A and B are connected through a shared name server and MX record despite being hosted on different servers. Domain C is linked to domain B through a shared host, but has no direct association with domain A.
This ability to quickly identify domain-host associations brings attention to further network assets that may have been compromised, or assets that will be used in future attacks.