Historical data is now an essential tool for businesses as they struggle to meet increasingly stringent regulatory requirements, manage risk and perform predictive analytics that help improve business outcomes. While recent data is readily accessible in operational systems and some summarized historical data available in the data warehouse, the traditional practice of archiving older, detail-level data on tape makes analysis of that data challenging, if not impossible.
Active Archiving Uses Hadoop Instead of Tape
What if the historical data on tape was loaded into a similar low cost, yet accessible, storage option, such as Hadoop? And then data virtualization applied to access and combine this data along with the operational and data warehouse data, in essence intelligently partitioning data access across hot, warm and cold storage options. Would it work?
Yes it would! And in fact does every day at one of our largest global banking customers. Here’s how:
Adding Historical Data Reduces Risk
The bank uses complex analytics to measure risk exposure in their fixed income trading business by industry, region, credit rating and other parameters. To reduce risk, while making more profitable credit and bond derivative trading decisions, the bank wanted to identify risk trends using five years of fixed income market data rather than the one month (400 million records) they currently stored on line. This longer time frame would allow them to better evaluate trends, and use that information to build a solid foundation for smarter, lower-risk trading decisions.
As a first step, the bank installed Hadoop and loaded five years of historical data that had previously been archived using tape. Next they installed Cisco Data Virtualization to integrate the data sets, providing a common SQL access approach that made it easy for the analysts to integrate the data. Third the analysts extended their risk management analytics to cover five years. Up and running in just a few months, the bank was able to use this long term data to better manage fixed income trading risk.
Are your Master Builders free to create? Are your Ordinary Builders helping them to execute? And more to the point, are you acting like the evil President Business, hindering innovation, placing talent in silos, and keeping your organization frozen in the past?
If so, you may find an unlikely role model in Emmet Brickowski.
OK, Emmet may be an animated character made of plastic blocks, but don’t dismiss him so easily. If you are a manager looking to ensure your team is the best it can be, you may want to check out Emmet’s starring role in “The LEGO Movie.” I believe there is deep wisdom in what this little character has to say.
One of the key themes of the film is that many organizations adhere too strongly to their legacy traditions. Though such traditions may have served them well in the past, they can also sow stagnation and put a brake on agility and adaptability. This is especially true in the Internet of Everything (IoE) era, as a massive wave of network connectivity and innovation upends organizations, business models, and entire industries. In the process, longstanding assumptions around strategy and success are falling by the wayside.
Emmet lives in a world run by President Business, the head of a successful corporation that fears any change to the status quo. President Business will even resort to supergluing LEGO pieces to keep them in their rightful places. President Business divides the world into two kinds of people: Ordinary Builders and Master Builders. He rewards Ordinary Builders who follow the rules, building from their LEGO Kits; he disapproves of the “anarchic” creativity of the Master Builders, who like to improvise from a pile of blocks, and he is determined to capture all of them.
This post is co-authored by Martin Lee, Armin Pelkmann, and Preetham Raghunanda.
Cyber security analysts tend to redundantly perform the same attack queries with different input data. Unfortunately, the search for useful meta-data correlation across proprietary and open source data sets may be laborious and time consuming with relational databases as multiple tables are joined, queried, and the results inevitably take too long to return. Enter the graph database, a fundamentally improved database technology for specific threat analysis functions. Representing information as a graph allows the discovery of associations and connection that are otherwise not immediately apparent.
Within basic security analysis, we represent domains, IP addresses, and DNS information as nodes, and represent the relationships between them as edges connecting the nodes. In the following example, domains A and B are connected through a shared name server and MX record despite being hosted on different servers. Domain C is linked to domain B through a shared host, but has no direct association with domain A.
This ability to quickly identify domain-host associations brings attention to further network assets that may have been compromised, or assets that will be used in future attacks.
“This conference is designed not only to make you think about the application of automation, but also to help you take action” – David Greenfield, Automation World, editor in chief and TAC event director
The conference achieved this goal and more. The framework of the sessions encouraged audience collaboration and dialogue around the challenges and practical steps and strategies being designed and deployed to achieve an integrated and scalable IoE architecture that drives value across the entire manufacturing value chain, as depicted in the video below:
What better way to meet that objective than to leverage a manufacturing use case around beer!!!!
Automating Brewing Operations from Two Different Perspectives
I attended this session where Highland Brewing, Sierra Nevada and Vicinity Manufacturing gave an interesting perspective around the challenges and strategies in deploying their next generation manufacturing operation.
Highland Brewing is a regional brewer of craft beers based in the Southeast and Sierra Nevada is a larger brewer with more of national brand. The interesting contrast between the two is that Highland Brewing is designing more automation into their operational facility and Sierra Nevada is scaling their automation and IoE strategies across all their facilities. Both perspectives and approaches have the same objective. How do I effectively integrate all the various technologies into an intelligent, flexible and scalable system/architecture to meet the following business outcomes:
Increase Customer Loyalty
Supply Chain Optimization
To paraphase Kevin Wheeler, Director of Operations, Highland Brewing Co,“Our core competency is crafting great beer. We have an opportunity to drive efficiency into our operation by an integrating IoT/IoE platform … the challenge is figuring out the best approach.”
Like Highland Brewing, manufacturers must begin to transform existing business processes and fundamentally rethink how they create, operate, and service smart, connected products in the IoE. For those that get it right, the future represents a huge opportunity to create product and service advantages.
Are you having challenges putting together the “IoE technology puzzle?” Is security the main barrier to IoE adoption?