There is a great debate in the security world right now: have SIEM and logging products run their course? Will Hadoop ride to the rescue? Can machines “learn” about security and reliably spot threats that no other approach can find?
Gartner calls this phenomenon Big Data Security Analytics, and they make a strong point to define BDSA solutions as a three-layer pyramid. At the bottom is the “data lake,” which is what most people equate with Hadoop. The next layer is context—the addition of relevant business, location, and other non-traditional security information to increase the precision of the next layer: applications and analytics (such as Machine Learning). It is this top layer where the real value of BDSA is realized in terms of finding new threats and remediating them before they do damage.
Read More »
Tags: big data analytics, Cisco EIR, Cisco Entrepreneurs in Residence, Hadoop, PetaSecure, security
Big Data is not just about gathering tons of data, the digital exhaust from the internet, social media, and customer records. The real value is in being able to analyze the data to gain a desired business outcome.
Those of us who follow the Big Data market closely never lack for something new to talk about. There is always a story about how a business is using Big Data in a different way or about some new breakthrough that has been achieved in the expansive big data ecosystem. The good news for all of us is, we have clearly only scratched the surface of the Big Data opportunity!
With the increasing momentum of the Internet of Everything (IoE) market transition, there will be 50 billion devices connected to the Internet by 2020—just five years from now. As billions of new people, processes, and things become connected, each connection will become a source of potentially powerful data to businesses and the public sector. Organizations who can unlock the intelligence in this data can create new sources of competitive advantage, not just from more data but from better access to better data.
What we haven’t heard about – yet—are examples of enterprises that are applying the power of this data pervasively in their organizations: giving them a competitive edge in marketing, supply chain, manufacturing, human resources, customer support, and many more departments. The enterprise that can apply the power of Big Data throughout their organization can create multiple and simultaneous sources of ongoing innovation—each one a constantly renewable or perpetual competitive edge. Looking forward, the companies that can accomplish this will be the ones setting the pace for the competition to follow.
Cisco has been working on making this vision of pervasive use of Big Data within enterprises a reality. We’d like to share this vision with you in an upcoming blog series and executive Webcast entitled, ‘Unlock Your Competitive Edge with Cisco Big Data Solutions’, that will air on October 21st at 9:00 AM PT.
I have the honor of kicking off the multi-part blog series today. Each blog will focus on a specific Cisco solution our customers can utilize to unlock the power of their big data – enterprise-wide– to deliver a competitive edge to our customers. I’m going to start the discussion by highlighting the infrastructure implications for Big Data in the internet of Everything (IoE) era and focus on Cisco Unified Computing System initially.
Enterprises who want to make strategic use of data throughout their organizations will need to take advantage of the power of all types of data. As IoE increasingly takes root, organizations will be able to access data from virtually anywhere in their value chain. No longer restricted to small sets of structured, historical data, they’ll have more comprehensive and even real-time data including video surveillance information, social media output, and sensor data that allow them to monitor behavior, performance, and preferences. These are just a few examples, but they underscore the fact that not all data is created equally. Real-time data coming in from a sensor may only be valuable for minutes, or even seconds – so it is critical to be able to act on that intelligence as quickly as possible. From an infrastructure standpoint, that means enterprises must be able to connect the computing resource as closely as possible to the many sources and users of data. At the same time, historical data will also continue to be critical to Big Data analytics.
Cisco encourages our customers to take a long-term view—and select a Big Data infrastructure that is distributed, and designed for high scalability, management automation, outstanding performance, low TCO, and the comprehensive, security approach needed for the IoE era. And that infrastructure must be open—because there is tremendous innovation going on in this industry, and enterprises will want to be able to take full advantage of it.
One of the foundational elements of our Big Data infrastructure is the Cisco Unified Computing System (UCS). UCS integrated infrastructure uniquely combines server, network and storage access and has recently claimed the #1, x86 blade server market share position in the Americas. It’s this same innovation that propelled us to the leading blade market share position that we are directly applying to Big Data workloads. With its highly efficient infrastructure, UCS lets enterprises manage up to 10,000 UCS servers as if they were a single pool of resources, so they can support the largest data clusters.
Because enterprises will ultimately need to be able to capture intelligence from both data at rest in the data center and data at the edge of the network, Cisco’s broad portfolio of UCS systems gives our customers the flexibility to process data where it makes the most sense. For instance, our UCS 240 rack system has been extremely popular for Hadoop-based Big Data deployments at the data center core. And Cisco’s recently introduced UCS Mini is designed to process data at the edge of the network.
Because the entire UCS portfolio utilizes the same unified architecture, enterprises can choose the right compute configuration for the workload, with the advantage of being able to use the same powerful management and orchestration tools to speed deployment, maximize availability, and significantly lower your operating expenses. Being able to leverage UCS Manager and Service Profiles, Unified Fabric and SingleConnect Technology, our Virtual interface card technology, and industry leading performance really set Cisco apart from our competition.
So, please consider this just an introduction to the first component of Cisco’s “bigger”, big data story. To hear more, please make plans to attend our upcoming webcast entitled, ‘Unlock Your Competitive Edge With Cisco Big Data Solutions’ on October 21st.
Every Tuesday and Thursday from now until October 21st, we’ll post another blog in the series to provide you with additional details of Cisco’s full line of products, solutions and services.
View additional blogs in the series:
9/25: Unlock Big Data with Breakthroughs in Management Automation
9/30: Turbocharging New Hadoop Workloads with Application Centric Infrastructure
10/2: Enable Automated Big Data Workloads with Cisco Tidal Enterprise Scheduler
10/7: To Succeed with Big Data, Enterprises Must Drop an IT-Centric Mindset: Securing IoT Networks Requires New Thinking
10/9: Aligning Solutions to meet our Customers’ Data Challenges
10/14: Analytics for an IoE World
Please let me know if you have any comments or questions, or via Twitter at @CicconeScott.
Tags: ACI, analytics, Big Data, blade server, Blade Servers, Cisco UCS, Cisco UCS C240 M3 Rack Server, Cisco Unified Computing System, Cisco Unified Data Center, Cisco Unified Fabric, Cloudera, data virtualization, Hadoop, Hortonworks, Internet of Everything, IoE, MapR, rack server, security, UCS Central, UCS service profiles
Every day, security threats continue to evolve, as cyber attackers continue to exploit gaps in basic security controls. In fact, the federal government alone has experienced a 680% increase in cyber security breaches in the past six years, and cybersecurity attacks against the US average 117 per day. Globally, the estimated annual cost of cybercrime is over $100 billion. Often, even when security breaches are identified, it can be extremely difficult to figure how they happened or who is responsible.
One company working hard to prevent these threats is Solutionary, a managed security services provider (MSSP) that actively monitors their customers’ technology systems in order to identify and thwart security events before any negative impacts occur.
In order to provide real-time analytics of client traffic and user activity, Solutionary, a wholly owned subsidiary of NTT Group, developed a patented Solutionary ActiveGuard® Security and Compliance Platform which correlates data across global threats and trends in order to quickly identify security alerts and provide clients with actionable alerts.
The patented, cloud-based ActiveGuard® Security and Compliance Platform is the technology behind Solutionary Managed Security Services
In order to keep up with growing data volumes, the need for fast security analytics, and their expanding client base, Solutionary needed to find a way to quickly scale their infrastructure, as their traditional server infrastructure was not able to easily scale and support in-depth analysis. Their challenge was to figure out how to:
1) Increase their data analytics capabilities and improve their clients’ security
2) Cost-effectively scale as their clients/data volume grows
When a security threat occurred in the past, the legacy systems could only be used to analyze log data; they couldn’t see the big picture. Thus, when an event happened, it would sometimes take weeks of forensics work to figure out what had occurred. In order to meet these challenges, Solutionary turned to the MapR Distribution for Hadoop running on the Cisco Unified Computing System™. By using Hadoop, Solutionary was able to smoothly analyze both structured and unstructured data on a single data infrastructure, instead of relying on a costly traditional database solution that couldn’t pull in both structured and unstructured data into a single platform for analysis.
Cisco UCS Common Platform Architecture for Big Data
Specifically, the Cisco/MapR environment consists of two MapR clusters of 16 Cisco UCS C240 M3 Rack Servers. Solutionary uses the Cisco UCS Manager to provision and control their servers and network resources, while the Cisco UCS 6200 Series Fabric Interconnects provide high-bandwidth connections to servers, and act as centralized management points for the Cisco infrastructure, eliminating the need to manage each element in the environment separately. Because of the environment’s high scalability, it’s easy for the fabric interconnects to support the large number of nodes needed for MapR clusters. Scalability is improved even further by using the Cisco UCS 2200 Series Fabric Extenders to extend the network into each rack.
Cisco UCS Components
With MapR and the Cisco UCS CPA for Big Data environment, Solutionary can now access a much greater amount of data analysis and contextual data, giving them a more informed picture of behavior patterns, anomalous activities, and attack indicators. By quickly identifying global patterns, Solutionary can identify new security threats and put them into context for their clients.
Let me know if you have any comments or questions, or via twitter at @CicconeScott.
Tags: Big Data, blade server, blades servers, C240 M3 Rack Server, Cisco UCS, Cisco Unified Computing System, Cisco Unified Data Center, Cisco Unified Fabric, Hadoop, MapR, rack server, Solutionary, UCS Central, UCS service profiles
In this week’s episode, Robert Novak (@gallifreyan) and Steve McQuerry (@smcquerry) talk about Big Data on UCS. Real world use cases including UCS Manager, UCS Central, and more.
Big Data, Big Unicorn, courtesy of Robert Novak and Steve McQuerry
This is Engineers Unplugged, where technologists talk to each other the way they know best, with a whiteboard. The rules are simple:
- Episodes will publish weekly (or as close to it as we can manage)
- Subscribe to the podcast here: engineersunplugged.com
- Follow the #engineersunplugged conversation on Twitter
- Submit ideas for episodes or volunteer to appear by Tweeting to @CommsNinja
- Practice drawing unicorns
Join the behind the scenes by liking Engineers Unplugged on Facebook.
Tags: Big Data, Cisco UCS, Hadoop, UCS Central, UCS Manager
Historical data is now an essential tool for businesses as they struggle to meet increasingly stringent regulatory requirements, manage risk and perform predictive analytics that help improve business outcomes. While recent data is readily accessible in operational systems and some summarized historical data available in the data warehouse, the traditional practice of archiving older, detail-level data on tape makes analysis of that data challenging, if not impossible.
Active Archiving Uses Hadoop Instead of Tape
What if the historical data on tape was loaded into a similar low cost, yet accessible, storage option, such as Hadoop? And then data virtualization applied to access and combine this data along with the operational and data warehouse data, in essence intelligently partitioning data access across hot, warm and cold storage options. Would it work?
Yes it would! And in fact does every day at one of our largest global banking customers. Here’s how:
Adding Historical Data Reduces Risk
The bank uses complex analytics to measure risk exposure in their fixed income trading business by industry, region, credit rating and other parameters. To reduce risk, while making more profitable credit and bond derivative trading decisions, the bank wanted to identify risk trends using five years of fixed income market data rather than the one month (400 million records) they currently stored on line. This longer time frame would allow them to better evaluate trends, and use that information to build a solid foundation for smarter, lower-risk trading decisions.
As a first step, the bank installed Hadoop and loaded five years of historical data that had previously been archived using tape. Next they installed Cisco Data Virtualization to integrate the data sets, providing a common SQL access approach that made it easy for the analysts to integrate the data. Third the analysts extended their risk management analytics to cover five years. Up and running in just a few months, the bank was able to use this long term data to better manage fixed income trading risk.
Tags: Archiving, Big Data, Cisco Data virtualization, data integration, Hadoop, Historical Data