Cisco Blogs


Cisco Blog > Data Center and Cloud

Turbocharging new Hadoop workloads with Application Centric Infrastructure

At the June Hadoop Summit in San Jose, Hadoop was re-affirmed as the data center “killer app,” riding an avalanche of Enterprise Data, which is growing 50x annually through 2020.  According to IDC, the Big Data market itself growing six times faster than the rest of IT. Every major tech company, old and new, is now driving Hadoop innovation, including Google, Yahoo, Facebook Microsoft, IBM, Intel and EMC – building value added solutions on open source contributions by Hortonworks, Cloudera and MAPR.  Cisco’s surprisingly broad portfolio will be showcased at Strataconf in New York on Oct. 15 and at our October 21st executive webcast.  In this third of a blog series, we preview the power of Application Centric Infrastructure for the emerging Hadoop eco-system.

Why Big Data?

Organizations of all sizes are gaining insight and creativity into use cases that leverage their own business data.

BigDataUseCaseTable

The use cases grow quickly as businesses realize their “ability to integrate all of the different sources of data and shape it in a way that allows business leaders to make informed decisions.” Hadoop enables customers to gain insight from both structure and unstructured data.  Data Types and sources can include 1) Business Applications -- OLTP, ERP, CRM systems, 2) Documents and emails 3) Web logs, 4) Social networks, 5) Machine/sensor generated, 6) Geo location data.

IT operational challenges

Even modest-sized jobs require clusters of 100 server nodes or more for seasonal business needs.  While, Hadoop is designed for scale out of commodity hardware, most IT organizations face the challenge of extreme demand variations in bare-metal workloads (non-virtualizable). Furthermore, they are requested by multiple Lines of Business (LOB), with increasing urgency and frequency. Ultimately, 80% of the costs of managing Big Data workloads will be OpEx. How do IT organizations quickly, finish jobs and re-deploy resources?  How do they improve utilization? How do they maintain security and isolation of data in a shared production infrastructure?

And with the release of Hadoop 2.0 almost a year ago, cluster sizes are growing due to:

  • Expanding data sources and use-cases
  • A mixture of different workload types on the same infrastructure
  • A variety of analytics processes

In Hadoop 1.x, compute performance was paramount.  But in Hadoop 2.x, network capabilities will be the focus, due to larger clusters, more data types, more processes and mixed workloads.  (see Fig. 1)

HadoopClusterGrowth

ACI powers Hadoop 2.x

Cisco’s Application Centric Infrastructure is a new operational model enabling Fast IT.  ACI provides a common policy-based programming approach across the entire ACI-ready infrastructure, beginning with the network and extending to all its connected end points.  This drastically reduces cost and complexity for Hadoop 2.0.  ACI uses Application Policy to:

-          Dynamically optimize cluster performance in the network
-          Redeploy resources automatically for new workloads for improved utilization
-          Ensure isolation of users and data as resources are deployments change

Let’s review each of these in order:

Cluster Network Performance: It’s crucial to improve traffic latency and throughput across the network, not just within each server.

  • Hadoop copies and distributes data across servers to maximize reliability on commodity hardware.
  • The large collection of processes in Hadoop 2.0 are usually spread across different racks.
  • Mixed workloads in Hadoop 2.0, support interactive and real-time jobs, resulting in the use of more on-board memory and different payload sizes.

As a result, server IO bandwidth is increasing which will place loads on 10 gigabit networks.  ACI policy works with deep telemetry embedded in each Nexus 9000 leaf switch to monitor and adapt to network conditions.

HadoopNetworkConditions

Using policy, ACI can dynamically 1) load-balance Big Data flows across racks on alternate paths and 2) prioritize small data flows ahead of large flows (which use the network much less frequently but use up Bandwidth and Buffer). Both of these can dramatically reducing network congestion.  In lab tests, we are seeing flow completion nearly an order of magnitude faster (for some mixed workloads) than without these policies enabled.  ACI can also estimate and prioritize job completion.  This will be important as Big Data workloads become pervasive across the Enterprise. For a complete discussion of ACI’s performance impact, please see a detailed presentation by Samuel Kommu, chief engineer at Cisco for optimizing Big Data workloads.

 

Resource Utilization: In general, the bigger the cluster, the faster the completion time. But since Big Data jobs are initially infrequent, CIOs must balance responsiveness against utilization.  It is simply impractical for many mid-sized companies to dedicate large clusters for the occasional surge in Big Data demand.   ACI enables organizations to quickly redeploy cluster resources from Hadoop to other sporadic workloads (such as CRM, Ecommerce, ERP and Inventory) and back.  For example, the same resources could run Hadoop jobs nightly or weekly when other demands are lighter. Resources can be bare-metal or virtual depending on workload needs. (see Figure 2)

HadoopResourceRedeployment

How does this work? ACI uses application policy profiles to programmatically re-provision the infrastructure.  IT can use a different profile to describe different application’s needs including the Hadoop eco-system. The profile contains application’s network policies, which are used by the Application Policy Infrastructure controller in to a complete network topology.  The same profile contains compute and storage policies used by other tools, such as Cisco UCS Director, to provisioning compute and storage.

 

Data Isolation and Security: In a mature Big Data environment, Hadoop processing can occur between many data sources and clients.  Data is most vulnerable during job transitions or re-deployment to other applications.  Multiple corporate data bases and users need to be correctly to ensure compliance.   A patch work of security software such as perimeter security is error prone, static and consumes administrative resources.

ACI_SecurityImperatives

 

In contrast, ACI can automatically isolate the entire data path through a programmable fabric according to pre-defined policies.  Access policies for data vaults can be preserved throughout the network when the data is in motion.  This can be accomplished even in a shared production infrastructure across physical and virtual end points.

 

Conclusion

As organizations of all sizes discover ways to use Big Data for business insights, their infrastructure must become far more performant, adaptable and secure.  Investments in fabric, compute and storage must be leveraged across, multiple Big Data processes and other business applications with agility and operational simplicity.

Leading the growth of Big Data, the Hadoop 2.x eco-system will place particular stresses on data center fabrics. New mixed workloads are already using 10 Gigabit capacity in larger clusters and will soon demand 40 Gigabit fabrics.  Network traffic needs continuous optimization to improve completion times.  End to end data paths must use consistent security policies between multiple data sources and clients.  And the sharp surges in bare-metal workloads will demand much more agile ways to swap workloads and improve utilization.

Cisco’s Application Centric Infrastructure leverages a new operational and consumption model for Big Data resources.  It dynamically translates existing policies for applications, data and clients in to fully provisioned networks, compute and storage. .  Working with Nexus 9000 telemetry, ACI can continuously optimize traffic paths and enforce policies consistently as workloads change.  The solution provides a seamless transition to the new demands of Big Data.

To hear about Cisco’s broader solution portfolio be sure to for register for the October 21st executive webcast  ‘Unlock Your Competitive Edge with Cisco Big Data Solutions.’  And stay tuned for the next blog in the series, from Andrew Blaisdell, which showcases the ability to predictably deliver intelligence-driven insights and actions.

Tags: , , , , , ,

The Future of Networking, Available Today

I am Soni Jiandani, SVP of Marketing for Cisco’s Insieme Business Unit. Together with a team of veteran leaders and engineers, we continue to disrupt markets to drive industry transformation. Our latest disruption is focused on leapfrogging Software Defined Networks (SDN) with a holistic approach to the future of networking: Application Centric Infrastructure, or ACI for short.

My blog is timed with announcing the shipment of ACI – namely the Application Policy Infrastructure Controller (APIC) with ACI mode for the Nexus 9000.  But this is not a corporate sales blog. My intent is to foster an open discussion about the future of the networking industry.

ACI: A key enabler to driving fast IT

We have spent the past few years to gather the best and the brightest engineering minds focused on one simple goal:  to design an infrastructure for our customers that meets the needs of applications today and in the future. These applications require dynamic, agile, fast, secure, scalable, reliable infrastructure that is automated as a native, baseline requirement.

Read More »

Tags: , , , , , , , , ,

Nexus Flexibility Eases Transitions

Cisco has a broad base of data center customers with a diverse set of requirements and we meet their needs with Nexus -- the most comprehensive switching portfolio in the industry.  This week, we are making announcements for both the Nexus 9000 series and the Nexus 3000 series that provide design and deployment flexibility for our commercial, enterprise, service provider, as well as cloud customers.  Key points of the announcement include:

  • ACI (Application Centric Infrastructure) is shipping this month;
  • Additional linecard and chassis options provide customer choice and flexibility;
  • 100G linecards for the Nexus 9500 will be available in Q4CY14 and will offer the highest density in the industry; and
  • New starter kits and bundles help customers ease transitions.

Nexus 9000 family

The Nexus 9000 Series

 ACI is shipping this month

 The Nexus 9000 series can operate in standard NX-OS mode or in ACI mode. In either case the Nexus 9000 portfolio delivers the value of the “5 P’s” of Power efficiency, Price, Port density, Performance, and Programmability. NX-OS mode provides customers with the value of the NX-OS operating system used by tens of thousands of customers in data centers around the world.  ACI mode adds to NX-OS capabilities by providing an application driven policy model, integration of hardware and software, and centralized visibility, among other things. ACI requires a controller and switch software.  Both are shipping this month. It is important to note that the pricing for this solution is simple and predictable.  There is a perpetual license for each leaf switch.  Other pricing approaches in the industry are monthly and are based on varying elements like number of VM’s.  Comparing the two approaches is somewhat like comparing a cell phone bill that is either flat rate or usage based.  Personally, I like the simplicity and predictability of flat rate.  See The Future of Networking, as well as SDN and Beyond  for additional details on new ACI announcements and how they can take you beyond SDN.

Additional linecard and chassis options underscore flexibility

We’ll consider how flexibility is delivered for both modular and fixed platforms.  For modular switching, the Nexus 9500 modular chassis family offers different line card options that can be mixed in the same chassis and allow customers to “dial up” or “dial down” their design based upon the price, performance, feature set, and scale they want to achieve.  There are basically 3 different ‘flavors’, all of which are now shipping:

  1. The Nexus 9500 X9400 set of 1/10G and 40G line cards are based on merchant silicon and provide industry-leading price and performance compared to other merchant silicon switches. These provide a very cost effective solution ideal for traditional modular data center designs.
  2. The Nexus 9500 X9500 set of 1/10G and 40G line cards are sometimes referred to as “merchant plus” because they have custom Cisco ASICs, in addition to merchant silicon, and are ideal for customers that need performance together with additional buffering and VXLAN routing capabilities. The X9500 line cards can be used in future ACI designs as well.
  3. The Nexus 9500 X9600 set of 40G line cards provide performance without compromise even for small packet sizes.

The Nexus 9300 series offers ACI capabilities (ala the X9500 linecards in item 2 above) in a fixed form factor.  For customers interested in a merchant only fixed form factor, we offer the Nexus 3000 family.  This week, we announced the new Nexus 3164, which provides 64 ports of 40G and is a great solution for 40G access or space constrained aggregation.

100G linecards

We are also announcing 100G linecards that we believe will deliver industry leading port density of up to 128 ports of 100G in a single chassis.  100G for both the X9400 and X9600 series will be available for the Nexus 9500 in Q4CY14.  Cisco will offer an 8 port 100G X9400 line card and a 12 port 100G X9600 line card.

New starter kits and bundles ease transitions

There are numerous packages available to ease transitions -- from 1G to 10G, 10G to 40G, or from traditional networks to ACI.  There are 2 bundles I want to quickly call out.  The first provides a smooth transition for customers with older End of Row Catalyst 6500’s in their data centers.  It occupies the same rack space and uses the same cabling as they currently have, but provides 10X the performance.  The second is basically an ACI starter kit, providing the APIC, spine switches and leaf switches, even optical cables – everything required to set up and get started with an ACI pod.

In summary, Cisco is continuing its rapid pace of innovation and execution around ACI and data center switching overall.  Ultimately, this means customers gain choice, flexibility and true innovation to support their business needs.

Tags: , , , , , , , , , , , ,

Cisco Network Convergence System (NCS) 6000 wins Best-of-Show Award at Interop Tokyo 2014

June 15, 2014 at 7:29 pm PST

dawoodWritten By Sultan Dawood, Marketing Manager

Internet of Things, Cloud Computing, SDN, NFV, Next generation Data Centers and Wireless Broadband are all technology themes that were discussed at the annual Interop 2014 show in Tokyo, Japan this week.

The week-long event which was co-located with IMC Tokyo 2014, DSJ 2014, Apps Japan and Location Business Japan 2014 saw over 140,000 attendees. With participation from 100+ vendors, Cisco led the event as one of three Global Partner Sponsors and provided one of the keynotes presented by David Ward, Senior Vice President and Chief Technology Officer. Read More »

Tags: , , , , ,

Cisco ACI at F5 Agility 2014, Copenhagen

At F5 Agility 2014 Copenhagen this month, Applications take the central stage.  The key focus area is Application Delivery Controllers (ADC’s as Gartner calls them) and how they are increasingly becoming more important to modern IT than they used to be, allowing scale, availability, orchestration and provisioning.

F5-1

Another key focus area is around the deployment of applications and how joint technology solutions present a tremendously powerful option for F5’s customers.  F5’s partners -- and Cisco is a key partner – are a large part of Agility.  Cisco is at the event to demonstrate how its ACI technology integrates with F5 BIG-IP to improve manageability, strengthen security, and ensure faster and more successful deployments.

F5-2

We have quite a few exciting things that we are showcasing at F5 Agility from a Cisco ACI perspective, and in this blog I want to take you on a quick tour of the highlights. As a testimony to our growing momentum with the Cisco ACI-F5 joint solution effort, we have demos, business and technical breakout sessions. The event features F5 CEO John McAdam’s keynote on Tuesday June 17, where he will … discuss how applications are impacting the architecture of the data center and driving IT strategy and alignment to key business drivers.

F5-3

Cisco Marketing Manager Ravi Balakrishnan is presenting a business breakout session-4 titled “Unleash the power of Cisco ACI and F5 Synthesis for Accelerated Application deployments,” on Wednesday June 18. This session gives a detailed overview on the benefits of our joint solution and the customer pain points it addresses, so do not miss this. I’d also encourage you to attend Paolo Pio and Nicolas Menant’s technical breakout session-3 on Wednesday June 18, where they will walk you through details of ACI/VMDC joint solution.

We also have an exciting demo at the Cisco ACI kiosk, where we are showing how the Cisco ACI and F5 BIG-IP joint solution works with step-by-step illustrations of configuration, deployment and execution. The demos run Tuesdays and Wednesday at the Exhibit Hall for the entire duration of the day. Stop by our demo kiosk to get a deep-dive architecture type white-boarding or brainstorming type engagement with Cisco subject matter experts on ACI-F5 integrated solution. There are several other solution areas where Cisco and F5 are working together including F5 LTM-Cisco Nexus 7000 integration, F5 LTM-SourceFire NGIPS integration etc, so come by our demo booth to learn more.

To learn more on Cisco and F5  partnership visit the Cisco-F5 Alliance web page.

 

Tags: , , , , , , , ,