Cisco Blogs

Cisco Blog > Data Center

Experience Day 0, 1, 2 and N Operations @ PuppetConf

Craig Huitema blogged about Cisco’s SDN strategy and one of the key pillars is programmable networks. Cisco’s programmable networks is based on Nexus operating system NX-OS and our Robb Boyd from TechWiseTV covers it here and goes in more depth about NX-API REST (Object model) here and here.

Also go here if you missed our September 25th SDxCentral DemoFriday where we looked at use cases and demos related to NX-Toolkit and NX-API REST. Bottom line is to drive operational agility in the data center by enabling IT admins to manage Nexus switches as a Linux server with open interfaces and integrating DevOps tools.

One of the DevOps tools is Puppet. Integrating Puppet Enterprise agent is an integral part of programmable networks as I touched on it in my previous blog.

As we break lifecycle management into Day 0, 1, 2 and N to install, configure, optimize and upgrade the network to meet application and user requirements, Puppet plays a key role in each step.

Day0_1_2_N Operations

Come and visit Cisco’s booth at PuppetConf October 7 – 9 to see demos and learn more about the integration of Puppet and its benefits on Day 0, 1, 2, and N. Also, visit our sponsor theater on Thursday, Oct 8 at 12:10 PM in the main exhibit hall as well as our breakout session Friday, October 9 at 2:30 PM. We will share how Cisco’s strategy of openness has helped the developer community.

To stay up to date on the latest version of the CiscoPuppet Module source code, visit this GitHub repository that allows network administrators to manage Cisco Network Elements using Puppet.

Tags: , , , , , , , , ,

Open NX-OS featured on TechWiseTV

Only on TechWiseTV

Only on TechWiseTV

This is the first in a multi-part series where we cover ‘programmability’ for networking.  The idea is to fully review the programming options now available inside the Nexus switches, (3000, 9000).  This first episode covers new access with Linux tools, NX-API and more. Further shows will be diving into the details around Object Models and orchestration partners.

The primary point for any of these is to understand how Cisco Open NX-OS extensibility exposes greater programmability and automation capabilities.  It is fascinating and full of new learning opportunities.  It does not come without a few career questions of course…usually, something along the lines of: do network engineers need to become programmers now too?  

Two answers:  Yes.  It depends.

Networking knowledge and skill should not be undersold here.  Programming capabilities should be additive.  They are useful in just about any tech career and obviously affecting the networking space.  I think it’s foolish to ever quit learning but it does depend on your aspirations, your current level of satisfaction and perhaps how narrowly defined your skill set might be.

Full disclosure: I am not a programmer. I have been learning the fundamentals of python and a few others as I work on this series but I am not hire-able for this skill by any means. But the distinct feeling I get, and the feedback I hear from you guys: its not that hard.  You are probably well versed in scripting for various CLI operations…take it up a few notches and work on some of these ‘readable’ languages that will have similar syntax.  This will give you the ability to judge the appeal of what we are offering with ACI and other solutions much more credibly…and I guarantee you will find ways to get rid of redundant crap and stupid errors you may be fighting with yourself or your team.


Live, interactive, never dull.

September 21, 2015

Programmable networks will forever change the way you manage infrastructure enabling you to dramatically accelerate configuration and deployment of your network, automate time consuming manual tasks, and allocate IT resources far more efficiently.  Are you ready for the revolution?  

Discover how to create a programmable network as we discuss and demonstrate the NX-API and NX-API REST (Object Model) in detail. Understand how Cisco Open NX-OS extensibility exposes greater programmability and automation capabilities that eliminate costly manual errors.

– You can sign up at the workshop tab when the date gets a bit closer,


Nicolas Delecroix in the TechWiseTV Lab

Nicolas Delecroix in the TechWiseTV Lab

TechWiseTV 176: Open NX-OS: Programmability


Two great experts on this episode.

Six Key Points: What OPEN means for NX-OS

Six Key Points: What OPEN means for NX-OS

Shane Corban shares Six Key Points: What OPEN means for NX-OS

Changes made across the software stack to address Extensibility, Openness, Programmability.

  1. Auto Deployment (Bootstrap and Provisioning)
    • Added support for PXE server, operationalize NX-OS software to match an existing server environment
  2. Extensibility – how we package software
    • We did not use to expose much beyond a bash shell
    • Now you can install native RPM’s, and third party applications running processes as they would on a Linux server
  3. Open Interfaces
    • We are now adding support to leverage Linux like tools for debugging, configuration and troubleshooting…manipulate those front panel ports as native Linux interfaces within our switch software stack.
  4. Application Integration (Adaptable SDK)
    • Published an SDK, a build environment that you can install on any Linux server, download the build agent, and put your source into that directory structure and build into an RPM for installation and run it natively.
    • Build your own custom automation apps, monitoring agents, and have them run natively on our platform
  5. Programmability Tool Choice
    • Sandbox
    • We have a native Python shell today that has a Native Cisco Library that you can utilize for automation
    • NX-API – the ability to embed CLI commands and structured data (JSON, XML) for execution on the switch via HTTP/HTTPS Interface to get back structured data back on show commands.
  6. Management Tools
    • Support for Chef and Puppet
    • Agents will be publicly available on the enterprise sites
    • Support for Open Stack, Neutron

NX-OS is now more modular, more open, more capable of third party integration providing a  wide variety of programmability choices ideal for Dev-Ops environments.  

Five case study examples

Five case study examples

Nicolas provides five case study examples. 

  1. Checking Software Version
  2. Using Python script with NXAPI and JSON to pull version numbers
  3. Python script to query multiple switches to check compliance against a specific version
  4. VLAN Provisioning
  5. Checking for proper VLAN provisioning

Special thanks behind the scenes to Rami Rammaha and Mark Jackson


More Reading:

Cisco Nexus 9000 Programmability Guide

Matt Oswalt is a great writer. You should follow his blog: Keeping it Classless.   I enjoy his angles on things.  Read up on his blog entry: Evolution of Network Programmability, Nexus 9000 NX-API,NX-API Update.

Some Learning Basics:

What do you think still needs to be covered?  I would love any thoughts on how the rest of this series should be shaped.  Leave your comments below and just to make sure…tag me on twitter.  We are diving into Object Models (taping next week) and then some angle with the Orchestration Partners.   Case in point: Puppet Labs is making available today a native Puppet NX-OS agent and Cisco Puppet Module.

Let me know!


Tags: , , , , , , , , , , , ,

The Puppet Labs Integration With NX-OS Is Here

As we continue our journey of openness that is summarized by ZK Research: Cisco’s Data Center Strategy is Built on Openness, we announced the Open NX-OS at Cisco Live San Diego in June 2015 that runs on Nexus 3K and Nexus 9K platforms.

The Open NX-OS extensibility supports:

  • Object store and model-driven NX-API enhancements. NX-API enables common programmatic approach across entire Nexus switch portfolio (Nexus 2000 through Nexus 9000 switches)
  • Built-in third party DevOps automation tools like Puppet
  • Secure SDK enabling third party and custom application development running natively on NX-OS

The new programmability features in Open NX-OS, such as the bash shell environment, python interpreter and NX-API access, it enables the built-in DevOps Puppet tool to be extended to automate anything on the platform. Cisco and Puppet Labs are excited to make available the Puppet Cisco [NX-OS agent]


and Cisco [Puppet Forge Module]


Companies are embracing software defined networking (SDN) and DevOps practices to deploy network changes repeatedly and consistently. Customers who run mega scale data centers like Web2.0/OTT and fortune 100 are looking to do more with less, increase “device:admin” ratio and agility, and respond faster to business needs in a world where continuous application update grows by the hour without breaking infrastructure operation.

Using Puppet Enterprise, you can not only realize those SDN benefits, but you also extend DevOps practices to network administration across mega scale data centers, commercial and  large enterprises by defining your desired network configuration with infrastructure as code.  Using infrastructure as code enables cross-team change collaboration, automated infrastructure testing, and automated application deployments that span compute, storage, and network.

Tags: , , , , , ,

Open Networking with Cisco ACI and Open NX-OS

Over the last couple of years, Data Centers have become a key focus on networking innovations, particularly around the broad area of Software Defined Networking (SDN).  At Cisco, for nearly one year, we have been shipping our new way to build a Data Center with our Application Centric Infrastructure (ACI) solution. ACI relies on the vision of using a policy based methodology to enable network switches, services, and hypervisors to establish network connectivity among each other.

At the Open Networking User Group (ONUG), a survey reports that 3% of the networks run by ONUG members are built on open networking whereas 71% are not open at all.  The assessment of any system being declared “open” is a subjective term.  At Cisco we have built a foundational infrastructure in ACI, that relies on open protocols and programming constructs such as APIs, in order to provide a solution where the network becomes ‘invisible’ to the end user, and the network devices and services modules, such as firewalls, load balancers, physical and, virtual switches are automatically configured based on the end user intent. Read More »

Tags: , , , ,

Why Upgrade to MDS 9700

MDS 9500 family has supported customers for more than a decade helping them  through FC speed transitions from 1G, 2G, 4G, 8G and 8G advanced without forklift upgrades. But as we look in the future the MDS 9700 makes more sense for a lot of data center designs.  Top four reasons for customers to upgrade are

  1. End of Support Milestones
  2. Storage Consolidation
  3. Improved Capabilities
  4. Foundation for Future Growth

So lets look at each in some detail.

  1. End of Support Milestones

MDS 4G parts are going End of Support on Feb 28th 2015. Impacted part numbers are DS-X9112, DS-X9124, DS-X9148. You can use the MDS 9500 Advance 8G Cards or MDS 9700 based design. Few advantages MDS 9700 offers over any other existing options are

a. Investment Protection – For any new Data Center design based on MDS 9700 will have much longer life than MDS 9500 product family. This will avoid EOL concerns or upgrades in near future. Thus any MDS 9700 based design will provide strong investment protection and will also ensure that the architecture is relevant for evolving data center needs for more than a decade.

b. EOL Planning – With MDS 9700 based design you control when you need to add any additional blades but with MDS 9500, you will have to either fill up the chassis within 6 months (End of life announcement to End of Sales) or leave the slots empty forever after End of Sale date.

c. Simplify Design – MDS 9700 will allow single skew, S/W version, consistent design across the whole fabric which will simplify the management. MDS 9700 massive performance allows for consolidation and thus reducing footprint and management burden.

d. Rich Feature Set – Finally as we will see later MDS provides host of features and capabilities above and beyond MDS 9500 and that enhancement list will continue to grow.

 Tech Refresh Example v1

  1. Storage Consolidation

MDS 9700 provides unprecedented consolidation compared to the existing solutions in the industry. As an example with MDS 9710 customers can use the 16G Line Rate ports to support massively virtualized workload and consolidate the server install base. Secondly with 9148S as Top of Rack switch and MDS 9700 at Core, you can design massively scalable networks supporting consistent latency and 16G throughput independent of the number of links and traffic profile and will allow customers to Scale Up or Scale Out much more easily than legacy based designs or any other architecture in the industry.

Moreover as shown in figure above for customers with MDS 9500 based designs MDS 9710 offers higher number of line rate ports in smaller footprint and much more economical way to design SANs. It also enables consolidation with higher performance as well as much higher availability.

Small SANs to Large SAN Design v1


  1. Improved Capabilities

MDS 9700 design provides more enhanced capabilities above and beyond MDS 9500 and many more capabilities will be added in future. Some examples that are top of mind are detailed below

Availability: MDS 9700 based design improves the reliability due to enhancements on many fronts as well as simplifying the overall architecture and management.

    • MDS 9710 introduced host of features to improve reliability like industry’s first N+1 Fabric redundancy, smaller failure domains and hardware based slow drain detection and recovery.
    • Its well understood that reliability of any network comes from proper design, regular maintenance and support. It is imperative that Data Center is on the recommended releases and supported hardware. As an example data center outage where there are unsupported hardware or software version failure are exponentially more catastrophic as the time to fix those issues means new procurement and live insertion with no change management window. Cost of an outage in an Data Center is extremely high so it is important to keep the fabric upgraded and on the latest release with all supported components. Thus for new designs it makes sense that it is based on the latest MDS 9700 directors, as an example, rather than MDS 9513 Gen-2 line cards because they will fall of the support on Feb 28, 2015. Also a lot of times having different versions of the hardware and different software versions add complexity to the maintenance and upkeep and thus has a direct impact on the availability of the network as well as operational complexity.


With massive amounts of virtualization the user impact is much higher for any downtime or even performance degradation. Similarly with the data center consolidation and higher speeds available in the edge to core connectivity more and more host edge ports are connected through the same core switches and thus higher number of apps are dependent on consistent end to end performance to provide reliable user experience. MDS 9700 provides industries highest performance with 24Tbps switching capability. The Director class switch is based on Crossbar architecture with Central Arbitration and Virtual Output Queuing which ensures consistent line rate 16G throughput independent of the traffic profile with all 384 ports operating at 16G speeds and without using crutches like local switching (muck akin to emulating independent fixed fabric switches within a director), oversubscription (can cause intermittent performance issues) or bandwidth allocation.


MDS Directors are store and forward switches this is needed as it makes sure that corrupted frames are not traversing everywhere in the network and end devices don’t waste precious CPU cycles dealing with corrupted traffic. This additional latency hit is OK as it protects end devices and preserves integrity of the whole fabric. Since all the ports are line rate and customers don’t have to use local switching. This again adds a small latency but results in flexible scalable design which is resilient and doesn’t breakdown in future. These 2 basic design requirements result in a latency number that is slightly higher but results in scalable design and guarantees predictable performance in any traffic profile and provides much higher fabric resiliency .

Consistent Latency: For MDS directors latency is same for the 16G flow to when there are 384 16G flows going through the system. Crossbar based switch design, Central arbitration and Virtual Output Queuing guarantees that. Having a variable latency which goes from few us to a high number is extremely dangerous. So first thing you need to make sure is that director could provide consistent and predictable latency.

End to End latency: Performance of any application or solution is dependent on end to end latency. Just focusing on SAN fabric alone is myopic as major portion of the latency is contributed by end devices. As an example spinning targets latency is of the order of ms. In this design few us is orders of magnitude less and hence not even observable. With SSD the latency is of the order of 100 to 200 us. Assuming 150 us the contribution of SAN fabric for edge core is still less than 10%. Majority (90%) of the latency is end devices and saving couple of us in SAN Fabric will hardly impact the overall application performance but the architectural advantage of CRC based error drops and scalable fabric design will make provided reliable operations and scalable design.


For larger Enterprises scalability has been a challenge due to massive amount of host virtualization. As more and more VMs are logging into the fabric the requirement from the fabric to support higher flogins, Zones. Domains is increasing. MDS 9700 has industries highest scalability numbers as its powered by supervisor that has 4 times the memory and compute capability of the predecessor. This translates to support for higher scalability and at the same time provides room for future growth.

MDS 9700 Capabilities v2

Foundation for Future Growth:

MDS 9700 provides a strong foundation to meet the performance and scalability needs for the Data Center requirements but the massive switching capability and compute and memory will cover your needs for more than a decade.

    • It will allow you to go to 32G FC speeds without forklift upgrade or changing Fabric Cards (rather you will need 3 more of the same Fabric card to get line rate throughput through all the 384 ports on MDS 9710 (and 192 on MDS 9706).
    • MDS 9700 allow customers to deploy 10G FCoE solution today and upgrade without forklift upgrade again to 40G FCoE.
    • MDS 9700 is again unique such that customers can mix and match FC and FCoE line cards any way they want without any limitations or constraints.

Most importantly customers don’t have to make FC vs FCoE decision. Whether you want to continue with FC and have plans for 32G FC or beyond or if you are looking to converge two networks into single network tomorrow or few years down the road MDS 9700 will provide consistent capabilities in both architectures.

Foundation for Future Growth v1

In summary SAN Directors are critical element of any Data Center. Going back in time the basic reason for having a separate SAN was to provide unprecedented performance, reliability and high availability. Data Center design architecture has to keep up with the requirements of new generation of application, virtualization of even the highest performance apps like databases, new design requirements introduced by solutions like VDI, ever increasing Solid State drive usage, and device proliferation. At the same time when networks are getting increasingly complex the basic necessity is to simplify the configuration, provisioning, resource management and upkeep. These are exact design paradigms that MDS 9700 is designed to solve more elegantly than any existing solution.

Although I am biased in saying that but it seems that you have voted us with your acceptance. Please see some more details here.



Live as if you were to die tomorrow. Learn as if you were to live forever.
Mahatma Gandhi



Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,