August 18, 2020 Leave a Comment
Avatar
Data Center

Network Automation and the Ingenuity of Data Models

10 min read

Network automation has evolved on Cisco switches through various features and protocols over the years. Network architects typically take a multi-pronged approach to network automation which includes aspects of network provisioning and configuration that can be automated using scripts and tools. In addition, telemetry data and operational data from devices can be used to further automate tasks and close the loop on intent-based networking. Having a suite of network automation capabilities in the enterprise is critical for innovation and continues to be a powerful investment going forward.

As Chuck Robbins mentioned last week, “We will also accelerate our investments in the following areas: cloud security; cloud collaboration; key enhancements for education, healthcare, and other industries; increased automation in the enterprise; the future of work; and application insights and analytics.”

The move to network automation, quite like the move from manual transmission to automatic transmission in automobiles, can be met with strong preferences for one way versus the other! While there are several applications for which we may prefer to use manual CLI methods, it is important to understand the value and capabilities of network automation on our switches. Many modern automation tools also use CLI on a bash shell for scripting and execution, with well-defined templates being integrated into a GUI. Once we get a handle on how to automate functions for network deployment in an efficient, predictable and consistent way, it becomes easy to apply these methods where they are most relevant. With that, let’s shift into drive and get started!

Network Automation with Open NX-OS

The introduction of model-based network programmability on our switches in recent years can be considered trailblazing in how we automate network functions. The paradigm shift to data models on our switches makes network automation a reality with the use of managed objects and their associated constructs using different toolchains. Cisco NX-OS now has new capabilities with OpenConfig and gRPC Network Management Interface (gNMI) support to provide an open and model-driven facility to automate data center networks. Open NX-OS also offers different methods of API abstractions that allow us to automate key network functions with simple Python scripts.

In this article, we are going to cover two new frameworks for network automation using Open NX-OS methods based on Python 3.0:

  1. PyDME: provides Python abstraction using Cisco DME and REST API methods
  2. cisco-gnmi: wraps gNMI implementation using OpenConfig and gNMI/gRPC methods

We will illustrate the use of these tools with a simple example using the IEEE protocol LLDP (Link Layer Discovery Protocol). We would like to detect Linux hosts connected to a switch and automatically configure the associated ports using a pre-defined template. In this scenario, we parse through the LLDP neighbors of a switch. If we find a Linux host attached to an ethernet port, we configure that port as a trunk. With the use of data models, we illustrate how this can be done with just a few lines of code, using the object structure to extract and manipulate the very specific attributes of configuration as desired. This basic example can be extrapolated to more complex deployment scenarios.

Both PyDME and cisco-gnmi consist of libraries that are installed off-box. They completely abstract the methods used to access NX-OS switches and retrieve data or apply configuration. While PyDME accesses the NX-OS managed objects through the Data Management Engine (DME) using REST API methods, the cisco-gnmi tool leverages OpenConfig and device YANG data models with gNMI/gRPC methods. It is not really fair to compare the two methods. However, I will be highlighting how each of them can be used to solve the same task. I’ll provide pointers to the actual code that implements this example and illustrate the value of the two different toolchains in this article. All code has been done using Python 3.0.

Note: These are not officially supported Cisco products, but can be used to streamline implementation with released Cisco NX-OS features such as REST API and gNMI.

Method 1: Open NX-OS Automation with PyDME

PyDME is tool that provides a Python abstraction over REST API using Cisco DME to access managed objects. It provides API constructs to access the switch and configure it. The library is available at the repository linked here.

To use it, install the library onto a host that has connectivity to your switches. Then setup a simple script in Python to perform the required task. The script runs on the host and uses the PyDME library to configure your switches and retrieve configuration and operational data from them using REST methods.

Switch Configuration

The example we use includes a Nexus 9000 with NX-OS Release 9.3(5). We have “feature nxapi” enabled on the switch. For our specific example, we also have “feature lldp” enabled, but this configuration can be included within the automation script.

Installation on the Host

PyDME can be installed on your host using a Docker install or a pip install (pip3 where appropriate). The required packages are installed, and you can optionally also install the associated utils to retrieve information about the managed object tree.

Code Constructs

To code your script, it’s helpful to understand the different constructs that PyDME uses.  These API constructs achieve tasks that would otherwise be done via REST API.

Node: To begin with, we define a node, which abstracts the switch we are about to access. The node is specified using the REST URL for the IP address of the switch. There are two associated methods used to access the switch: Login and LoginRefresh. Login is used to access the switch using a POST() method. LoginRefresh uses a GET() operation to prevent the session from timing out. Once we establish access to the switch using the username and password, we can then begin to apply REST API calls.

my_switch = Node(host_url)
result = my_switch.methods.Login(username,password).POST()

Managed Objects: We instantiate DME managed objects locally from the node using the “mit” which represents the Managed Information Tree (MIT). PyDME requires a thorough understanding of DME and its data models. It is important to note that each time the “mit” property is invoked on a node, it generates a different Managed Information Tree which the PyDME script uses as a local cache. Once this done, we can use GET/POST/DELETE methods to retrieve, post and delete data respectively using their corresponding REST operations.

Here is a snippet of the code for a GET and a POST operation. The GET() method illustrated below queries the lldpAdjEp model and all its children, which includes information about the switch’s LLDP neighbors.

mit = my_switch.mit
lldp_neighbors = mit.GET(**options.subtreeClass('lldpAdjEp'))

If we stop here and look at the structure of data in lldp_neighbors, this is what it looks like:

"lldpAdjEp": {
"attributes": {
"capability": "bridge,router,station,wlan",
"chassisIdT": "mac",
"chassisIdV": "0050.56b4.4bf0",
"childAction": "",
"dn": "sys/lldp/inst/if-[eth1/3]/adj-1",
<--- snip --->
"sysDesc": "Ubuntu 18.04.4 LTS Linux 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64",
"sysName": "dirao-lnx1.aci.local"
}
}

We will iterate through all the lldpAdjEp instances and extract the interface ID from the “dn” attribute when the sysDesc attribute matches the string ‘Linux’.

If you’re wondering how to know which model to use, the DME model reference is a good resource. In parallel, the PyDME repository includes a util called buildMoTree.py that allows us to find the model and attributes we desire.

divya@Ubuntu-host:~/pydme/utils$ python3 buildMoTree.py ../archive/dme-9.3.5-meta.json lldpAdjEp | grep -A 3 properties
properties of lldpAdjEp:
['capability', 'chassisIdT', 'chassisIdV', 'childAction', 'dn', 'enCap', 'id', 'mgmtId', 'mgmtIp', 'mgmtPortMac', 'modTs', 'monPolDn', 'name', 'persistentOnReload', 'portDesc', 'portIdT', 'portIdV', 'portVlan', 'rn', 'stQual', 'status', 'sysDesc', 'sysName', 'ttl']

Once we detect a Linux neighbor, we will set the configuration of the associated port as a trunk using POST(). To do this, we will need to re-initialize “mit” since we access “InterfaceEntity”, which is in a different branch of the Managed Information Tree. Here, we can modify the required attributes as part of the POST() method.

if_status = mit.topSystem().interfaceEntity().l1PhysIf(lldp_if)
if_status.mode = 'trunk'
if_status.trunkVlans = '1 - 512'
result_config = if_status.POST()

Now, we’re going to execute the script on our host and then verify the switch configuration for interface eth1/3.

divya@Ubuntu-host:~$ python3 pyDME-neighbor-trunk.py
We will set eth1/3
Ubuntu 18.04.4 LTS Linux 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64
eth1/3 has been configured as a trunk

The references for this example can be found at the PyDME documentation, the DME model reference and the REST SDK guide. For the complete code, you can refer to this GitHub link and look for the script pyDME-neighbor-trunk.py.

Method 2: Open NX-OS Automation with gNMI and OpenConfig

My previous blog post referenced the gNMI support we have on Nexus 9000 switches with OpenConfig and YANG. We discussed the different gRPC operations supported with gNMI, namely, CapabilitiesRequest, GetRequest, SetRequest and SubscribeRequest. We then illustrated the process of using gNMI Subscribe to subscribe to telemetry data on the switch and stream it to an open source collector, Telegraf. In this article, we will describe a tool that abstracts Capabilities, Get, Set and Subscribe using gNMI and OpenConfig. We are going to illustrate the same example above with LLDP and use this tool to “Get” and “Set” our data using gNMI.

The repository for this tool can be found in GitHub here. The library “cisco-gnmi-python” wraps the gNMI implementation to facilitate ease of use of Python programs with different Cisco implementations (IOS-XE, IOS-XR and NX-OS). It also includes a CLI form of the tool which can be used to implement gNMI functionality without the use of a Python script.

We will briefly go over the two methods here and leave you with a reference to the complete code.

Switch Configuration

In order to set up the switch for gNMI, we will need to follow the same steps as we did in the gNMI Subscribe example (refer to Steps 1 and 3). This includes installing the RPM packages for OpenConfig and configuring gRPC on the switch. Once we have the gRPC certificates installed on the switch, we also need to copy the certificate file (public key) onto our host where we will be installing the Cisco gNMI tool. In addition, since our example is based on LLDP, we enable “feature lldp”.

Installation on the host

Installation of the library on your host can be done with a pip or pip3 install like we did for PyDME. Once we have finished installing all the packages, we will be able to use cisco-gnmi both as a library with Python scripts as well as with the CLI tool.

pip3 install cisco-gnmi

gNMI CLI

Here is an example of how we can retrieve gNMI Capabilities using cisco-gnmi. This gives us information from the switch about gNMI versions it uses and data models and encodings it supports. The -ssl_target_override parameter overrides the hostname of our host. We also specify the credentials to access the switch including the certificate and the gRPC port number that is configured on the switch.

divya@Ubuntu-host:~$ cisco-gnmi capabilities -os NX-OS -root_certificates ./gnmi.pem -ssl_target_override dirao 172.25.74.84:50051
Username: admin
Password:
<--- snip --->
supported_models {
name: "openconfig-lldp"
organization: "OpenConfig working group"
version: "0.2.1"
}
<--- snip --->

The following CLI can be used to retrieve data from the switch using gNMI Get, for example.

divya@Ubuntu-host:~$ cisco-gnmi get -encoding JSON -data_type STATE -os NX-OS -root_certificates ./gnmi.pem -ssl_target_override divya -xpath "/interfaces/interface[name='eth1/1']" 172.25.74.84:50051

It specifies the path, type and encoding for which data is requested. As we can see with the xpath definition, we are using OpenConfig as the underlying data model to retrieve the state of an interface (the tool also supports device YANG as the data model which is specific to NX-OS). The type of information being retrieved could be config, state or all in NX-OS. In this example, we specify JSON as the encoding since it’s what is currently supported on NX-OS.

Use the example below to try a gNMI Set operation to update, replace or delete configuration on switches.

divya@Ubuntu-host:~$ cisco-gnmi set 172.25.74.84:50051 -os NX-OS -root_certificates ./gnmi.pem -ssl_target_override divya -update_json_config ./int_trunk.json

We specify our configuration to be applied in a JSON file called int_trunk. The SetRequest operation typically includes a path similar to the above example. It also includes a value which is the data to be applied on the switch.

Similarly, the cisco-gnmi CLI can also be used to do the gNMI SubscribeRequest operation.

Python script to automate configuration using LLDP and gNMI

Now that we’ve established that the cisco-gnmi library is installed and we are able to perform the different gRPC operations on our switch, we are already halfway there! This shows us that our switch configuration, OpenConfig RPM packages, and certificates are all working correctly. It also shows us that we can do a gNMI Capabilities, Get and Set successfully.

With all of this established, how do we write a Python script to do our original task? The script will have to first apply a gNMI Capabilities method to check if the OpenConfig model for LLDP is supported on the switch. Next, we will have to do a gNMI Get to retrieve the state of LLDP neighbors on the switch. With this information, we can extract the interfaces where a Linux host is detected and do a gNMI Set to set our interfaces with “switchport mode trunk”. And that’s it! We now have a working Python 3.0 script which is able to automate our task, with a completely open model using gNMI!

The complete code can be found on GitHub, but I would like to point out a few things here.

The code defines a new class called ConfigFunctions(). Within that class, the following functions are defined: Init, lldp_capability, get_lldp_ifs and set_trunk_host to perform our required operations. We also have a helper function called get_gnmi_json_val to convert our data from protobuf to JSON and decode the base64 string to UTF-8, so we can easily parse through it.

Now with all of this in place, let’s fire this script up!

divya@Ubuntu-host:~$ python3 lldp-gnmi-getpython.py
Overriding SSL option from certificate could increase MITM susceptibility!
openconfig-lldp model supported on device
Setting up Interface: eth1/3
response {
path {
origin: "openconfig"
elem {
name: "openconfig-interfaces:interfaces"
}
}
op: UPDATE
}
timestamp: 1597283880756493972
divya@Ubuntu-host:~$

As you will see in the code, the xpath which is used to specify the OpenConfig model is invoked in the set_trunk_host function.

xpath = "openconfig-lldp:lldp/interfaces/interface[name='"+interface+"']/neighbors/neighbor/state/system-description"

There are a couple of important points to note when using the OpenConfig model:

First, we currently do not have a method to convert an interface from Layer 3 to Layer 2 mode using OpenConfig. The example script assumes that the interface being configured is in the default Layer 2 mode. However, if we’d like to add this configuration through gNMI Set, we can use the device YANG models.

Secondly, the OpenConfig model for LLDP that I used in my example did not include information about the local interface. Due to this, the example script iterates through the interfaces to note the interface in question when the Linux host is detected.

Conclusions

The two toolchains above both provide wrappers to be able to implement Python scripts on NX-OS in an easy and elegant way. With this set of tools and new investments in NX-OS in the area of automation and telemetry, we hope to pave the way for future network needs!

References



Authors

Avatar

Divya Rao

Technical Marketing Engineer

Data Center Networking