The author of this blog is Khaled Abuelenain. Khaled is the Consulting Director at Acuative, and a double CCIE (RS & SP). He is an expert on Service Provider networking, SD-WAN, Data Center technologies, Python, Linux, Network Programmability, Automation and Cloud Architectures.
Are you a network engineer and have had to repeat the same boring task at work, every day? Do you feel that there must be a way for you to do a task once, and then “automate” it? Theoretically, an infinite number of times? Or, have you been spending more time cleaning up and correcting configuration mistakes than you spend implementing those configurations? Or maybe you have been hearing a lot about this hot new “thing” called network programmability, but in the middle of the hype, could not figure out what exactly it is?
If any of those cases (and many others) apply to you, then you are in the right place. The fact that you are here, reading this now, means you know that there is probably a solution to your problem(s) in the realm of automation and/or programmability. In this case, buckle up because you are in for a ride!
If you are a network engineer and browsed to this page by mistake, I still urge you to read on. Netflix, Youtube, Facebook and Twitter will still be there when you are done. (Or not.) This is more fun. Trust me!
A Few Definitions For The Road
Before we dive into the nuances of network programmability and automation, let’s clear up some confusion. I hate nothing in the world more than definitions – well, maybe greasy pizza – but this is a necessary evil! In order to start clean, you must understand each of the following: network management, automation, orchestration, modeling, programmability and APIs.
Network Management is an umbrella term that covers the processes, tools, technologies, and job roles, among other things, required to manage a network and the lifecycle of the services offered by that network.
Many standards and frameworks exist today to define the different components of network management. One of them is FCAPS, where the acronym stands for Fault, Configuration, Accounting, Performance and Security Management. FCAPS is geared towards managing the systems that constitute the network.
Another is ITIL. The acronym stands for Information Technology Infrastructure Library and covers an extensive number of practices for IT Services Management (ITSM), which is basically the lifecycle of the services provided by the network. ITIL is divided into 5 major practices: Service Strategy, Service Design, Service Transition, Service Operation, and Continual Service Improvement. Each practice is divided into smaller sub-practices. For example, Service Design includes Capacity Management, Availability Management and Service Catalogue Management while Service Operation includes Incident Management, Problem Management and Request Fulfilment. Some people make a career being ITIL practitioners.
The Merriam-Webster dictionary defines Automation as “the technique of making an apparatus, a process, or a system operate automatically”. In other words, having a system of some sort do the work for you, work that you would otherwise do manually. However, you will have to tell this system what is it that you want to get done, and sometimes, how to do it.
So, configuring a network of routers with dynamic routing protocols, so that these routers speak with each other and figure out the shortest path per destination, is a form of automation. The alternative would be having someone do the calculations manually on a piece of paper, and then configuring static routes on each router. And so is writing a program that configures a VLAN on your switch – or your 500 switches – without someone having to log in to each switch individually and configuring the VLAN via the CLI.
As you have already guessed, the power of automation is not intrinsically in the automation itself. Logging into one switch manually and configuring one VLAN is probably much faster than writing a Python program to do that for you. So why automate? Obviously, the importance of automation is its application to repetitive tasks.
Automation will not only save the time you will spend repeating a task, it also maintains consistency and accuracy of performing that task, over all its iterations. It does not matter if you have 10 or 500 switches. The program you wrote will always go through the exact same steps for eachevery switch, with the exact same result. Every time. Of course, the assumption here is that no errors will happen because of factors external to your program, such as an unreachable switch, wrong credentials configured on a switch, or a switch with a corrupted IOS. Although, you can write a program to detect and mitigate these error conditions!
When you have several systems working together to get a job done, there is typically a need for a system, or a function, to coordinate the execution of the tasks performed by the different systems towards getting this job completed. This coordination function is called Orchestration.
For example, a private or public cloud that provides virtual machines to its users will include different systems to provision the network, compute, virtualization, operating systems, and maybe the applications, for those VMs. Orchestration will provide the function of coordination between all the different systems and applications to get the VM up and running.
Automation and orchestration work well in tandem. Automation covers single tasks. Using software to configure a VLAN on a switch is automation, and so is provisioning a VM over ESXi, or installing Linux on that VM. Orchestration, on the other hand, is the function of coordinating the execution of these automated tasks, in a specific sequence, each task using its own software and each on its respective system. The scope of automation involves single tasks. The scope of orchestration involves a workflow of tasks.
The concept of Modeling Data is not a new concept and is not exclusive to networks or even automation. Data modeling is very involved and is a major branch of data science. For the humble purpose of this blog, let’s use an example to demonstrate what a model is. In Example 1, you can see a configuration snippet of BGP on an IOS-XR router in the left column. In the right column, the specific values for this particular device were removed and replaced with a description of what should be there. A template of sorts.
Example 1 – BGP configuration snippet on IOS-XR and
the corresponding data model in tree notation
As you can see, a model is a little more than just a template.
A model describes data types. An IP address is composed of four octets separated by the “.” character and each octet has a value between 0 and 255. An ASN is an integer between 1 and 65535. These two objects, an IP address and ASN, are leaf objects, each having a specific type and each instance of that leaf has a value.
As you have already guessed, a model also describes the data hierarchy. Address families are child objects to the main BGP process. Then the networks injected for a particular address family are children objects to that address family. And the same applies to neighbors: there are global neighbors that are children to the BGP process, and then there are neighbors defined under the different VRFs. You get the point.
So, in order to reflect hierarchy, other object types may exist in a model besides a leaf, such as leaf lists, lists and containers. A leaf-list is a list of leafs. For example, a snmp server configured on router is a leaf object. A list of snmp servers make up a leaf list. All leafs under a leaf list are of the same type.
A list is a group of other objects and has many instances. For example, the VRF in Example 1 is a list. It has children objects of different types (some of which are themselves lists), and at the same time you may have more than one VRF configured under the BGP process. A container is a group of objects of different types, but a container will only have a single instance. An example of a container is the BGP process itself. This is an over simplification of what a data model is just to elaborate on the concept.
Data models used in the arena of network programmability are described using a language called YANG. A “YANG model” is nothing more than a data model described using YANG.
Defining Programmability is not as easy as the previous terms. The reason for this is that the term is used across a very wide spectrum, and means different things depending on what context it is used in. Programming a device or a system basically means giving it instructions to do what you want it to do. A programmable device is a device that can execute different tasks based on the instructions it is given. In the world of electronics, an ASIC (Application Specific Integrated Circuit) is a chip that does one specific function. If this ASIC is built to accept two numbers as input and add those two numbers, it will always do that. A Microprocessor, on the other hand, accepts instructions describing what you want it to do with the input it is given. You can program it to add two numbers, multiply them, or subtract one from the other. A microprocessor is a programmable device while an ASIC is not.
But doesn’t this equate programming to configuration? Configuring a network device is basically telling that device what to do … right? Well, that is tricky question, and it is here that we discuss programmability as used today in the context of network automation.
Programmability for the purpose of network automation is basically the capability to retrieve data, whether configuration or operational data, from a system, or push configuration to a system, using an Application Programming Interface, or API for short. An API is an interface to a system that is designed for software interaction with this system. Contrast this to a router CLI. A CLI requires human interaction to be useful. An API on that same router would be designed so that a Python program, for example, can interact with the router without any human intervention.
But what really is an API?
An API is software running on a system. This software provides a particular function to other software, while not exposing this other software to how this function is performed. An API will typically have a predefined way to reach it, for example, over a specific TCP port. The API will also define the format of the data that it accepts, as well as the data it sends back. It may also define different message types and specific syntax and semantics to avail the services provided by the API. A device that implements an API is said to expose an API to be consumed by other software.
Now to connect the dots. Orchestration coordinates a number of automated tasks in a workflow to implement one of the disciplines of network management. Automation may leverage an API exposed by a device in order to manage this device programmatically. When a data model is used as a reference during programmatic access, the device is said to leverage model-driven programmability.
Quite a mouthful!
Network Programmability: The Details
Network Programmability as a practice is best summarized by the Programmability Stack in Figure 1 below. The stack defines six layers:
- Application
- Model
- Protocol
- Encoding
- Transport
- Infrastructure
Figure 1 – The Network Programmability Stack
(Disclaimer: The Programmability Stack has not been standardized by any industry-recognized entity such as the OSI 7-Layer Model. Therefore, you will probably run into a number of variations of this stack as you progress in your studies of programmability. I found that the one I have drawn here is the best version for the sake of getting the point across. Feel free to contrast it to other version you find elsewhere and tell me what you think in the comments below.)
At the top of the stack is an application that may be a simple python script or a sophisticated network management system such as Cisco Prime. At the bottom of the stack is the device exposing an API. In order for the application to programmatically speak with the device’s API, it will leverage a choice of components at the different layers of the stack.
The application will choose a model. Different types of models exist, the majority today described in YANG. A model may be vendor-specific or standards-based. In either case, the model will define the structure of the data that the application sends to or receives from the device (through the API).
The application will have to choose a protocol that defines the message types as well as the syntax and semantics used in those messages. There are three primary protocols used today for network programmability: NETCONF, RESTCONF and gRPC.
NETCONF, for example, defines three message types: hello, rpc and rpc-reply. It also defines specific operations that may be used in the rpc message to perform tasks such retrieving operational data from a device or pushing configuration to a device. The messages will use specific, well-defined syntax.
The protocols themselves are sometimes described as the APIs. Don’t get confused just yet ! When a device exposes a NETCONF API, then the application will have no choice but to use the NETCONF protocol to speak with the device. The same applies to the other protocols.
A protocol will have a choice of a data format, typically referred to as the data encoding. The most common data encodings in use today are XML, JSON, YAML and GPB. For example, NETCONF will send and receive data only in the form of XML documents. RESTCONF supports both, XML and JSON.
Then this data will be transported back and forth between the application and the device that is exposing the API using a transport protocol. For example, NETCONF uses SSH while RESTCONF uses HTTP.
This is network programmability in a nutshell!
In the next few blog posts in this series we will take a deep dive into each layer and discuss the different programmability protocols, data encodings, HTTP as a transport-layer protocol along with REST and advanced API topics, YANG models, Linux for Network Programmability and Python.
Got comments or questions? Please leave me a message in the “Comments” section below.
Related Resources
- Visit the DevNet Networking Dev Center to learn more network programming and Cisco networking platforms.
- Learn about Cisco’s professional DevNet Certifications
Khaled Abuelenain has spent the last 18 years either employed by or consulting for service providers, operators, banks and fortune 500 companies. Khaled is contributing author of the best-selling Cisco Press book “Routing TCP/IP Volume II, Second Edition” by Jeff Doyle: https://www.ciscopress.com/store/routing-tcp-ip-volume-ii-ccie-professional-development-9781587054709. He is also the lead author of “Network Programmability and Automation Fundamentals, First Edition” from Cisco Press: https://www.ciscopress.com/store/network-programmability-and-automation-fundamentals-9780135183656. Khaled is an expert on Service Provider networking, SD-WAN, Data Center technologies, Python, Linux, Network Programmability, Automation and Cloud Architectures. He is also an ITIL practitioner and an IEEE member since 1997. Khaled can be reach on Twitter and LinkedIn.
We’d love to hear what you think. Ask a question or leave a comment below.
And stay connected with Cisco DevNet on social!
Twitter @CiscoDevNet | Facebook | LinkedIn
Visit the new Developer Video Channel
What a great blog! Thank you for taking the time to write it.
Thank you very much Mike ! Let me know if you have any specific areas you would like me to cover in Parts II and III and I would be happy to add them in.
Thanks Mr.Khalid
What a challenging ease way..!!
Best Regards
Awesome job Khaleb, explaining what network programmability and APi are in such an easy way to understand. Thanks!
You are welcome Jose, I’m very happy you found the post easy to follow and helpful ! Look out for Part II the coming couple of days 🙂
Excellent! I think this is the best explanation or rather formulated/described information that I’ve read on the matter. Good flow, great coherence. Look forward to part two and three.
Thank you very much Mays and I’m glad you enjoyed the post. Part II should be published in the next couple of days. Looking forward to your feedback there as well !
Wonderful! Awesome! Fantastic!
Thank you very very much .. much appreciated 🙂
Wonderful! Awesome! Fantastic!
Thank you very very much Trevor .. much appreciated 🙂
Thank you
Nice and easy explanation
Thank you very much Alaa – I’m glad you found it nice and easy to follow.
Check out Part II here https://blogs.cisco.com/developer/intro-to-programmability-2 and let me know what you think.
I hope “Part 1” is an indicator that there is a Part 2, Part 3, … Part ‘N’ planned for the future. I would certainly welcome more of these, and I’m sure the DevNet community would as well.
Ask and you shall recieve Trevor!
Oh yes, as Stuart said, ask and you shall receive 🙂
Thanks for the clarity . This now gives me the confidence to get started in the new world of automation, programmability and orchestration.
The stack puts it all together . Thanks again
You are very welcome nandit. It is my great pleasure to see that this post is encouraging people to embrace emerging technologies in network management and I’m very happy to be part of their development.
Make sure to check out Part II here: https://blogs.cisco.com/developer/intro-to-programmability-1 and let me know what you think !
And please do not hesitate to post any questions here, to my accounts on Twitter/LinkedIn or my email !
It seems I put the wrong link to Part II. The correct one is: https://blogs.cisco.com/developer/intro-to-programmability-2
Your explanation of an API was brief, but it was just what the doctor ordered!
Thank you very much Trevor for this compliment and thank you following the series and taking the time to write your feedback and questions. I will be looking out for your comments on all the upcoming parts !
Great summary Khalid and overview!! Well done.
Thank you Abdullah !
Please check out Part II of this series and tell me what you think: https://blogs.cisco.com/developer/intro-to-programmability-2.
Part III should be released soon as well (probably tomorrow or the day after).
What a great blog to get started with Network Automation!!
Thanks !!!
Thank you very much Ajay !
You may want to check out the following two parts here:
Part 2: https://blogs.cisco.com/developer/intro-to-programmability-2
Part 3: https://blogs.cisco.com/developer/intro-to-programmability-3
Nice write up! This is a very concise and excellent introduction. I do have a comment/question about the programmability stack diagram. The stack seems to imply that applications like Ansible will make use of a data model but there are many cases where programmability applications interact with the CLI directly over SSH or in the case of NXOS, over NXAPI (CLI exposed over REST). From an introductory perspective it might be worthwhile to update the stack to indicate this.