Programmatic Interfaces Are Not Taxing

August 28, 2012 - 2 Comments

Network programmability has certainly captured the imagination of the industry.  Efforts by Open Networking Foundation (ONF) and early adopters have helped demonstrate the promise of the technology and captured a lot of attention and it’s been very useful to get attention to a real problem. I simply love it when structured assumptions are challenged and new ideas are found to be useful. But, over the last couple of years, I have seen my share of “magic happens here” presentations and being someone known to enjoy a glass of BGP or MPLS; I’ve become accustomed to the devil hiding behind the details.  Dust off your wand and fire up the espresso maker because it is going to take a concerted effort across the industry, involvement of a number of SDOs, as well as some new thinking to truly deliver on the promise of programmable networks and an open network environment.

The Mission

Looking ahead, one of the most basic objectives is to move beyond the current box-centric/box-specific approach and instead create an environment where applications and services can customize and specify infrastructure requirements (i.e. policy, topology, features) via a set of standardized interfaces and control points.  This approach allows the creation of feedback loops between previously “closed” elements of the middleware infrastructure that allows free-flowing interaction and exchange of information such as state, events, and policy.  While one benefit of this approach is to improve the status quo, the real benefit becomes the ability to create innovative new service delivery models that are not possible today. The goal is to enable control points for services, appliances and networking both physical and virtual. Add to the mix programmable interfaces to collaboration and content technologies and monetization API management and now we are getting somewhere.

Moving beyond the Status Quo

As discussed earlier, visibility between multiple layers of a hierarchical network topology has become table stakes to deliver a network with service assurance.  Virtual networks must also work across multi-vendor and multi-functional equipment while managing ‘state’ across optical, transport, trunk, virtual and service layers. So, as cool and fast as it is for a sysadmin to build virtual overlays; it’s impossible to guarantee service delivery constraints without having netops having full awareness of the network that they are to run over. Enabling these two critical groups of engineers to be able to work seamless together is the target that we are aiming for.

We’ve learned a lot from the many generations of programmable interfaces to networking devices that have been built and we still have a ways to go and a number of basic problems still to be solved.  For example, although we have many protocol interfaces to routers, switches and transport devices, we aren’t adequately getting real-time information out of the devices. In some incarnations of these interfaces, we are again revisiting the costs and benefits of a PVC (Permanent Virtual Circuit) based architecture and just starting to realize that horizontal communication between all the databases (identity, authorization, addressing, discovery, path topology, service policy, service chaining, etc) to generate the customized state that’s expected on the Internet. Having spent a ton of time on working through FCAPS architectures at different networking layers, domains and the event triggers and monitoring required for service delivery; these simply have to be added to the architecture of a computer driven, programmable network.

Service Abstractions are the Key

My perspective is that service abstractions are a vital next step in the evolution of network programmability.  Abstractions will allow for the definition of layered APIs and NPIs (Network Programming Interfaces).  Enabling multi-layered APIs across all of the underlying network elements will be a critical first step to ensure integration with operator development environments.

Integrated network and service control points from device-to-network-to-stack-to-developer ecosystem must be created to accelerate the development of network applications and be supplemented by local and remote APIs and multiple languages and virtualization options.

Evolving the open networking model will mean we must do more than just capture diverse information, we must be able to manage it through a series of full duplex interfaces.  Simple, fast, standardized and hide-the-sausage-making control points to enable orchestration of the network, appliances and services into the multi-stack, multi-framework, multi-platform environment developers are used to working in.

It should be obvious that not all programmatic interfaces are the same.  Different network planes require different programmatic interfaces that are appropriate to and based on the needs of each layer (referring to the OSI network layers).

Additionally when programming state, policy, features into a network I need to reiterate the criticality of topology awareness. When coding to the network you what you often want to program is dependent on the function, position and role of the device.  What I’m trying to say is that not only are not all programmable interfaces the same but, not all network functions are the same. A topologic view allows you to program one node, a path, a set (e.g. area, level), a subscribers “place” (I often refer to this as telemetry), or attributes/weather condition of the network.  Sorry, I just have a huge block on PVC (programming link by link, hop by hop routing) based networks that have no control plane or horizontal communication between control nodes and devices. The Internet has been designed for maximum reliability under catastrophic events and this is based on distributed, dynamic routing based on independent response to changes in the network or network conditions. Seconds of response time from a single controller of a network to reroute or protect a service just isn’t good enough when we are already tuning to low 10s of milliseconds and being driven to optimize further on the existing internet.

This next picture is a very simplified view of two common architectures for programmable interfaces. On the left, you have a logically centralized controller that programs state (e.g. route entries, classifiers, tunnels) into a router. On the right is the architecture I am proposing which is to be able to extract and program state from a WAN router that’s integrated with the router’s OS and databases.

The architecture on the left makes sense in a constrained topology with few exit points and peers. All loop detection, route calculation, error checking and event handling is done in the logically centralized controller.

If the network is very large with an unconstrained topology and many, diverse peering points (e.g. Telco, MSO internet backbone); the existing route selection algorithms that are run in each router throughout the network enables the fastest possible reaction time to network changes and shortest path routing to a peer. What’s also shown is that augmenting this system with programmable interfaces and an open networking environment is what gives operators and developers the best of both worlds.  Add in a full view of the hierarchical network to get a real-time weather report, and now to have the maximum flexibility to operate a network, raise efficiency of path utilization and have the most resilient network possible.

Key to understand my thinking is that there is a massive Internet that’s deployed and what needs to be enabled and augment are programmable interfaces to the critical features that currently have no other control point than a command-line-interface with transaction and persistency semantics.   What I am trying to say is that overcoming the existing user interface (the CLI) and  the current database decision to write the desired state of a network into tens of thousands of configuration files; needs to change. What’s key to make this successful is to hide the complexity of all this under some very simple abstractions. Clearly, no matter how cool I think BGP (Border Gateway Protocol) policy syntax and semantics are; I realize that not everyone that wants to build a live teleconferencing session from the Bay Area to Boston will want to know every detail of how it’s done. That programmer/service operator is more than happy to have the WAN orchestrated for them by linking sysadmins and netops. I want to enable that orchestration on the existing internet.

IRS and Beyond

Cisco and a lot of friends have committed to build a model that not only incorporates existing mechanisms — where appropriate —but also builds extensions and new protocols — wherever necessary— to create consistent abstracted interfaces that can offer a multitude of tangible benefits to applications developers and the service providers that rely upon their work. The difference in the architecture of this work is:

–          Focus on extracting the state of the device, services and hierarchical topology in real time as a first order priority vs no consideration whatsoever

–          Have duplex communication of “events” associated with devices, adjacencies, paths, etc available to other applications

–          Create programmable interfaces that work directly with the existing control plane to take advantage of the discovery, signaling and distributed communication that exists

–          Reuse the OAM mechanism that are widely deployed and continue to extend the protocols that can pass critical attributes of an adjacency, path and topology (e.g. delay, utilization, jitter, performance  — loss)

–          Work to create an open network environment that also contains the features that are not standardized, under-standardized or only available via proprietary CLIs.

The good news is that there are a number of efforts spinning up across a number of Working Groups to advance the scope and richness of programmable interfaces to the routing system.  For example, working with long time friends at Juniper and across the industry from the operator and developer communities and system integrators; we recently introduced proposal to IETF for an Interface to Routing System (IRS). This framework enables full duplex, streaming programmatic interfaces to the Internet’s routing and signaling infrastructure.  Beyond being able to pull only feature configuration or statistics from a router, the IRS also allows interaction in the other direction. For example an application could install a specific route in the Routing Information Base (the database on a router that contains all the destinations reachable on the internet or in a VPN). “Gee Dave, that’s exciting; you’ve reinvented static routes again” are the way a few friends like to rib me (bad joke intended). Although this is a basic example (IRS specifies many more things), today an operator is basically only able to do that through the CLI.  With IRS, this route instead of being programmed directly into the Forwarding Information Base (aka Table) like other models; IRS puts the route directly into the operating system of the router so that it can be retransmitted via BGP. Maybe into a VPN with a little policy sprinkled on top, add a community, maybe an encapsulation for good measure.

The point is that the WAN Internet needs to be augmented. The augmentation of Internet needs a system approach to the problem not just a single, point solution or limiting architecture.  Programmable interfaces to access the state of the hierarchical network and features such as routes, encapsulations, cross-connects, paths, VPNs, NetFlow, QoS, Policy, Multicast trees, OAM are critical and just the ticket.

Beyond IRS and considered under the umbrella within IETF, there are a number of other efforts under way:

The Opportunity

There is a strong need to develop control points and integrate them into stacks and development frameworks that developers work within. Programmatic interfaces must be the bridge between operations, applications, real-time applications, routing, Internet, security and transport.

Overlay technologies are getting a lot of attention, but they will solve only part of the problems the industry encounters.   To get tighter SLAs, better return on capital investment and devise an underlying network environment that does everything faster,  we must create programmatic interfaces that are integrated into the existing Internet and understand the approach must be more than the sum of the parts but, as a functioning system.

To give applications access to and control over information that comes from the underlying network, we need a publicly documented set of interface specifications, APIs and data models.  The interfaces must support real-time, transaction-based interactions using efficient encodings.  They must also support a growing set of use cases and these use cases must include linkages between the WAN and the cloud.

As networking evolves, so must our SDOs. It comes down to developing new approaches to how we architect our service delivery models and solutions.  At the same time, the industry isn’t going to wait for SDOs to modify or grow their processes or be limited to their publication schedules. We’ve all learned a lot from the early stages of SDN concepts and open networking and thankfully some gedenken-silos are being knocked down along the way. Hopefully, the ideas about augmenting the Internet with programmable will continue beyond IRS as there is an incredibly rich mine of information available on transport equipment, routes, switches and service devices whether physical or virtual.

In case you missed it before … what problem are we trying to solve with this work? Make provisioning of more and new services faster, reduce operational windows, make it easier to debug and troubleshoot, give the netops folks the same functionality as the sysadmins and enable a real-time “weather report” of the hierarchical network. With these tools, developers can continue to tune in and turn on the Internet.

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.


  1. Good Article. I do liked the concept discussed over here regarding time and space sharing.Its always a time for reinventing in this field and as I can see it`s going on all the time. The SDN Model has been discussed in a great manner.

  2. Well, I will take a stab at a reply, I’m a good pinnate. Thanks for sharing some thoughts. Recognizing the time and space constraints, there was a nice white paper recently at sigcomm: “The Controller Placement Problem”. inter-AS, CPE awareness is clearly a big focus for Cisco. I think your Draft and the Huawei Alto drafts are a some good starts for the service provider space.

    That said, much of the use SDN use cases for SPs are reticent in the enterprise. In my neck of the woods we have hospitals, campuses and a regional network, the problem is still the same, management of the distributed firmwares we operate today, the ability to implement rudimentary policy (bump in the wire) at scale, all while avoiding proprietary solutions. Effecting delivery paths of dropping a letter in the mailbox would be great, but that is icing compared to inability to execute policy at scale today.

    I love MPLS as much as the next guy, it is our primitive form of virtualization today, if it wasn’t for it I would probably have been a full time programmer at this point. Making a pile of gear sing is an art in a multivendor env using proprietary HW, SW and APIs. BGP is here to stay (due to its ability to evolve) in my humble opinion but it may be time to look at different solutions to solve different segments of the network. I hope to hear a Enterprise and Data Center strategy articulated next. They seem to very different solutions with regard to levels of decentralized CP as your enterprise wireless products exemplify.

    Thanks for the open dialogue, it is important to those of us not doing 50mil+ annually w/o a seat at the table but still caring about the future of the industry.