Cisco Blogs

Cisco Blog > Architect & DE Discussions

What if you had a trip-recorder for all your traffic at line rate performance?

The case for “In-band OAM for IPv6”:  Operating and validating your network just got easier

How many times have you wanted to gain a full insight into the precise paths packets take within your network whilst troubleshooting a problem or planning a change? Did you ever need to categorically prove that all packets that were meant to traverse a specific service chain or path really made it through the specified service chain or path? “In-band OAM for IPv6 (iOAM6)” is now here to help, adding forwarding path or service path information as well as other information/statistics to all your traffic. It is “always on” OAM – and a new source of data for your SDN analytics and control tools.

Read More »

Fosdem 2015: a status update

As is our tradition by now a team of volunteers helped out with the network setup and operation of Free and Open-source Software Developers’ European Meeting (FOSDEM). The network was very similar to the one used last year and we wanted to report on the evolution of the traffic we measured.

First the bad news: due to the increased use of IPv6 we have less accurate data. This is because while IPv4 uses a unique MAC address which we can use to count the number of clients, IPv6 uses ephemeral addresses, and one physical device can use multiple global IPv6 addresses. In fact we noticed one client using more than 100 global IPv6 addresses over a period of 240 seconds. Why this client is doing this is a mystery.

The unique link local IPv6 addresses were only kept in the neighbour cache of the router for a limited time, so we have no good numbers for the amount of clients. The good news is we can still use traffic counters to compare with the previous year.

Internet traffic evolution

Internet traffic evolution

Compared to 2014 we saw a 20% increase in traffic to more than 2 terabytes of traffic exchanged with the internet.

Fosdem 2015 wireless traffic distribution

Fosdem 2015 wireless traffic distribution

More interestingly the IPv4 traffic on the wireless network decreased by almost 20% with the net result that now the IPv6 traffic is 60% of the traffic on the wireless network, while IPv4 traffic is only 40%. So IPv6 traffic is 1.5 times the IPv4 traffic. This is a good indicator that most clients now can use NAT64 and can live on a IPv6 only network.

Internet IPv4 versus IPv6 for Fosdem 2014-2015

Internet IPv4 versus IPv6 for Fosdem 2014-2015

On the internet side the IPv4 traffic increased by 5% while the IPv6 traffic almost doubled. As we use NAT64 to give access to IPv4 only hosts using IPv6 only on the internal network this measurement is a clear indication that more content is now available via IPv6.

For next year we plan to setup some more tracking systems in advance so we can investigate the number of clients on the wireless network and why some clients are using hundreds of global IPv6 addresses.

Tags: , , , ,

Cisco Live Milano……..IPv6, Spaghetti and Innovation

Last week was “that time of year again”: Cisco Live Europe !! One of my favourite weeks of the working year when (together with 11,000 other Networking Professionals from over 110 countries) I packed my bag and boarded a plane for what I always affectionately call the “Cisco Live Zoo“. 

My major “personal stake” is as the “co Session Group Manager” for the IPv6.  This means that (together with my colleague Eric Vyncke) we are jointly responsible for all “IPv6 Content” which includes Breakout presentations, Technical Seminars, show infrastructure and the World of Solutions exhibition floor. 

This year our breakout sessions have reached over 1600 attendees and the feedback we are seeing looks as if people have had a great time and learned a lot from our IPv6 speakers. Many thanks to all speakers and attendees for a great content track. I personally took a role as a speaker in the IPv6 Techtorial: Advanced Practical Knowledge for Enterprises Deploying IPv6

I was also lucky enough to be invited to participate as a member of a very interesting IPv6 Panel which discussed the question of whether the time is now right to move towards an “IPv6 only / IPv6 centric infrastructure”. Our panel was very ably supported by

  • Alain Fiocco (Cisco)
  • Gert Doering (SpaceNet)
  • Jen Linkova (Google)
  • Patrick Grossetête (Cisco)
  • Tore Anderson (Redpill Linpro)

Some of the more interesting quotes that we heard from this panel included:

Gert: “running a network dual stack causes lots of extra effort”

Jen: “One Network is Better Than Two”

Tore presented his approach to running IPv6 only Data Centres and showed how to enabled this with Cisco IOS XE and Patrick explained how with the use of MAP technology Cisco have enabled a large Electricity Distribution company to deploy “IPv6 only SmartMeters” alongside legacy devices that not only cannot support IPv6 but actually have no support for IP at all.

My colleague Andrew Yourtchenko was once again leading the way in the Cisco Network Operations Centre (NOC) supporting both ‘Dual Stack’ on the show WiFi and featuring an IPv6 only SSID. The results of his work can be clearly seen in the statistics we gathered from the NOC:

The main point we noted from these figures was the rise in IPv6 attached devices to 90% (up from 80% in 2014)

The other major part of the IPv6 / Cisco Live Program that I personally drive is what we call the  “IPv6 Enabled” program in the World of Solutions. This is all about highlighting whether a particular platform is IPv6 capable AND HAS BEEN ENABLED FOR IPv6.

In 2014 I had run the same program:

I sent out advanced warning to exhibitors that I was running this program (for the second year running) and told them that if they enabled IPv6 and could demonstrate that to attendees then they would qualify for an “IPv6 Enabled” logo:

Screen Shot 2013-12-18 at 18.47.38

‘Armed’ with my box of badges and camera I reached the Exhibition Hall around 08:00 on Tuesday morning (around 2 hours before the show opens to the public). I already had a list of some names of Partners and colleagues from Cisco who had told me that they would be IPv6 enabling their demonstrations and my job was to badge them and “find the rest”. I spent the next 8 hours walking from booth to booth with a discussion that went various ways:

“Are you showing a demonstration on your stand and does it visualise an IP component in any way ?”

 “No we are not running any demonstration” or “No our solution runs above the IP layer and no IP addresses are visible in our demonstration”. 


“Yes we do visualise IP in our running demonstration”

“In that case  do you show IPv6 running ?”

“IPv6 is not currently supported on our platform”


“Yes we support IPv6….look here in our data sheet…or look here where the CLI shows you how you can enable it”

None of the above responses qualified for an “IPv6 Enabled” sticker. I responded in each case with a brief explanation about the fact that there are no more IPv4 addresses left and that in 2015 IPv6 really ought to be “Centre Stage” in all such demonstrations with a brief visit to the 6lab stats portal.

These exchanges always ended with my leaving my business card and asking them to please come back to me when they had enabled IPv6 and telling them that I was looking forward to seeing this in San Diego in June. I was actually delighted by the fact that some partners actually contacted me overnight on Tuesday night to let me know that they had actually enabled their demonstrations and could I please return with my badges. I was delighted to agree.

In many other cases I was pleased to find that IPv6 was enabled as an intrinsic part of the demonstration:

Partner IPv6 Enabled Demonstrations

  • British Telecom
  • Infoblox (Gian Carlo Palmieri and Mara Bisti)
  • Paessler (Konstantin Wolff)
  • Arbor Networks (Kiril Kassavchenko)
  • Ixia
  • Mida Solutions (Ronny Tiotto)
  • Netformix (Gidon Leizer and Robert Hall)
  • Packet Design (Angela Reyna and Peter Frame)
  • SevOne (Matt Goldberg)
  • Wild Packets (Linus Brand)
  • PRTG (Konstantin Woff)
  • Isarflow
  • Tiger Communications (Phillip Smith)
  • Men and Mice (Dagmar L. Hilmarsdottir and Martin Metz)
  • Plixer
  • InfoVista

Cisco IPv6 Enabled Demonstrations

  • Autonomic Networks
  • IoE/ IoT
  • Wireless Lan Controller
  • Connected Cities
  • Connected Transportation
  • ISR / TrustSec
  • Cat6K / VSS
  • Segment Routing
  • iOAM6

Photos of many of the “happy exhibitors” are at this link:

I look forward to repeating the award of the “IPv6 Enabled” logos in San Diego at Cisco Live US.

Indeed I would like to challenge every one of my Cisco colleagues and our Partners who will be in the San Diego WoS to reach out to me in advance of the show and tell me that they will be qualified for the program in June and will be enabling. We expect to find many many more IPv6 enabled platforms and demonstrations and look forward to presenting many more badges and meeting old friends and new again in 5 months time.

Tags: ,


In June, 2014, I attended the DevOps Days (un)Conference at the Computer History Museum in Mountain View. I reflected on that in this blog post. At the end of that blog I said:

“At DevOps days, I was kindly introduced, by Paul Peissner of CollabNet, to John Willis and Dave Nielsen, who helped organise the event. John and Dave are keen to do something, similar to DevOps Days, focused on the intersection with the networking world, enabled by emerging network programming concepts.”

On October 14 Paul, John, Dave, I and a whole host of others attended DevOps 4 Networks, which was the event that we wanted to put together back in June. On reflection (as I do that a lot), four months from “we should …” to “we did …” is more than a little impressive. Dave and John pulled it all together, and I am very grateful to the wider DevNet team of Ruth, Ed, Chuq, Mandy and others who helped make it possible. Thanks guys and gals!

As part of the organisation process, we also started this DevOps Tools Survey. Please take a moment to complete that. I have added this link at the end of the blog also.

The event itself was organised as an (un)conference, with an agenda in the morning (details of videos below) and self-organised discussions in the afternoon. The attendance was very good. Over 300 people registered, with over 150 actually attending (which is very typical of such events).  The panorama photo below was taken during a break in proceedings during the morning.


The videos from that day, and others, can be found at YouTube. I have curated a list below in rough temporal order to provide a sense of how the thinking and conversation has evolved over time, and during the day of the event.

John Willis’s thoughts on this space from early 2014, John Willis — The Network – The Next Frontier for Devops?, and mid 2014, Alice In Wonderland – DevOps and OpenStack Networking help illustrate where his thinking started, what he has observed as this space has evolved, and lays some of the conceptual groundwork for what follows.

John’s interviews with Jeremy SchulmanBrent SalisburyLorie McVittie and Mat Peterson, in the lead up to the DevOps4Networks event, provide insights from industry leaders in this space.

The Opening Remarks by John Willis at DevOps4Networks 2014 helped set the context nicely, and explained why we needed a focused event. As John says, if you think this is cool, then please say so.

The DevOps 4 Networks Keynote by JR Rivers (18:11) posed the “vampire tap” challenge. If you don’t know that is, look here, and accept that you have failed JR’s challenge ;-).  Perhaps more importantly, JR brought his decades of experience, gained at Cisco, Google, Cisco, Cumulus, to the conference to help us understand why we were all there. Think Linux, code in Git, fully automated, hosted, evaluation environments, the myth of the universal data model versus working, consumable, technology, with guardrails built with defensive programming in byte-sized chunks!

Test Driven Development for Networking by Colin McNamara (36:23) was arguably the most quoted presentation of the day, especially the maturity levels chart and value stream mapping.  The why aspect of test driven development helped capture the essence of what the day was about. Agile, DevOps, SDN and Cloud all fit together in Colin’s vision. Configuration as code, infrastructure is code; DevOps is the new network operations. Change control boards suck (how can anyone ever know what will really happen?!). Managing a network without tests is like driving a Ferrari without seatbelts, eventually it won’t work out. The exposition of basic CI for networks was a very practical example that everyone can benefit from, including an explanation of what GitGerrit and Jenkins are for. VIRL was also explained, so look for more about how we plan to use that in a future blog from me.

The panel session with Alex Honor, Jeremy Schulman and Nathan Sowatskey, Panel: Do Network DevOps Pros Need to Code? (59:05) addressed one of the key questions facing network engineers and operators. The upshot is that understanding automation and how developers, and developer tool chains, work is key; actually being able to write code yourself, less so. It also reminds me how weird it is to see oneself on a video (useful though).

When DevOps & Networking Intersect by Brent Salisbury (41:23), which starts off with slightly weird audio as Brent had the benefit of two microphones, and someone else in the venue (God?), with a friend waiting in front of the museum (radio mikes!). Having gotten past all that, Brent’s observations on operational evolution and the value of the team make it all worth it. Highlights include application virtualisation, exponential growth, commoditisation, vertical integration, unused hardware capacity and scale, what SDN is for (the edge), contradictions between innovation and stability, L2 suckiness, and cumulative years of experience about what does not scale.

What the Business Thinks about Network Programming by Nathan Sowatskey (6:14), based on my interactions over the years with customers thinking about network programming and automation.

The Making of a “Hybrid” Engineer by Salman Asadullah (6:44) covers the challenges of training network engineers to understand the programming and automation worlds, and application engineers to understand the network.

The Chef Cookbooks we use at Ooyala by Bao Nguyen (6:22) provides a very pragmatic overview of using Chef and Git (another shout out for Git, a theme of the day) for network configuration management (because they were using Chef in IT anyway, so why not? That’s how IT people think, network dudes have to get used to that).

SDN for Hybrid Clouds by Vinothini Raju (5:41), with whom I had great sympathy as she had also just flown in, but from India! The key points focused on dynamic capacity provisioning, replication of active storage, i.e. caches, data distribution, which has the compute service in the cloud, but the data in a private store.

Burning Man – Scaling for an Extremely Temporary Network by Matt Peterson, just to reinforce how sucky L2 is, what a truck-roll is like on a bicycle, how people (L8) really get L1 together in a DevOps way, dehydrated, in the desert whilst possibly not entirely sober.

Infrastructure API Lightning Talk by Jeremy Pollard of (23:11) what if your network was smarter than you? Configuring networks manually is boring and error prone, and you have better things to do with your life. Such as write formulas that generate all that stuff for you.

If you like all of this, and want to see more, please let us know!

As part of the organisation process, we also started this DevOps Tools Survey. Please take a moment to complete that.

Seeds of Change

While change is a hallmark of the IT industry, the actual levers for change are BLOG-OPEN-COMPUTE-PROJECT-LOGO-FEB12have actually remained fairly stable. Vendors were the initial agents of change largely because they were the only ones with the critical mass of smart people, R&D, manufacturing and service delivery to seed and then maintain a fledgeling industry—barriers to entry were a bit higher 30 years ago than they are today because the innovation was happening at the physical layer—we were still fighting over layer 1 and layer 2. The best thing that happened to this industry was the rapid emergence of standards developing organizations (SDOs) as the next arbiter of change. The action moved up the stack and networking exploded because protocols like Ethernet, TCP/IP and BGP were standardized and created a stable, level playing field that benefited everyone alike. Over the last few years, the open source movement has emerged as the latest lever for change in the industry. By democratizing the whole process of innovation, open hardware and software is giving rise to an astounding rate of change.

Now, there is many a VC pitch that’s hinges on painting Cisco as the ossified incumbent (trust me, I have seen a few), but the inconvenient reality is we have been active contributors in the open networking initiatives that have emerged in the last few years including ONF, OpenStack, OpenDaylight, and OPNFV. To that list, I am pleased to announce that we recently joined the Open Compute Project as a Gold member. The motivation behind our membership is similar to our involvement in the aforementioned open networking projects: we see the OCP community as an excellent forum to work  with our customers to co-develop solutions to meet the challenges they face.

As you many know, OCP is structured into a number of projects (networking, server design, storage, etc). While there are a number of areas where we could (and will likely) engage, the first project will be Networking (shocking, I know), where we feel we can make some useful contributions to the existing work underway.

Beyond this, I do not have a whole lot more to share—to borrow a phrase from a friend of mine, the coin of the realm is code and specs and the work is just getting started for us, but expect to see some cool stuff in the near future.

Tags: , ,