Continuing the analysis of the data collected during Fosdem 2016.

This year, we replaced the router with a more powerful model: an ASR 1006 with RP2 and SIP20 modules. This allowed us to enable the NBAR 2 feature to analyse the traffic crossing the router.

NBAR 2 is a traffic analysis engine which allows the router to classify traffic into applications. It does this on the basis of ports, but also based on the content of the traffic itself. It recognizes, for example, the difference between general web traffic and traffic to youtube.

This feature is designed to be used as a classifier for use with QoS, so that one can give higher priority to important web traffic and less to some other traffic. Now that HTTP and HTTPS are the new “TCP sockets”, meaning that more and more applications use HTTP or HTTPS to interact with the world, we need deeper insight into the traffic and just using port numbers isn’t enough anymore to discriminate between different kinds of requests.

For Fosdem, we used this to learn what kind of traffic is being generated on the network and to extract interesting information from traffic flows (but more on that in a later article).

As part of the ‘discovering the usage of your networks’, enabling NBAR2 is a simple step beyond using access-lists and interfaces counters which we used in the previous article.

Over time protocols change, so one can download updated ‘protocol packs’ which include the description of newer protocols and applications.  Simply upload them to the router and do:

asr1k#conf t
asr1k(config)#ip nbar protocol-pack bootflash:/pp-adv-asr1k-155-3.S1-23-15.0.0.pack

This will teach NBAR2 about the new protocols and applications. Then to enable the discovery on the desired interface, configure:

interface Gig2/2/1
  ip nbar protocol-discovery

From this moment on, the NBAR2 engine will start collecting information. Allow some traffic to flow across the interface, and then see what NBAR2 knows about it:

asr1k#sh ip nbar protocol-discovery

 Last clearing of "show ip nbar protocol-discovery" counters 5d20h

                            Input                    Output
                            -----                    ------
   Protocol                 Packet Count             Packet Count
                            Byte Count               Byte Count
                            5min Bit Rate (bps)      5min Bit Rate (bps)
                            5min Max Bit Rate (bps)  5min Max Bit Rate (bps)
   ------------------------ ------------------------ ------------------------
   binary-over-http         50067876                 109178677
                            5035136708               159890721603
                            0                        0
                            4762000                  116145000


This shows the number of packets, bytes and traffic rates for a large number of applications. Here, “Input” traffic is traffic arriving on the interface, so in this case from the wireless clients towards the internet or the other networks on the router. Similarly “Output” is traffic send out on the interface towards the clients.

The CLI allows you to interrogate this information in a liberal manner:

asr1k#show ip nbar protocol-discovery interface <interface> stats max-bit-rate top-n 3


 Last clearing of "show ip nbar protocol-discovery" counters 00:10:53

                            Input                    Output
                            -----                    ------
   Protocol                 5min Max Bit Rate (bps)  5min Max Bit Rate (bps)
   ------------------------ ------------------------ ------------------------
   ssh                      743408000                505670000
   rtmp                     93801000                 1499000
   binary-over-http         527000                   45774000
   Total                    845853000                576473000


Here we asked for the top 3 application, sorted by maximum bit rate on a specific interface.

I collected this output for the different interfaces and, after some transformations, one can plot, for example, the total number of bytes send or received on the Internet interface for the top 20 applications:


As it is clear video-over-http, ssh and ssl dominate.

When we look at the maximum bitrate for the internet side we see that the highest rate is for ssh:

Max Bit rate top 20 for Internet

In the above graph ‘in’ means traffic arriving to the router from the internet.

This high rate for ssh (almost 1Gpbs) is due to the Video team who send quite a bit of rtmp traffic, but even more ssh traffic, as one can see from the statistics for the Video network interface traffic:


For the outgoing direction (from the point of view of the interface, as always) we see that the Video network itself received mostly ssh traffic:


Looking at the wireless networks we can see a clear difference between the traffic requested by the IPv6 only clients:


and the legacy dual stack clients:


It seems IPv6 clients watch video streams a lot more.

One can also see the traffic the wireless clients generated towards the internet:



One can conclude that bittorrent traffic seems to prefer legacy networks as bittorrent traffic disappears on the IPv6 only network.

The NBAR2 data also answers the question: which network is the most looking for love:

asr1k#show ip nbar protocol-discovery interface <Wifi> stats byte-count protocol okcupid
                            Input                    Output
                            -----                    ------
   Protocol                 Byte Count               Byte Count
   ------------------------ ------------------------ ------------------------
   okcupid                  7395809                  26178355
   Total                    217481608124             1893527651702
asr1k#show ip nbar protocol-discovery interface <legacy> stats byte-count protocol okcupid
   okcupid                  4167154                  54499457
   Total                    147878754133             950455266790


So about 32 Mbyte of OkCupid traffic on IPv6 but almost 56 Mbyte on the dualstack network. Clearly people looking for love are prepared to accept working with legacy protocols.

As one can see NBAR2 offers an easy and simple way to discover what users are doing on your network, both on the command line or when you slice’n dice the extracted information.