Algo Boost Series: Part 2 Customer Perspective on Cisco Nexus 3548
On October 5th I posted part 1 of the Algo Boost series with a fantastic discussion around the latency innovations on the Nexus 3548. Today, we announced that these units are now shipping to customers and the much anticipated wait is over to get this game changing technology! This is perfect timing as I introduce part 2 of the series with Errol Roberts, Distinguished System Engineer for the top Data Center accounts, to bring a customer perspective to the ultra-low latency Nexus 3548 in a High Performance Trading fabric.
[GD] I know that you spend a lot of your time talking with customers. What are our Financial Services customers telling you about their environments and requirements?
[ER] When meeting with these customers, I like to ask a single question – “What value can an infrastructure company provide to high-performance trading workloads”. Key points relating to the switching are captured by the following:
- First, customers ask for a network solution and architecture that provides them with the fastest end-to-end functionality. Providing the “Lowest latency possible” is one vector, another vector being a rich “feature-set” answering the different architecture and network requirements end-to-end. Naturally there is a need for speed while at the same time providing the features within the same device. For example in collocated High Frequency Trading environments; the lowest latency being key; it’s not the only factor; support for the routing protocol such as BGP, multicast with PIM Sparse mode, ultra low latency SPAN at linerate with multiple ports; this is achieved with the technology called Warp SPAN.
- Next, “Handle microbursts”. Volatility is correlated. When you are running cross-asset class, cross-liquidity venue strategies, there is often short-lived congestion that increases latency. These volatile periods are often the most opportunity rich.
- Also, “Unique features”. They want features like Network Address Translation to meet their business needs. You don’t want these features to add latency. In fact you don’t want to have any of the L4 or services applied on the network to add latency.
- Next, “Flexibility and Programmability”. They want to control their traffic flow, mirror relevant traffic, have fine-grained flexibility and also have reactivity on events. Python scripting language is a good example of automation. With Python script, you can have the switch react on different environmental changes such as a sanity check when the device comes online as well as for example triggering emails when the burst happening at the buffer level exceed for example 10 nanoseconds.
- In addition, facilitate “Precision Time”. You cannot control what you cannot measure. Without precision time, you invest in an infrastructure and just hope you get optimal performance. With precision time protocol you can keep all of your servers and network elements highly synchronized at the nanosecond level. You can even measure the accuracy of the tool through a 1 pulse per second output port. Also, the Nexus 3548 can timestamp traffic with IEEE 1588, which allows analyzers to replay events.
[GD] Can you elaborate a little more on microbursts? What is unique in how the Nexus 3548 handles them?
[ER]: Microbursts are a problem for firms when there is a spike in data network traffic that overwhelms the buffer capacity of the network. In trading environments, these spikes typically occur during times of high volatility such as market open or market close, or when a specific news event occurs in the world. When trading systems are overwhelmed they typically drop or delay the flow of packets going through them. Since today’s trades are measured in nanoseconds, dropped or delayed packets have the potential to cost organizations a lot of money.
The industry leading Cisco Nexus 3548 has a large 18MB buffer along with a toolset to provide real-time performance visibility monitoring to identify bursts at the nanosecond level; which could potentially compromise system performance.
[ER]: This is a great question. Let’s take the example of a trading firm. The trading firm is connecting to multiple exchanges and receives exchange feeds for the trading premises they provide services to the traders and brokerage clients. In this type of design, as depicted on the diagram bellow; the exchange feed provides the market data information to the trading firm. The network infrastructure and feed handlers can be collocated in the exchange location itself, in order to receive the information as quickly as possible, this is the reason why ultra low latency is key. At the same time in this example, you can see that we have a single tier network collapsed design, where the Nexus 3548 is acting as an L3 device to peer with the exchange for the market data feed while also supporting L2 for all the local feed handlers connecting to it. With the Nexus 3548 enabled in warp span mode, the traffic from the market data feed is replicated and sent downstream to the feed handlers at 50 nanoseconds (ns). At the same time the order execution devices will send an order upstream to the exchange leveraging the warp mode, which provides latency as low as 190 ns. In this type of deployment, you can see that a customer can benefit from a 50ns latency down to the feed handlers and 190 ns upstream for the order execution. It’s important to understand that the same device can have the warp mode enabled as well as the warp span mode, reducing one more time the amount of network infrastructure required to achieve an ultra low latency end to end. In the colocation example, monitoring the traffic going through the trading firm provides a guarantee of delivery as well as a possibility to replay market data traffic. This is achieved with the SPAN functionality of the Nexus 3548. The switch is capable to provide:
- Linerate SPAN
- Realtime SPAN, the replication is done at the sametime as real traffic is being sent out of the device, one a single or multiple ports and even port-channel.
- Advanced SPAN feature set: ie. Truncated, sampled, PTP timestamped or filtered with up to 8 bi-directional sessions.
One interesting customer use case is to use the Nexus 3548 for a remote span destination with multiple traffic analyzers connected on multiple ports, as illustrated on the picture bellow.
I’d like to thank Errol for his customer insights. To get more info on the Nexus 3548, visit http://www.cisco.com/go/nexus3548