Rakuten Mobile & Cisco CX: Disrupting the Industry, Together

As our COO Maria Martinez has said, “customers buy outcomes, not products”. And it is incredibly exciting when that outcome is complete industry transformation.

Over the years, Cisco has been one of the proven enablers of industry transformation. The Telco & Media industry is one example of an industry that’s been ripe for disruption, thanks to operators such as Rakuten Mobile.

Together, Rakuten Mobile and Cisco have had a three-year, fast-paced journey. First, we built the world’s first cloud native mobile IPv6-only network—automated from top to bottom and powered by programmable software stack. Recently, we took a huge step further by introducing  segment routing. This uniquely positions Rakuten Mobile to build a better, more inclusive internet for the future.  And we’re not done yet.

The work is challenging for sure—but worth the effort. Rakuten CTO Tareq Amin said it best:

“Our challenges were numerous. And to be very frank with you, this journey hasn’t been easy. But I think it’s the right journey. And if you ask me, again, would you take the path that we have taken to achieve the results and be where we are today? I would say absolutely. It’s the right thing to do.”

The work with Rakuten Mobile is an excellent example of how Cisco Customer Experience (CX) teams work hand-in-hand with customers, Cisco’s technology groups and other vendors to completely change the game. When I look back on the body of work so far with Rakuten Mobile, three tenets stand out as key to our success:

1) Agree to do things that have never been done before

Nelson Mandela said, “it always seems impossible until it’s done.” Creating the world’s first end-to-end fully virtualized, ready for 5G, cloud-native mobile network with IPv6-only did seem impossible at first, given the stringent timeline and potential gaps. I remember one meeting where plenty of other vendors pointed that out. And key decision-makers had some healthy skepticism about whether we could pull it off. It was a hard decision to make—we had not done this anywhere for a mobile network before.

We carefully specified few key architectural strategies:

• Keep network state lean
• Enforce simpler design
• Allow for efficient incremental deployment of new features and code updates
• Avoid complexity associated with multiple forwarding plane technology, traffic engineering and complex convergence patterns under network failures

To achieve these, we had to:

– Go against the status quo.  This meant not leveraging MPLS, the dominant forwarding plane technology for more than a decade, and preferring simpler IPv6 technology.

– Keep the number of network prefixes small (on the order of thousands) by specifying strict summarization boundaries and encoding the service and subscriber awareness in the packet header itself (first 64-bits of the address itself).

– Leverage a well-understood combination of an IGP and a BGP with route reflection to formulate simpler routing topologies that NetOps teams can understand (& diagnose, if/as needed) very quickly.

– Infuse software-based controllers & orchestrators to take advantage of programmable infrastructure becoming cloudified.

After a bit of deliberation, we all aligned to the disruptive notion of “if it only works with IPv4, then it does not actually work”. While this was a very bold and radical stance, it was in line with Rakuten Mobile’s vision of doing things differently. This would serve as the all-important fuel to make the impossible possible.

2) Embrace radical transparency and data-driven recommendations

A lot would need to change for us to do this “impossible thing”.  To make those changes we’d need to be not just transparent but radically so. A lot more than is usually expected. And not just from Cisco but from all the vendors. There could be no secrets. Even within Cisco, we knew that some of our solutions (e.g. cloud platform) would need to quickly evolve to meet this goal. We needed to work as a team to identify and share our collective gaps so we could quickly fill those gaps.

Championing this radical transparency is what led Cisco CX to take on the program management office. At Rakuten Mobile’s request, we laid out a plan from both a program management and architecture management point of view and established the ground rules for how all the different vendors would work as a team to make it happen. For example, Cisco CX teams collaborated closely with each application workload vendor (e.g. Radio, EMS, IMS etc.) to ensure the readiness and effective adoption.

It was critically important for us to collectively formulate the recommendations based on due diligence rooted in data-driven pros & cons analysis. We went above & beyond in ensuring that our expert teams would show up at the meetings (even if not attended by executives) to not only drive recommendations centric discussion for the closure, but also fine-tune based on the new set of information. Pivot as/when needed. This significantly cut down on time it took for us to collectively conclude on open items, whether related to strategy or execution, in an iterative manner.

3) Keep pushing

One impossible thing leads to another. We all need to keep our eyes and ears open for the next one—and speak up and push for solving the next set of challenges, often in parallel.

One example: Disrupting the Cloud Orchestration Software Stack. In the first few months since our kick-off, as we started to architect for the application centric Rakuten Cloud Platform (RCP) covering thousands of far Edge locations (arguably the largest distributed telco cloud deployment till date) as well as a few data center locations, we realized that it would not be possible to use the recommended virtual infrastructure management (VIM) software platform as-is, due to far Edge location specific constraints (mainly power and space) and standard x86 COTS servers, though it would be fine for the centralized data center locations having no such constraints.

In the next few weeks, our multi-domain multi-functional teams came up with an innovative software –only solution that maximized the usage of Edge hardware resources (from ~30% to >90%) for the application workloads. It comprised of decoupling the storage service (e.g. object storage) & moving it to remote locations, coupling control service with compute nodes while ensuring deterministic Linux OS performance, and allowing for remote management & monitoring. We even devised the backhaul network infrastructure to be used as the distributed storage for VIM using CDN constructs.

Over the next several months, our teams collaborated diligently with x86 COTS hardware vendor, application workload vendor(s) etc. to ensure that our software stack worked better than ever before.

Another example: Rakuten Mobile’s IPv6 block authorization. Every operator has to get an IPv6 block, and Rakuten Mobile had requested and was given one from APNIC—the addressing allocation body in Asia Pacific. When we saw the size of this allocation, we knew it was not enough, based on our detailed analysis & projection (for the growth in not only wireless services, but also wireline services).

For most operators, the allocation would have been ample. But most operators get constrained by their networks, and Rakuten Mobile is not. That’s because together we had architected, designed and deployed an incredibly flat, simple and efficient infrastructure for network, storage and compute platform that would allow Rakuten Mobile to deliver myriad new services with no strain on the network. No changes needed given the well-defined routing summarization & boundaries.

And so, we took a highly unusual approach: Cisco CX went back to APNIC on Rakuten Mobile’s behalf. We looked five years down the road at all the amazing services this network made possible and modeled how big of an address pool Rakuten Mobile would need. We brokered the discussion and provided justification as to why Rakuten Mobile needed a substantially larger address block.

Did we compromise on our IPv6-only mandate? Yes, we did in a few cases. Despite the stringent timeline, most of the vendors managed to get the compliance for their application workloads, but in a few cases, where the applications workloads needed only layer 2 connectivity, we relaxed our mandate to allow for IPv4 as a temporary solution until the roadmap was adjusted & committed.

Did we learn anything? Absolutely. We learned a great deal about realizing a disruptive architecture and design as well as building and implementing it. While we uncovered new issues early on, such as separating the management plane reachability from forwarding plane reachability, intersecting xHaul with Data center etc., the resulting network has proven to be simple, highly resilient and efficient so far.  We will continue to go through the growing pains of maintaining the simplicity and consistency

The bottom line: these three tenets –do things that have never been done before, embrace radical transparency, and keep pushing—form the foundation of the work that Cisco CX does with customers. Other customers might be reaching towards very different outcomes, but the core values that get you there are the same.

Learn more about Cisco and Rakuten Mobile by watching this video interview of Tareq Amin, Group CTO of Rakuten.