A consequence of the Moore Nielsen prediction is the phenomenon known as Data Gravity: big data is hard to move around, much easier for the smaller applications to come to it. Consider this: it took mankind over 2000 years to produce 2 Exabytes (2x1018 bytes) of data until 2012; now we produce this much in a day! The rate will go up from here. With data production far exceeding the capacity of the Network, particularly at the Edge, there is only one way to cope, which I call the three mega trends in networking and (big) data in Cloud computing scaled to IoT, or as some say, Fog computing:
Dramatic growth in the applications specialized and optimized for analytics at the Edge: Big Data is hard to move around (data gravity), cannot move data fast enough to the analytics, therefore we need to move the analytics to the data. This will cause a dramatic growth in applications, specialized and optimized for analytics at the edge. Yes, our devices have gotten smarter, yes P2P traffic has become largest portion of Internet traffic, and yes M2M has arrived as the Internet of Things, there is no way to make progress but making the devices smarter, safer and, of course, better connected.
Dramatic growth in the computational complexity to ETL (extract-transform-load) essential data from the Edge to be data-warehoused at the Core: Currently most open standards and open source efforts are buying us some time to squeeze as much information in as little time as possible via limited connection paths to billions of devices and soon enough we will realize there is a much more pragmatic approach to all of this. A jet engine produces more than 20 Terabytes of data for an hour of flight. Imagine what computational complexity we already have that boils that down to routing and maintenance decisions in such complex machines. Imagine the consequences of ignoring such capability, which can already be made available at rather trivial costs.
The drive to instrument the data to be “open” rather than “closed”, with all the information we create, and all of its associated ownership and security concerns addressed: Open Data challenges have already surfaced, there comes a time when we begin to realize that an Open Data interface and guarantees about its availability and privacy need to be made and enforced. This is what drives the essential tie today between Public, Private and Hybrid cloud adoption (nearly one third each) and with the ever-growing amount of data at the Edge, the issue of who “owns” it and how is access “controlled” to it, become ever more relevant and important. At the end of the day, the producer/owner of the data must be in charge of its destiny, not some gatekeeper or web farm. This should not be any different that the very same rules that govern open source or open standards.
Last week I addressed these topics at the IEEE Cloud event at Boston University with wonderful colleagues from BU, Cambridge, Carnegie Mellon, MIT, Stanford and other researchers, plus of course, industry colleagues and all the popular, commercial web farms today. I was pleasantly surprised to see not just that the first two are top-of-mind already, but that the third one has emerged and is actually recognized. We have just started to sense the importance of this third wave, with huge implications in Cloud compute. My thanks to Azer Bestavros and Orran Krieger (Boston University), Mahadev Satyanarayanan (Carnegie Mellon University) and Michael Stonebraker (MIT) for the outstanding drive and leadership in addressing these challenges. I found Project Olive intriguing. We are happy to co-sponsor the BU Public Cloud Project, and most importantly, as we just wrapped up EclipseCon 2014 this week, very happy to see we are already walking the talk with Project Krikkit in Eclipse M2M. I made a personal prediction last week: just as most Cloud turned out to be Open Source, IoT software will all be Open Source. Eventually. The hard part is the Data, or should I say, Data Gravity…
Since Henry Ford, the alchemy of turning raw materials into mass-produced products has been complicated and challenging. At best, it has been a delicate and precarious balancing act; at worst, something akin to herding cats.
The trick has always been to align ever-shifting patterns of customer demand with far-flung ecosystems of miners, designers, suppliers, engineers, factory workers, truck drivers, sellers, and so forth. Yet the process of orchestrating such intricate value chains has often been based on art (hunches) more than science (data).
Today, however, the Internet of Everything (IoE) — the ongoing explosion in networked connectivity among people, process, data, and things — is transforming manufacturing in startling ways, just as it is changing so many other industries.
IoE delivers seamless, intelligent connections to every corner of the manufacturing value chain, optimizing the flow of products, information, and payments in real time.
The Cisco IoE Value Index study found that in 2013, manufacturing had the largest potential share of IoE Value at Stake, at $224 billion. Yet, it was poised to realize only 46 percent of that potential bottom-line value. The key to closing that gap lies in much-improved machine-to-machine and machine-to-people connections, resulting in smart factories, smart grids, and connected supply chains, among many other IoE-related innovations.
Never before has mankind had access to such an ever-widening range of personal communication options, giving us the ability to create, disseminate and consume information immediately. The frenetic pace at which devices join the Internet is unprecedented, and the constant growth in the amount of data traversing the Web is far from peaking. This whirlwind of data surrounding us will continue to expand as more devices push and pull content across the Internet faster and faster.
Ye Olde Story of Big Data
Disclaimer: Jeff Jarvis and Kindle were not involved in this article -but you may want to check his book!
Before the Internet began its deluge of data, the world was overwhelmed by another data explosion when, in the mid-15th century, Johannes Gutenberg invented the printing press. In the 50 years that followed, Europeans printed more books than all of the manuscripts written in the previous 950 years, prompting the great Dutch humanist Desiderius Erasmus to ask, “Is there anywhere on earth exempt from these swarms of new books?”
People of the 16th century responded to the unbridled volume and variety of printing press output with waves of innovation. With the slow output of manuscripts behind them, strategies emerged to manage the burgeoning content, including the development of bibliographies to catalog all the books written, advances in note taking to summarize the information learned, encyclopedias to organize information by subject and public libraries to share the expanding content.
Big Data Redux
Today, we create more data in two days than all the data produced from the dawn of civilization until 2003 (Tweet This). That’s 5000 years of data overrun every 48 hours. Erasmus’s question is still applicable today, with a slight twist: “Is there anywhere on earth exempt from these swarms of new [content]?” Additional devices connect to the Internet daily, while content grows exponentially, which leaves me wondering what will happen when the swarms of new content overrun 5000 years of data in an hour or less?
The 21st century is also responding to its unbridled volume and variety of content. However, the proliferation of the number of devices adds a third dimension with a timely twist: velocity. Velocity is derived from the Latin word Velox, meaning swift or rapid. While volume and variety describe the size and shape of data, velocity describes the rate at which data moves, and data cannot move without infrastructure. The swiftness of infrastructure (megahertz, input/output, bandwidth and latency) and the ability to rapidly enable optimal resources (Network, CPU, Memory and Storage) both directly impact the velocity of data. When data velocity increases the value of information rises, which lifts business performance.
Cisco UCS and Big Data
The Unified Computing System is designed so businesses can harness the power of velocity. UCS successfully combined network and compute with the ability to assign resources rapidly. Tens of thousands of customers confirm the benefits derived from dynamic provisioning, reduced management time and efficient data center utilization. UCS extends swift performance with the addition of solid state memory, validated by 82+ World Record benchmarks. UCS combines network, compute and flash memory within a modular, scalable and extensible architecture.
UCS’s agility means workloads can move into service quickly. Its performance enables multiple workloads to consistently operate at high velocity. It shifts effort away from configuring and tuning infrastructure and towards new application deployments and feature enhancements. With UCS, businesses can address expected and unexpected demands with equal aplomb.
The printing press of everything rapidly spread across Europe in the 16th century. The flood of books reshaped European societies as they transformed in response to the outpouring of content. In The Internet of Everything, our devices (which serve as printing press and books) spread data between people and autonomous devices immediately. We attempt to synthesize data in real-time as the number of people and autonomous devices communicating increase globally.
Big Data Version One emerged 500 years ago to wrestle with data volume and variety. Today, Big Data Version Two grapples with data velocity (time) in addition to wrestling with volume and variety. Timely information rules when the Internet rewards the swift and penalizes the slow (Tweet This). Now is the moment to master velocity. What would your business be able to do with more time? Let us know in the comments.
March is a rather event-laden month for Open Source and Open Standards in networking: the 89th IETF, EclipseCon 2014, RSA 2014, the Open Networking Summit, the IEEE International Conference on Cloud (where I’ll be talking about the role of Open Source as we morph the Cloud down to Fog computing) and my favorite, the one and only Open Source Think Tank where this year we dive into the not-so-small world (there is plenty of room at the bottom!) of machine-to-machine (m2m) and Open Source, that some call the Internet of Everything.
There is a lot more to March Madness, of course, in the case of Open Source, a good time to celebrate the 1st anniversary of “Meet Me on the Equinox“, the fleeting moment where daylight conquered the night the day that project Daylight became Open Daylight. As I reflect on how quickly it started and grew from the hearts and minds of folks more interested in writing code than talking about standards, I think about how much the Network, previously dominated, as it should, by Open Standards, is now beginning to run with Open Source, as it should. We captured that dialog with our partners and friends at the Linux Foundation in this webcast I hope you’ll enjoy. I hope you’ll join us in this month in one of these neat places.
As Open Source has become dominant in just about everything, Virtualization, Cloud, Mobility, Security, Social Networking, Big Data, the Internet of Things, the Internet of Everything, you name it, we get asked how do we get the balance right? How does one work with the rigidity of Open Standards and the fluidity of Open Source, particularly in the Network? There is only one answer, think of it as the Yang of Open Standards, the Yin of Open Source, they need each other, they can not function without the other, particularly in the Network. Open Source is just the other side, the wild side!
I have just returned from a very interesting and jammed-packed week at Mobile World Congress 2014 in Barcelona. More than 75,000 people were estimated to have attended this year’s MWC, and its fabulous new conference facilities proved a great place to celebrate the industry’s accomplishments and catch a glimpse of its potential future. Much has changed in the industry over the last year since I reported my observations of MWC 2013. However, what is most remarkable is how the boundaries of mobility continue to expand and morph – everything now seems to be mobile?
The following are my personal observations and extrapolations from the show, based on my conversations with operators, customer meetings, analysts, and colleagues, as well as from simply walking the show floor: Read More »