I am attending South Korea’s Big Data Forum in Seoul, and one question here is, “How big is Big Data?” My friend and colleague Dave Evans has pointed out that by the end of this year, more data will be created every 10 minutes than in the entire history of the world up to 2008. Now, that’s big!
Much of this data is being created by billions of sensors that are embedded in everything from traffic lights and running shoes to medical devices and industrial machinery—the backbone of the Internet of Things (IoT). But the real value of all this data can be realized only when we look at it in the context of the Internet of Everything (IoE). While IoT enables automation through machine-to-machine (M2M) communication, IoE adds the elements of “people” and “process” to the “data” and “things” that make up IoT. Analytics is what brings intelligence to these connections, creating endless possibilities.
To understand why, let’s step back and take a look at the classic approach to Big Data and analytics. Traditionally, organizations have tended to store all the data they collect from various sources in centralized data centers. With this model, if a retailer wants to know something about the buying patterns of a certain store’s customers, it can create an analysis of loyalty card purchases based on data in the data warehouse. Collecting, cleansing, overlaying, and manipulating this data takes time. By the time the analysis is run, the customer has already left the store.
Big Data today is characterized by volume, variety, and velocity. This phenomenon is putting a tremendous strain on the centralized model, as it is no longer feasible to duplicate and store all that data in a centralized data warehouse. Decisions and actions need to take place at the edge, where and when the data is created; that is where the data and analysis need to be as well. That’s what Cisco calls “Data in Motion.” With sensors gaining more processing power and becoming more context-aware, it is now possible to bring intelligence and analytic algorithms close to the source of the data, at the edge of the network. Data in Motion stays where it is created, and presents insights in real time, prompting better, faster decisions.