To cross a busy intersection safely, it’s best to have all of your senses alert. That way, if you don’t happen to see that oncoming truck ignoring the “Walk” sign, you will probably still hear it. In the case of a heavy cement mixer, you may even feel the low rumble of its powerful engine first.
In the Internet of Everything (IoE), a similar principle applies. We call it “sensor fusion,” and it involves combining two or more sensors — often of different types — to monitor a specific environment and offer actionable insights more intelligently. These could be cameras and Wi-Fi tags or weight-sensing shelves and ultrasonic imaging, to name just two combinations. Moreover, the combined sensor data can itself be fused with other information streams — for example, those relating to weather, operations, news, or social media.
The result? Highly informed, real-time decision making and richer customer experiences.
Until recently, sensor fusion has been mostly exploited in specialized devices such as robots, but it is now driving a revolution in enterprise systems. This will bring new life to entire industries and completely transform stores, manufacturing floors, and transportation corridors. By greatly improving the accuracy of their measurements, organizations will be able to offer rich new experiences and gain substantial competitive advantage.
Let’s look at some examples of how sensor fusion can help transform specific industries:
- Retailers would better understand shopper behavior if they could use a camera to sense a customer’s demographics upon store entry. The retailer could then employ Wi-Fi to sense the customer’s path while shopping. In addition, traceable tender would further correlate the customer’s market basket to any other purchase information on record, telling a complete 360-degree story of that shopper’s trip.
- Energy companies could plan maintenance cycles more efficiently by measuring vibrations in pipes while also analyzing videos for signs of pipe corrosion.
- Pharmaceutical companies could deploy pills embedded with chips to measure vital signs, then combine that data with patient records and social-media streams. The combined data would, for example, offer new insights into the effectiveness of drugs.
- A rail operator could improve safety and reduce operating costs through simultaneous monitoring of key track components via vibration, temperature sensors, and video, predicting the optimal time for maintenance and averting potentially disastrous malfunctions.
More fundamentally, sensor fusion can be used across industries to provide precise indoor location and navigation by combining GPS, accelerometers and gyros, Wi-Fi, and other radio sources. For example, a mobile device could switch from GPS tracking to dead reckoning if a signal were lost upon entering a building, then switch to Wi-Fi-based location system in areas of good coverage. In addition, accelerometers could determine the floor on which the device is located, and subtle variations in electromagnetic field patterns would give the building a “signature,” allowing ever more precise indoor location.
Precision indoor location in turn provides a wealth of detailed information that can yield new insights into traffic patterns, asset location, product availability, process efficiency, and human behavior. This will enable a new generation of rich interactive experiences and operational excellence. For example, precision indoor location can offer retailers a detailed picture of conversion rates, spending per minute, or margin per minute in specific areas as compared with the average for the store’s total floor space.
In addition to providing unprecedented insight and fostering operational excellence, sensor fusion powers a wide range of new experiences. Here are two examples:
- Diabetics can receive tailored recommendations on which drink to purchase from a vending machine. This can be based on their medical histories (linked from their phones), combined with analysis of their breath captured near the machine, and then tuned to provide the nutrients most needed at that moment.
- Hesitant electronics shoppers can be guided through the decision process on their mobile devices and converted, based on a personalized decision tree created from knowledge of products they have investigated, their body language, eye movements, and what they have bought in the past.
With sensor fusion set to transform industries, it is essential for CXOs to begin brainstorming opportunities for combining sensors and data streams — if they hope to remain competitive.
Here are some first steps for CXOs to consider:
- Look for ways to combine multiple types of sensors and actuators to gain competitive advantage — IoE has already matured to the point where single-sensor networks are prevalent, and advantage will accrue to those who pursue multi-sensor fusion today.
- Establish standards for sharing information from different types of sensors, aligned with the enterprise data model (i.e., a common “canonical” messaging format that provides consistency in data communications).
- Establish a data-quality team to validate and improve the accuracy of data from sensors and to look for ways they can work in concert to improve sensing accuracy, completeness, and reliability.
- Focus on creating new experiences that are most helpful to the target audience —make the experiences delightful and valuable first, and rich consumer insight will follow.
- Include wearable computing in the repertoire of sensors and actuators — wearables are poised to explode in popularity and provide valuable sources of input, as well as experience platforms.
Sensor fusion: it can lead you across the street or into a whole new realm of transformation. I’d love to hear your thoughts on how sensor fusion can impact your company or industry.
Combining data from multiple sensors and converting that to some sort of actionable information is absolutely critical. I would propose that the sensors should be connected with some standardized web services interface which would allow backend services to combine this information in ways that has yet to be imagined.
Greg, Thank you for your comment, and excellent points – standards for connectivity and discoverability of sensors and actuators will indeed be key to accelerating to the value of the IoE, and exposing them through web services is a great way to catalyze innovation.
Thank you, Shaun – some very good examples. I’ve never heard of sensor fusion before, and it seems (compared to increasing the sensitivity of the single sensor to greater degrees) a great way to increase accuracy without dramatically increasing cost.
This sensor fusion solution does require a good deal of immediate and local data processing, however. So along with lots of local sensors you’ll need lots of local processing. And a local network that can handle the sensor data. (Not sure if that’s a lot of bandwidth though – I guess it depends on the frequency of measurements and the number of sensors per network receiver?)
Hi Rich – thanks for the comments – that’s a great point about the need for local processing, and one that Cisco is focusing significant effort on with our drive to enable Fog Computing by powering the network to filter out only the most useful information at the edge, saving a great deal of bandwidth overall. Our Data in Motion technology for edge routers is a great option for controlling the volume and frequency of data reported up from sensors at the edge.
Sensor fusion is definitely a thing. Take air purifier for example, I know there are appliance vendors incorporating all sorts of sensor onto their designs like odour, light opacity, temperature, humidity, even PIR to detect presence of motions. The data collected from the sensors aren’t just for the consumer’s comfort but also could be collected to pin point environmental conditions at a very local level for big data processing. Very useful!
One point from your post that immediately jumped out at me too,
“Focus on creating new experiences that are most helpful to the target audience —make the experiences delightful and valuable first, and rich consumer insight will follow.”
This is OH SO important and unfortunately I do find it often not being treated at a higher priority. If you want those sensors continue to output data you need the users to continue to use it.
Finally a comment on Rich’s post regarding local data processing on the sensor data. My experience is at the moment most sensor signals are processed on the device by the CPU/MCU and only simple data is being outputted onto the network through wireless. The biggest reason is the power consumption on the embedded wireless module is still too high to send out frequent large size data especially for consumer wearables. You don’t want to have to charge your smart watch every day… battery technology is still not up to pace unfortunately.
Thank you Shaun, I find the idea of sensor fusion really interesting but that of combining pills with chips very scary, I hope you don’t imply that patients will swallow a new chip per pill and keep all in their bodies, ouch!
Rana, thanks for the kind words – sensor fusion does indeed present some far reaching possibilities, and we need to make certain that it’s used to serve the good of businesses and society. Proteus Biomedical is a company that’s meeting that challenge today by creating a digestible sensor that you don’t even notice when taking a pill, but it can provide a wealth of useful health information! There’s a great video you may want to check out at http://www.proteus.com/