Companies with many employees face various challenges w.r.t. their size. One of these challenges is to identify key people, skills and information across (and outside) the organization and use them in the most effective way to drive innovation, new initiatives, but also sales.
The natural way for employees to cope with such a challenge is to build social networks (I mean networks of people not software) and collaborate across the organization. This organic social network building happens through various activities such as projects with colleagues, social activities, company events, etc… However, such social networks take time to create, and are typically not that extended. The effectiveness of these social networks is hard to measure (unless perhaps you equip your employees with location trackers). This social network building, I call the qualitative approach to organizational collaboration.
Connect with the right people inside and outside the enterprise
Many companies have deployed technology solutions (tools) to cope with the challenge described above. Companies have personnel directories that show employees’ groups and the official organization hierarchy. Some of these personnel directories allow users to add more personalized information (but this information is not always up to date). More sophisticated personnel directories (or other collaboration tools) also feature timelines of activities/tasks, blogging, integrated search, etc…. .Video conferencing enables people to connect people remotely. Despite these new social tools, email and mailing lists still play an important role in connecting people and disseminate information as well as external social networks and resources. All these tools provide a wealth of information. In essence: collaboration is Big Data.
Do all these social/collaborative enterprise tools help us doing our job better and promoting innovation? From a personal point of view I am tempted to say yes, but much more can be done. My main concern with most of these tools is the lack of analytics features to quickly identify user-relevant information or contacts. New tools – and newer versions of already existing tools – are starting to provide some of these capabilities, but IMO that is still not enough (or not accurate enough) to fully understand the evolving social networks or the relations between people and information (documents, emails, etc…).
What can be Improved
The goal of exposing more of the right analytics to end users would be, for a user, to faster gather insights, new ideas, and enable quicker decision making and eventually translate these insights and ideas to new opportunities, projects and/or costs savings.
Correlate different sources to identify information
To achieve this goal, users should be able to identify patterns in their organization’s data, specifically on threads or evolving thoughts and interactions that can be relevant for their particular projects or questions. In essence, analytics should foster more and improved collaboration with like-minded people, or people that share a common goal. As mentioned earlier, people naturally do this already, but in large organizations it is humanly impossible to scale this effectively and fast, without the help of analytics tools. This type of analytics I call the quantitative approach of organizational collaboration, which I see as complementary to the qualitative approach.
When I look for example at mailing lists or video conferencing, a few questions always pop up in my mind that modern enterprise collaboration tools should be able to answer in just a few clicks:
- What topics are trending during the last week/month? Perhaps type a topic in and get trend information or have the computer generate topics based on a context analysis of your posts or email conversations.
- How are the groups and hierarchies evolving over time (who is talking to who)? Can software recommend groups of people that are relevant for me and my projects?
- What people can be considered as experts on certain topics, based on their posts, replies, published articles, etc…?
- For particular topics, who are the top contributors and how do they relate to the experts? Are people clustering around certain topics?
- Who are the influencers/thought leaders, and how do they relate to experts?
From a strategic point of view companies can leverage analytics from social/collaborative tools to answer questions like:
- Are best practices shared across the organization between the appropriate groups?
- Is there an alignment between strategy and direction of the company?
This is not an exhaustive list and as a software engineer I think that an additional relevant feature for any tool should be the ability to provide an environment to mashup and integrate data by employees, to answer some of these questions.
How can it be Leveraged
Various groups and people (MIT, Virginia Tech, …) do research on this subject and translate this research into strategic insights at the enterprise level. The next step will be to provide the insights to individual employees as well. Enterprise tools with more sophisticated analytics capabilities (many focused on machine learning) are beginning to emerge. Perhaps the biggest challenge is integration of such capabilities across multiple internal and external tools and platforms.
Organizational collaboration is for me not limited to an enterprise environment. Groups with different affiliations who organize themselves as “virtual organizations” to work together towards common goals (for example, Open Source communities or standard bodies) can benefit from this type of analytics too.
To be more successful in capturing the value of collaboration, companies not only need to deploy the right tools, but also need to foster a mashup environment to leverage the organizational insight and tacit knowledge of its employees through analytics.
Special thanks to Marco Valente and Yannik Messerli for the discussions and insight on this subject.
Tags: analytics, collaboration, Corporate Technology Group, CTG, innovation, social, strategy
Data has always been important for many (if not all) companies. Today however it is becoming increasingly easier to collect data about customers, business transactions, devices etc…, . This data enables companies to (more dynamically) optimize their business models and processes and differentiate themelves from their competitors. Before embarking on a (Big) data strategy it is important to understand what is driving this evolution and what are basic architecture building blocks to take into account when transforming into a more data driven enterprise.
Several trends are fueling the evolution towards a more data-driven enterprise: Price/performance improvements in network, storage and computing and the rise of Cloud computing , make it more cost effective to deploy large IT infrastructures and capture data. New data management technologies and products such as Hadoop (MapReduce) and NoSQL/NewSQL provide scalable and more cost effective solutions than traditional SQL databases for various classes of problems. The IT consumerization trend results in departments more actively pursuing analytics capabilities. Another important trend is the Internet of Things (IoT): The advent of cheaper sensors and improved connectivity are bridging the gap between the physical and digital world, enabling collecting data from more devices and environments. This sensorization is unlocking the potential to gather enormous amounts of data and details about almost everything.
These trends are creating new challenges and opportunities to harness and understand the data tsunamis, and leverage the analytics for decision making purposes, to better monitor, control, and understand complex systems from business dashboards to IoT eco-systems. Read More »
Tags: analytics, architecture, Big Data, Corporate Technology Group, CTG, Enterprise
More and more enterprises are managing distributed infrastructures and applications that need to share data. This data sharing can be viewed as data flows that connect (and flow through) multiple applications. Applications are partly managed on-premise, and partly in (multiple) off-premise clouds.
With the advent of the Internet of Things (IoT) the need to share data between applications, sensors, infrastructure and people (specifically on the edge) will only increase. This raises fundamental questions on how we develop scalable distributed systems: How to manage the flow of events (data flows)? How to facilitate a frictionless integration of new components into the distributed systems and the various data flows in a scalable manner? What primitives do we need, to support the variety of protocols? Read More »
Tags: Amazon Cloud Connector, Corporate Technology Group, CTG, internet of things, SDN, Service Provider
More and more enterprises are managing distributed infrastructures and applications that need to share data. This data sharing can be viewed as data flows that connect (and flow through) multiple applications. Applications are partly managed on-premise, and partly in (multiple) off-premise clouds. Cloud infrastructures need to elastically scale over multiple data centers and software defined networking (SDN) is providing more network flexibility and dynamism. With the advent of the Internet of Things (IoT) the need to share data between applications, sensors, infrastructure and people (specifically on the edge) will only increase. This raises fundamental questions on how we develop scalable distributed systems: How to manage the flow of events (data flows)? How to facilitate a frictionless integration of new components into the distributed systems and the various data flows in a scalable manner? What primitives do we need, to support the variety of protocols? A term that is often mentioned within this context is Reactive Programming, a programming paradigm focusing on data flows and the automated propagation of change. The reactive programming trend is partly fueled by event driven architectures and standards such as for example XMPP, RabbitMQ, MQTT, DDS.
One way to think about distributed systems (complementary to the reactive programming paradigm) is through the concept of a shared (distributed) data fabric (akin to the shared memory model concept). An example of such a shared data fabric is Tuple spaces, developed in the 1980’s. You can view the data fabric as a collection of (distributed) nodes that provides a uniform data layer to the applications. The data fabric would be a basic building block, on which you can build for example a messaging service by having applications (consumers) putting data in the fabric, and other applications (subscribers) getting the data from the fabric. Similarly such a data fabric can function as a cache, where a producer (for example a database) would put data into the fabric but associates this to a certain policy (e.g. remove after 1 hour, or remove if exceeding certain storage conditions). The concept of a data fabric enables applications to be developed and deployed independently from each other (zero-knowledge) as they only communicate via the data fabric publishing and subscribing to messages in an asynchronous and data driven way.
The goal of the fabric is to offer an infrastructure platform to develop and connect applications without applications having to (independently) implement sets of basic primitives like security, guaranteed delivery, routing of messages, data consistency, availability, etc… and free up time of the developer to focus on the core functionality of the application. This implies that the distributed data fabric is not only a simple data store or messaging bus, but has a set of primitives to support easier and more agile application development.
Such a fabric should be deployable on servers and other devices like for example routers and switches (potentially building on top of a Fog infrastructure). The fabric should be distributed and scalable: adding new nodes should re-balance the fabric. The fabric can span multiple storage media (in-memory, flash, SSD, HDD, …). Storage is transparent to the application (developer), and applications should be able to determine (as a policy) what level of storage they require for certain data. Policies are a fundamental aspect of the data fabric. Some other examples of policies are: (1) time (length) data should remain in the fabric, (2) what type of applications can access particular data in the fabric (security), (3) data locality, the fabric is distributed, but sometimes we know in advance that data produced by one application will be consumed by another that is relative close to the producer.
It is unlikely that there will be one protocol or transportation layer for all applications and infrastructures. The data fabric should therefore be capable to support multiple protocols and transportation layers, and support mappings of well-known data store standards (such as object-relational mapping)
The data fabric can be queried, to enable discovery and correlation of data by applications, and support widely used processing paradigms, such as map-reduce enabling applications to bring processing to the data nodes.
It is unrealistic to assume that there will be one data fabric. Instead there will be multiple data fabrics managed by multiple companies and entities (similar to the network). Data fabrics should therefore be connected with each other through gateways creating a “fabric of fabrics” were needed.
This distributed data fabric can be viewed as a set interconnected nodes. For large data fabrics (many nodes) it will not be possible to connect each node with all other nodes without sacrificing performance or scalability, instead a connection overlay and smart routing algorithms are needed (for example a distributed hash tables) to ensure scalability and performance of this distributed data fabric. The data fabric can be further optimized by coupling this fabric (and its logical connection overlay) to the underlying (virtual) network infrastructure and exploit this knowledge to further optimize the data fabric to power IoT, Cloud and SDN infrastructures.
Special thanks to Gary Berger and Roque Gagliano for their discussions and insights on this subject.
Tags: application centric infrastructure, cloud, Corporate Technology Group, CTG, data fabric, distributed systems, IoT, SDN
“Software is Eating the World” is a quote attributed to Marc Andreessen and somewhat further explored by his business partner Ben Horowitz. Mark Andreessen gives compelling reasons to validate this quote. To some extend I have to agree with some of his reasons (but I am also a little bit biased as a software engineer). On the other hand, when I read this (and this is partly based on working in different domains on software), I wonder if software is that disruptive? If you look “under the hood” of software applications, you find that a lot of software is based on fundamental software principles that are already 20-30 years old, yet Read More »
Tags: analytics, API, Corporate Technology Group, CTG, SDN, sensors, Service Provider, software