However, it throws up major privacy concerns for the customers and any major business admitting to selling off the data it analyses. All too often, many of the businesses are not willing to risk their customer relationships in return for better knowledge and partnerships with other organisations that could further improve the service or customer experience provided.
Facilitating better, faster decisions and arming employees with the correct tools is the correct route to success as big data analysis could be made as simple as possible, if it weren't for complex regulations and an unwillingness to share what is perceived as competitive insight.
What the big data potential does demonstrate, however, is an opportunity for open innovation. By empowering all the users within a business, and the possibility for change, innovative transformation should theoretically become feasible. Not only do organisations in all sectors have to remain agile and able to adapt quickly to stay ahead of the competition in their specific vertical sector, but also as companies' traditional roles evolve. This comes back to the diversification challenge and opportunity - and several failed attempts.
On the other hand, it is these information asymmetries that are enabling technology companies such as Google and Microsoft to do just that. Diversify by moving into the healthcare market, allowing customers to track their health and record their treatments. The same can be said for the Internet of Things and in creating a smarter world reliant on the connections around us but also the analysis of these, how else do we move from electronics companies creating smart fridge that speak directly to your preferred retailer to order your milk and yoghurt when you've run out?
The best big data is the data generated as a by-product of operational, customer and supplier processes. The data that people naturally share, and are willing to, in return for a better experience or end product. And the best big data is when it becomes information that is readily analysed by business users for useful insights.
Sign up for Computerworld eNewsletters.