Listen to much of the well-peddled advice in the enterprise tech world today, and you'd have to be excused for coming away with the belief that "big data" holds all the answers your company is looking for. Too bad it often can't live up to that promise -- at least, not in its traditional form.
Turns out, what's commonly referred to as big data -- all those vast "lakes" of numerical measures captured by the enterprise resource planning (ERP), consumer relationship management (CRM) and other business systems so enthusiastically mined by today's analytics tools -- actually amounts to only 10 percent of the data an average company has at its fingertips, according to IDC.
The rest is "unstructured" or "qualitative" data, and it can be messy. Included in this type is information from customer surveys, response forms, online forums, social media, documents, videos, news reports, phone calls to call centers and anecdotal evidence gathered by the sales team, to name just a few examples. It's typically textual rather than numerical, and it's not easily "quantified," or turned into numerical values.
Therein lies a problem. While most analytics tools are set up for quantified information -- crunching numbers, in other words -- it's often the unstructured data that provides the context and the meaning companies needs to make that information useful.
"Data can often raise more questions than provide answers, and there is always the question of 'why?' behind the quantitative data trends," said Anjali Lai, an analyst with Forrester Research. "Data analyzed in a vacuum risks telling an incomplete story, and qualitative data can provide this contextual view."
Imagine you're at a company trying to understand why online sales aren't where you'd like them to be. You can invest heavily in marketing analytics tools that give you activity-based data such as how long users spend on which Web page on average or the abandonment rates for users' shopping carts, for example. Such data, however voluminous, still won't necessarily give you the "why?" part of the equation.
"You may know you have 10 thousand unique visitors to your site -- that's quantitative data," said Collin Sebastian, chief product officer for YouEye, which offers software and services designed specifically with qualitative data in mind. "Qualitative data can tell you that four thousand of them came in excited about a particular topic, this is what they hoped to learn, this many did not have their expectations met, and this is what they did instead."
Qualitative data can go beyond identifying correlations between data points, which may tell you, for example, that people who spend longer on your website tend also to make purchases. Instead, qualitative data can begin identifying causal relationships, or that elusive "why?" Do people make purchases because they've spent longer on your site, for instance, or are they simply on the site longer because your purchase process is cumbersome?
Sign up for Computerworld eNewsletters.