As investments go, Big Data is currently dramatically underperforming the market. Driven in part by media hype, enterprises are investing in Big Data well in advance of actually understanding how to derive value from it (http://readwrite.com/2013/09/18/gartner-on-big-data-everyones-doing-it-no-one-knows-why#awesm=~ohSGAsXumnTLgC). Small wonder, then, that a new analyst report indicates that enterprises are deriving far less value from Big Data than they expect, or even than they invest.
Pennies On The Big Data Dollar How much less? Dramatically less, according to preliminary findings from Wikibon research (http://wikibon.org/wiki/v/Enterprises_Struggling_to_Derive_Maximum_Value_from_Big_Data).
(http://readwrite.com/files/Big%20Data%20ROI.png)
Not surprisingly, Wikibon found that 46% of Big Data practitioners have eked out only partial success from their projects, and an unfortunate 2% have had to write off their investments as complete failures.
Which is not to suggest that Big Data is a sham that will inevitably end in tears. Not at all.
As Wikibon points out, one of the biggest reasons for Big Data failure is that "enterprises invest in Big Data technology such as Hadoop without specific and measurable business applications tied to the projects." They hear it's a big deal and throw money at it without really understanding what they're hoping to achieve. This is consistent with the Gartner analysis I shared recently (http://readwrite.com/2013/09/18/gartner-on-big-data-everyones-doing-it-no-one-knows-why#awesm=~ohSGAsXumnTLgC).
Have Hadoop, Will Dump Data
See also: Big Data And The Landfills Of The Digital Enterprise (http://readwrite.com/2013/02/11/big-data-and-the-landfills-of-our-digital-lives)
For too many enterprises, Big Data equals Hadoop, which roughly translates into "unsupervised digital landfill," as one IT executive shared with me. It's therefore not surprising that Julien Simon, vice president of Engineering at Criteo (http://www.criteo.com/), suggests (https://twitter.com/mjasay/status/380605469808553984) that "Too often Big Data is simply Big useless Data."
We hoard data without knowing why we need it, and end up collecting data that simply creates noise and obscures the signal in our data. As renowned statistician Nate Silver posits (http://readwrite.com/2013/03/29/nate-silver-gets-real-about-big-data#awesm=~ohT4K2wdnczRqM):
If the quantity of information is increasing by 2.5 quintillion bytes per day, the amount of useful information almost certainly isn't. Most of it is just noise, and the noise is increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine--but a relatively constant amount of objective truth.
Adding more data, in other words, is not the solution to a Big Data problem. It's often the cause of the problem, and one big reason that so many companies are getting very little value from their Big Data investments.
What Big Data Success Looks Like The best Big Data projects, according to Wikibon's research, "are generally not initiated by IT but driven by line-of-business departments, often marketing, and focus on small but strategic use cases." They tap into in-house expertise and are realistic about how much they can do with the people they have.
Such projects tend to start small, iterate and scale out based on early wins.
Lest you find yourself setting big money on fire to run your Big Data project, you would be advised to do the same. Given that all of the best Big Data technologies are open source, it's easy to try before you buy, and get up-to-speed on the best technologies for your requirements.
Image courtesy of Shutterstock (http://www.shutterstock.com).ReadWrite