Alas, poor Yorick! I knew him, Horatio; a fellow of infinite jest, of most excellent fancy; he hath borne me on his back a thousand times; and now, how abhorred in my imagination it is!
From the play Hamlet by William Shakespeare
Big Data is dead! Long live Information Management.
Of course I am overstating. It isn’t tangibly dead, because outside of PR and Marketing, and the gullible imaginations of a few punters with more dollars than sense, it was never actually alive.
I don’t blame the industry for ‘bigging up’ Big Data, it was a ‘common sense’ continuation of the hubris and nonsense that began with total hegemony and total information awareness. It was a ‘natural’ response to a set of grandiose, flawed and quite unrealisable strategic objectives.
This is why Creative Directors in Advertising have such a hard time with it, because it’s a dopey ‘brand’ of sorts that cannot even be pinned down. It’s vague, amorphous and brittle, that tries to be all things to all people, and trying to turn that into attention grabbing advertising is like trying to nail Jell-O to an elephant.
So, Big Data will be no more. It will be ‘deceased’. As deceased as a Norwegian Blue, as dead as a Dodo, and as stiff as a cardboard box on the steppes of Russia in winter. But its demise is not going to substantially change a thing.
Why?
Because before Big Data there was everything that Big Data is supposed to be uniquely capable of doing, and much more, and this will be the case when the faddish use of the Big Data label is dropped in favour of other (possible equally fuzzy or meaningless terms).
Before Big Data came along we could do distributed and parallel processing, and performed database loading and querying, handling larger and larger volumes of data, from an ever growing variety of sources.
I know younger readers may be shocked to discover this, but before Big Data came along we could also perform database restrictions, projections, joins and relational operations.
Finally, before Big Data came along we could perform statistical analysis on data.
Remarkable, huh?
You see, Big Data has been a faddish industry label, and for some, as confusing as it has been misleading.
I only have five issues with Big Data.
The first is the hype and all that serves to perpetuate it.
The second issue lies in the pervasive idea that more and more data is necessarily better.
The third issue is with the notion that extending information architecture and management will fall short when dealing with growing volumes of data, greater velocity, greater variety and increased needs for data veracity, especially in connection with the euphemistically termed unstructured data – which is in fact typically complexly structured information.
The fourth issue is the incessant claim that every organisation will need to be doing Big Data in order to survive and thrive.
The fifth issue concerns the widespread notion that Big Data has necessarily disregarded any ethical considerations in the use of data, or any legal requirements in terms of data privacy and data protection.
I have an issue with the hype[1] because in so many occasions it’s utterly preposterous and thoroughly misleading. One may also be lead to ponder if in fact the famous Big Data Vs of volume, velocity, variety and veracity refers to the data or to the hype. So, why should I care about that? Because, for better or worse, Information Management is my chosen profession, and as an ethical professional I want to see higher standards of professionalism. Nothing exceptional. It’s quite straightforward.
Just consider this dialogue from the absolutely fabulous British comedy show, Absolutely Fabulous:
Saffie: I'm sorry, mum, but I've never seen what it is that you actually do.
Eddie: PRrr.
Saffie: Yes, but...
Eddie: PR. I PR things. People. Places. Concepts...
Patsy: ...Lulu.
Eddie: Lulu... I make the fabulous... I make the crap into credible. I make the dull into...
Patsy: ...Delicious.
The second notion is that more data will always lead to better analysis[2]. This is certainly not something that has universal applicability. Random sampling has provided quite a reasonable and sufficiently sound basis in the past, so what changed? Just take the old saying of opinion poll analysts “If you don't believe in random sampling, the next time you have a blood test tell the doctor to take it all”. Added to that, I am always a little wary of people who claim that they can help others do a ‘better job’ - and not only for the degree of arrogance that this type of comment typically conveys.
Thirdly, there is a marked tendency to dismiss evolving best principles in information architecture and management as being unfit for the Big Data revolution[3], and incapable of adapting to increasing volumes, velocity and variety of data. A claim that is both appealing and absurd. Not for nothing did Bill Inmon take people to task[4] for even suggesting that Big Data should somehow replace Data Warehousing as the preferred means to support strategic and tactical information needs.
Fourthly, do all organisations need to maximise their use of all of their data? There are parts of the Big Data campaign that attempts to bludgeon all and sundry into submitting to the Big Data mantra[5]. Entire industrial sectors are railed against for not embracing the faith. Take the energy industry for example, parts of which have been analysing very large data sets for decades. Did these oil companies and other energy companies pre-empt the Big Data revolution or did Big Data hype find something tangible to attach itself to?
As Dan Ariely so succinctly put it: “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it...”
So, in fifth place in my list of Big Data issues[6] is the notion that we must accept what is to all intents and purposes a conscious disregard for ‘some’ of the legal requirements pertaining to the data protection and data privacy[7], and that ethical considerations regarding the use of data are somewhat irrelevant.
For these five issues alone we should welcome with open arms the overdue demise of the Big Data campaign. With this we can forge ahead and continually seek to incorporate new data requirements and information supply frameworks into a well-architected, well-engineered and evolutionary approach to the timely, adequate and appropriate supply of data and information.
So, just to wrap up, let’s revisit some contemporary replies to recycled old chestnuts from the Big Data circus:
  • Big Data is not a fad: Big data is a fad, but it can’t even manage the tangible fads and fashions of youth. It’s not even a smart fad.
  • We never had this before: More and more data is being generated, this has been happening since the beginning of time, but this does not mean that more and more data has value or that it is really usable for strategic and tactical purposes.
  • It's all pervasive: Big Data will not affect every aspect of human and organisational life as we know it. This is pure nonsense.
  • Analyse everything, now: We neither can nor should we strive to analyse everything, and such claims only serve to highlight the crass manipulation that is associated with this tsunami of fatuous and malignant hype.
Big Data is dead. Long live DW 3.0 and comprehensive Information Architecture and Management.
In subsequent blog pieces I will be sharing my views on the evolution of information management in general, and the incorporation of ‘Ad hoc Speculative-Predictive Analytics’ into well architected mainstream information supply frameworks for primarily strategic and tactical objectives.
This piece was first published on the Good Strat Blog:http://goodstrat.com/2014/12/20/big-data-is-dead/
As always, please feel free to share your questions, views and criticisms on this piece using the comment box below. I frequently write about strategy, organisational, leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the 'Follow' link and perhaps even send me a LinkedIn invite. Also feel free to connect via Twitter, Facebook and theCambriano Energy website.
For more on the topic, check out my other recent LinkedIn posts:
[1] Buytendijk, Frank. Gartner. Hype Cycle for Big Data, 2014. https://www.gartner.com/doc/2814517/hype-cycle-big-data-
[2] Big Data and competition – data-rich does not mean dominant. Global Data Hub. http://www.taylorwessing.com/globaldatahub/article_big_data_dominant.html
[3] Eckerson, Wayne. Big data vendors should stop dissing data warehouse systems. http://searchbusinessanalytics.techtarget.com/feature/Big-data-vendors-should-stop-dissing-data-warehouse-systems
[4] Inmon, W. H. Big Data or Data Warehouse? Turbocharge Your Porsche – Buy An Elephant. http://www.forestrimtech.com/big-data-vs-data-warehouse
[5] Gault, Sebastian. How oil and gas firms are failing to grasp the necessity of Big Data analytics. Alberta Oil. http://www.albertaoilmagazine.com/2014/10/big-data-blowout/
[6] Marr, Bernard. The Big Data Economy: Here's What You Must Know. https://www.linkedin.com/pulse/big-data-economy-heres-what-you-bernard-marr
[7] Does Big Data Mean the End of Privacy? - http://www.dataversity.net/big-data-mean-end-privacy/