Big data has been correctly labelled a disruptive force in IT as it caught the market by surprised and left organisations flummoxed about how to deal with it. No one could have anticipated that the third platform technologies they had adopted as a way of doing business would result in the Big Bang that created the Digital Universe.
A sense of ownership
Discussion about big data continues as technology vendors vie with one another to "own" the concept of big data by putting certain exclusivity to it through some cutting edge technology in attempts to harness its commercial potential. This is not necessarily a bad thing.
The phenomenon of big data, along with other disruptive forces such as cloud computing, mobility and social media, has fuelled a new renaissance in innovation and technological advancement that has implications far beyond the corporate sphere.
The discussion around big data has led to a certain air of expectation. The Australian market was initially seen as slow in adopting big data technologies, but this was fuelled by a misconception that big data was only about volume.
As yet, most Australian organisations do not have anywhere near the hundreds of terabytes of data typical of organisations in Europe and the US. The focus for Australian organisations revolves more around the variety of data they collect and what use they put it to.
There are many in the market and in the industry that dismiss big data as mere hype. However, this does not make the reality of the situation go away. We are still faced with a growing volume and variety of data at a pace never experienced before.
Where technology has traditionally been able to evolve to meet the demands of the collection and analysis of information, the data deluge we are seeing currently is like a tsunami we were unprepared for.
It is not difficult to see how hype and misconception of big data have led some to become disillusioned with the whole concept. I would suggest however, that the disillusionment is only a result of not being able to see its full potential.
There are already examples of it being used in many instances outside the corporate sphere such as crime prevention, where law enforcement agencies are able to identify crime hotspots through crowd sourcing and take pre-emptive action that in the end contributes towards a better quality of life in the neighbourhood.
While this is just one example, there are many other use cases for big data that would have a tremendous impact on the quality and indeed the value of human life. The digital universe is constantly expanding and if not already, will soon hold the formula to eradicate terminal disease; and to predict weather pattern that lead to drought and famine; and to identify and trace the root of terrorism.
Sign up for Computerworld eNewsletters.