This vendor-written piece has been edited by Executive Networks Media to eliminate product promotion, but readers should note it will likely favour the submitter's approach.
Recently I was invited as a speaker by one of the premier technology think tank group in Thailand on the topic - 'Big Data and evolving privacy concerns'. The technology industry is quite interesting for its "seasonal infatuations" on certain topics that gets discussed to the hilt, but quite often remains shallow in its analysis and at times even the understanding. However this session was quite different with the audience and fellow speakers that included academicians to policy makers looking at the topic from various lenses.
The topic of privacy and big data is quite a relevant one for sure. The explosion of data creation and the 'beginning' of its capture and 'thinking towards doing an almost instant analysis' has elevated the issue of personal privacy to a matter of debate. The industry's attempt towards finding a solution or a coordinate where the line could be drawn has also started. However what is missing is the fundamental understanding that this conundrum belongs to a segment, which does not and cannot have a straightjacketed answer.
For the sake of clarity let us look at the fundamental premise of 'why' and 'what' is big data all about. The basic premise behind bothering about big data and its analysis is that 'randomness is a rendition of the limitation of human perception'. The world and the way it works has an underlying pattern which is why our research findings indicate that history repeats and humans are increasingly understood as 'creatures of habit'. However all decisions and plans that drive and run this world are made with very limited amount of data leaving the remaining part of the probability to 'chance'. However chance to a great extent is nothing but missing information, which by definition is 'processed data'. With the advent of technology to capture and store the remaining 80% the thinking towards analyzing that has started taking shape, and with it the debate on 'intrusion'
This debate to me is analogous to the social debate on "whether the society or the individual is more important". The reason I find this analogous is the common and broader objective of both sides of the debate, which is 'making the world a better place for its inhabitants'. The debate on the society v/s individual is broadly based on the thinking that if the individual is given more priority than the society and social norms, the individual is more enlightened and collective enlightenments make the world great. The other half of the debate is based on the assumption that the society is pivotal and needs to be the only priority which might mean norms and social practices that could be detrimental to individual freedom and empowerment but on the whole makes the world better.
Sign up for Computerworld eNewsletters.