In my last Blog – Finding value in the Big Data Mine, I spoke about putting big data analytics to work, the power of embracing technology and the data explosion to identify and extract maximum value from the vast amounts of information, particularly where it is unstructured.

Now, I want to look at some of the risks associated with Big Data and ways in which to ameliorate them.

Many companies are now looking to understand what is on their file servers, so that they can assess the risks and opportunities that may lurk within that data. Unmanaged file servers can pose legal and compliance risks and may cause vast disruption if a company is asked to produce documents to a court or for an internal enquiry and this is an essential starting point.

Unmanaged information represents a risk because it makes it hard to find information – particularly when much of that information is unstructured. When litigation occurs, if information cannot be found, organisations may ultimately face court sanctions.. In situations where an organisation does not have a company-wide litigation readiness strategy that involves properly indexed information, the risk is that they may have to settle cases they could otherwise win and do so on punitive terms. Alternatively, the organisation risks losing cases because they cannot find the evidence that they are otherwise certain exists.

Another form of risk which ultimately develops from unmanaged information is compliance risk. For example, sensitive client information that may reside on servers that are not managed may be misused, lost or even destroyed.

So, what about the risk inherent in having to produce documents during a discovery process? In the early days of eDiscovery a few years ago, organisations mainly took a reactive approach to the process of identifying relevant. As companies started to see the impact of this approach, in terms of cost, disruption and legal risk of not being able to easily recall information they began to move towards a more proactive approach. Such an approach involves insuring that information is properly indexed and either retained in the right place or destroyed. This involves what we call an index-once, use-many approach; the idea that if you have a single index of all corporate information it makes finding information considerably easier and it can be used for many reasons, beyond legal ones. That is because not only are all the key repositories connected to the index but also the users can use a single query to find what they are actually looking for, so they don’t have to learn the quirks of numerous search engines.

In my days as an analyst I saw numerous end user customers – mainly financial services companies, as they tend to be the ones leading the charge – that didn’t think it was cost-effective to proactively manage information in this way. Now many of these same companies are realising that such an approach represented a false economy and it creates too many risks to not have such information policies in place. They are now looking to implement technology that enables them to quickly identify what information resides where, what it is and therefore whether it needed to be retained or destroyed. The latter approach is seen as controversial by some, but organisations are embracing the idea that information deletion is a legally-sound policy as long the deletion is defensible; if it can be shown that information was deleted defensibly, there’s nothing to fear from such a policy. It’s a case of having both the right technology and the right policies in place

The rewards of analysing Big Data can be great and the risks of retaining what you need and getting rid of what you don’t are manageable. Welcome to the new reality of information management.