Don’t let security concerns impact use of big data

Security is an on-going issue in IT, with data breaches and how to prevent them, top of the agenda for many CIOs. This has been exacerbated by the rise of big data. Business lines are keen to make the best use possible of big data, for marketing, HR or indeed any area of a firm’s operations that can benefit from the improved business insight that big data can deliver.

But is it possible to keep the business lines and IT happy? Is big data really that much of a security issue anyway? 

Big data = big security threat?
There have always been information leaks in business. But a number of high profile cases over the past 18 months have made it feel like the problem has reached epidemic proportions. With organisations increasingly holding and managing more data on consumers than ever before it does feel like there is a heightened security threat, but is this fair? Does it follow that because there is more data, the associated risk is greater too?

Part of the issue is that the nature of certain big data technologies means that users have less need than previously to even involve IT. The increase in use of bare metal cloud over recent months has meant that not only do organisations have sufficient power to manage large data sets, they only have to pay for what they actually use. This in turn means no significant infrastructure investments of millions of pounds – big data queries can be done in an afternoon for a matter of pennies.

But this easy availability of computing power means that different business lines within a business can mine their own data. This can mean that security controls are less rigorous than they would be if IT was involved. But the situation is definitely improving.

Taking big data security seriously
When Hadoop and partner organisations such as Cloudera first started to gain a foothold in the market, there weren’t many options available to protect data. Security vendors were yet to update their products to work within a Hadoop environment. But this has started to change. Last year Cloudera launched Sentry, an Apache licensed open source project that delivered the industry’s first fine-grained authorisation framework for Hadoop – a major leap forward when it comes to keeping big data secure and protected. In fact, most vendors have now got much better at addressing security, with many integrating identity and access management functionality into their big data applications.

Organisations are also starting to take more initiative internally, too. The ‘walled garden’ approach has gained traction, while others are using a similar approach to that seen in web application security, wrapping security into the application and user identity layer

Security and bare metal cloud
But it is in the vendor and supplier market that the biggest security advances have been made. Our big data infrastructure is based on a bare metal cloud and we believe this to be both robust and secure. The hardware is physically isolated with no shared resources, meaning there is no danger of outside interference and security risk. This approach allows businesses to ensure more stringent security but also allows them to scale.

Big data is so rich in potential that the industry will not let security concerns be a significant impact on take-up. The associated vendors are making great strides in helping to reassure the industry big data is secure and this will only continue as it grows more mature.