A bare metal cloud is Hadoop’s best friend

Hadoop is power-hungry – we all know that, right? It needs immense computing power to work effectively and for all the insight it can deliver, there is a risk it will drain an organisation’s resources too much.

So we were interested to see the recent launch of Altiscale, a new Hadoop-as-a-Service (HaaS) that makes Hadoop easier for companies that don’t have the time, knowledge or resource to run it without external assistance. A noble idea, but we feel that bare metal - and our bare metal cloud in particular - is Hadoop’s best friend and here is why. 

The rise and rise of Hadoop
Hadoop is without doubt one of the rising stars of technology. As organisations seek to better understand and get value from their big data, Hadoop’s powerful analytics capability has seen it become the number one tool for crunching big data.

But there is an elephant in the room. Hadoop is power-hungry and any organisation running Hadoop (or indeed any similar big data applications) in a virtual environment is going to see performance suffer.

This is something that vendors in the virtualisation space are understandably keen to play down. But the fact is, hypervisors are a major drain on performance. This is why our big data infrastructure comes without a hypervisor, allowing users to benefit from the full performance and power of bare metal.

That’s not just us saying that either. There have been a number of industry benchmarks that have said the same, from IBM to Redis. Nati Shalom of GigaSpaces has written that ‘big data on a virtualised infrastructure would require 3x more resources than its Baremetal equivalent’.

Can Hadoop-as-a-Service work?
Even if Nati Shalom’s assessment is even half right, there is a major differential in the resources required to process big data in a virtualised cloud environment versus bare metal. So the recent launch of Altiscale and other organisations such as Xplenty have the very laudable aim of providing Hadoop as a Service to make big data available for companies of all sizes.

But big data already is within reach for even the smallest firms! Our bare metal cloud is available to anyone on a pay-per-hour basis, so even early stage start-ups can benefit from cost effective big data processing, without the need to compromise performance in a virtualised environment.

Big data – a big processing challenge
At the recent Hadoop Summit in Amsterdam, Forrester principal analyst Mike Gualtieri argued that even 800MB of data brings with it a lot of information to be analysed, declaring ‘it’s a big processing challenge, it’s a big compute challenge’.

Gualtieri also mentioned research that showed 45 per cent of big companies say they’re doing a Hadoop proof of concept, with 16 per cent already using it in production. Relatively small numbers, but as organisations increasingly invest time and resource into unlocking the potential of their big data, they are realising that Hadoop needs a powerful infrastructure to truly thrive.

So despite the admirable attempts by organisations to show that Hadoop can be deployed without impacting performance, we believe that bare metal is Hadoop’s best friend and only with a bare metal cloud can Hadoop be effective and not be a drain on computing power.