Low latency and big data

Low latency was once only really spoken about within financial trading firms. But in 2013 low latency networks are desired and used by businesses of all types. Anyone in fact, who is demanding fast, predicable and deterministic response time as a business priority.

The good news is that latency of 5-10 microseconds can be achieved…for our customers anyway. But why is low latency so important and how is it best achieved? 

Linking latency and big data
Financial firms still probably have the greatest need for ultra low latency, but low latency is now important for many businesses, irrespective of market. Organisations are increasingly moving large volumes of data across their network and doing so quickly and efficiently is critical.

Big data and low latency are heavily linked. The potential value of big data is immense but it does depend on being able to analyse and derive insight from it in real-time. We’ve written before about our high-performance cloud being the perfect infrastructure for big data - a key element of that is achieving and maintaining low latency across the network.

Achieving low latency
Of course, there is not a one-size-fits-all approach that an organisation can follow to achieve low latency. The impact and importance of latency depends on the specific application and achieving the lowest possible latency does require a trade-off between other network characteristics.

Having identifed the importance of low latency in our big data infrastructure that offers bare metal power for speed and performance, we made it an integral part of our offering. Because we have removed the hypervisor, all switching is done on physical networking equipment. This immediately improves networking latency dramatically, as well as frees up processing resources in the physical servers. An additional benefit is providing a customer with direct control over the physical ports in the switch and facilitating port-to-port networking.

Ultra-fast pseudo-wire connectivity
But in terms of achieving low latency it is our ultra-fast pseudo-wire connectivity that really makes a difference. Using pseudo-wire networking, machines communicate over the internet as if they were connected in the same switch and this gives our customers a cross-country, cross-datacenter latency of 5-10 microseconds.

Such speeds are barely believable and when realised in conjunciton with the bare metal computing power and I/O all-SSD storage (a standard feature for all dedicated machines) it is clear why companies such as Cloudera were one of a number of partners involved in the benchmarking and testing of Bigstep.

Reducing network latency adds value right across the enterprise, from improved application performance to faster transaction times and including the moving terrabytes of data around a network. Any organisation that is serious about big data must also be serious about achieving low latency.