Technically Speaking

The Official Bigstep Blog

How Big Data is Delivering Operational Intelligence in Real Time

Big data is often associated with batch processing—analytics that takes a lot of time. But increasingly, new data processing and analytical tools are empowering real-time analysis. There are many potential uses for real-time analytics that simply could never be done with traditional batch processing. Here are the potential uses for real-time analytics, along with the tools you need to get it done.

The Uses for Real-Time Data Analytics

Processing online transactional data is just one use for real-time analytics. Every industry has found ways to put it to good use, including improvements in manufacturing operational intelligence and improving the care of patients in the healthcare industry.

Perhaps the most visible and easy to understand use for real-time analytics is in processing online transactional data and engaging customers. Companies have realized that the ability to strike while the proverbial iron is hot—in other words, when customers are actively shopping—they can tremendously boost sales and customer satisfaction. This means delivering popup ads, emails and onsite offerings that are relevant to an immediate search.

It also means analyzing data on the fly to determine when fraud is being committed. Real-time data can be analyzed against a customer’s normal shopping habits to determine when out of the ordinary activity on their account could indicate fraud or identity theft. This practice is not just useful for e-commerce, but is revolutionizing the banking and credit industries.

But real-time analytics has far more potential than e-commerce. It is a powerful tool for delivering operational intelligence to the manufacturing industry. Big data can be processed on the spot to identify bottlenecks in manufacturing processes, detect and prevent quality problems, help schedule more effective production, and even keep workers safer around the machinery and equipment.

Real-time analytics is also used in the tech industry to detect a potential network breach. Like in the finance industry, it is done by establishing a baseline for network and systems activities, and then detecting anomalies in that activity in real-time so that intruders can be caught and stopped before damage is done.

Cities like Chicago are using real-time analytics to develop better response plans to emergencies. Hospitals use real-time analysis to determine the best course of treatment for a given patient, and even to identify babies or post-surgery patients that are most at-risk of developing complications like life-threatening infections. The list truly does go on.

The Tools for Real-Time Data Analytics

[caption id=”” align=“aligncenter” width=“403”] Most of the standby big data tools, like Hadoop, are being updated with real-time processing capabilities. Other new tools are also emerging, including Splunk, Spark, Storm, and more.[/caption]

Hadoop was notoriously slow from the starting line, but as Apache Spark and Storm began making huge strides in real-time processing, Hadoop was forced to step up its game. While Hadoop is boasting ‘near real-time’ processing and not exactly actual real-time processing, it is more than capable of performing many of the tasks listed above.

Splunk is another tool capable of real-time processing, and the latest version, Splunk Enterprise 6.4, packs true real-time capabilities for ultra-fast processing tasks line online transactions and fraud detection, where latency isn’t tolerated at all. As mentioned, Apache Spark and Storm are also delivering real-time analytics, and each has pros and cons based on the specific use case in question.

You Can Have Your Cake and Eat It Too

Fortunately, gaining real-time analytical capabilities doesn’t mean you have to sacrifice your current big data infrastructure, or that you even have to build an infrastructure in-house at all. The Full Metal Data Lake is here. Discover the first Full Metal Data Lake as a Service in the world. Get 1TB free for life - limited to 100 applicants. Start here.

Got a question? Need advice? We're just one click away.
Sharing is caring:TwitterFacebookGoogle+PinterestEmail

Readers also enjoyed:

Cloud-busting – is there really only one true cloud?

The terms ‘cloud’ and ‘cloud computing’ and their exact definitions generate more heated debate in the technology sector than perhaps any other terms.When…

Best Books to Put on Your Big Data Reading List for 2015

Last year, the rage in big data literature included Big Data: A Revolution That Will Transform How We Live, Work, and Think by Kenneth Cukier and Viktor…

Leave a Reply

Your email address will not be published.