Hadoop Then & Now: Doug Cutting's Take on the Past, the Present, and the Future of This Game-Changing Technology

Not only is Doug Cutting the brains behind the most successful big data platform, Hadoop, he is also one of the reasons why the open source community was finally able to push its way into the enterprise, alongside the proprietary software giants who dominated those arenas for decades.

Now on board with Cloudera, Cutting’s heart is still well rooted in his brainchild Hadoop. Named for a yellow toy elephant that lives in his sock drawer, the name Hadoop was actually created by his son.

Cutting wanted a name that didn’t mean anything, so that the product wouldn’t be limited by its name as it evolved. He actually held onto the name idea for years before the opportunity presented itself. Aside from the unique (and undoubtedly cute) name, what else does Cutting have to say about his open source prodigy?

Why Hadoop Took Off as Well as It Has

Not long ago, Cutting was interviewed by the enterprise big data news source Data Informed. One of the questions the interviewer asked was why Hadoop has been so successful. In fact, it’s practically synonymous with big data itself. Cutting believes that its success is mostly due to its open source nature. The platform made it easy and inexpensive to adopt, therefore widely used. Plus, every user had the ability to contribute to fixing its faults and improving it. Success, therefore begot success.

The Most Surprising Thing About Hadoop’s Success

What Cutting told Data Informed was most surprising to him was the level of trust Hadoop and open source was able to gain rather quickly within enterprises that had traditionally only dealt with the top software proprietors. Hadoop was perhaps the first major example of enterprises being willing to adopt open source and embrace it for use with their mission-critical applications. According to Cutting, Hadoop is still in its adolescent phase. It isn’t so young that it is immature, but it’s certainly not in the legacy stage, either.

What are Some of the Strongest Use Cases for Hadoop?

According to Cutting, Hadoop has made a huge impact in banks, where it is used for risk detection. It’s also doing a lot in manufacturing, where it can provide lots of valuable information on processes in real time. Where it is poised for additional growth is in the area of delivering information from the field, such as data on equipment maintenance. There are still a lot of relatively untapped use cases for Hadoop.

What’s Next for Hadoop?

Cutting admits that even for industry giants like himself, it’s always hard to tell what will take off in software. One thing Hadoop really needs is improvements in terms of security, ease of use, and its ability to integrate seamlessly with existing tools within the enterprise. For example, there needs to be a way to set data security policies and have those policies be consistent as data moves from system to system.

What Hadoop Needs to Evolve

Another thing Cutting believes will be important to the evolution of Hadoop is a vendor-neutral cloud. Where lots of progress is already being seen is in the hardware. There are better solutions now than ever for high-performance, super-fast storage. This will help drive data analytics, particularly in the realm of real-time analytics.

Are you ready to get started with your own use cases for Hadoop? You can see our products at Bigstep and find what you need today!