Recapping the Year of Transition for Hadoop & NoSQL

In 2015, Hadoop and NoSQL gained mainstream adoption and big data moved from buzzword to real investment. Both platforms grew and matured, with numerous updates—both of the critical nature and the nice-to-have category. What were the most prominent changes and improvements to Hadoop and NoSQL for 2015?

A Whole New Set of Database Options

Relational database options had grown limited. As NoSQL hit the scene, an entire new set of options have become available, including a brand new core engine for MongoDB. This product offers a wide range of plug-ins and much more flexibility. Named WiredTiger, this engine is designed to address the locking issues that have plagued the old MongoDB. Additionally, DataStax, which provides a commercial version of the popular Cassandra database, is teaming up with the open source community at Cassandra to produce a new updated version this year.

Though NoSQL is gaining significant traction in the marketplace, there are some times when a good old SQL relational database just works perfectly. To answer that cross-need of SQL features to enhance NoSQL databases, Couchbase introduced the N1QL query language tools. (To avoid confusion, N1QL is pronounced “nickel”.) These tools provide an SQL-ish environment in which to work with Couchbase’s NoSQL database management system.

Bridging the Skills Gap Between SQL and NoSQL

Lots of the popular Hadoop query engines got facelifts in 2015. Hive, Impala, and Presto mark the advancement of adapting SQL for Hadoop. These tools are powering Hadoop-based data lakes and are keeping data lakes from devolving into useless swamps where good data goes to die. However, there is some room for improvement. Enterprises are still staffed with IT workers who possess strong SQL skills, and companies need to leverage those skills even in the era of Hadoop. Expect new products and product improvements to continue to bridge the gap between SQL and NoSQL skill sets.

Advancements in Hadoop Core Technologies

Hadoop grew a lot in 2015 in terms of technologies. Cloudera introduced the Kudu columnar data store, which is designed to work with Impala. It powers real-time analytics applications and is capable of working with super fast data inserts and updates. In different IT environments, Kudu can either serve as a companion to or a substitute for Hadoop Distributed File System.

Spark Takes Center Stage

Perhaps no big data technology made more of a, well, spark, in 2015 than Spark. Spark is faster, more powerful, and much easier to work with than MapReduce when it comes to batch jobs. Spark supports real-time analytics and makes a sweet companion for Hadoop in environments where streaming and real-time analytics is a core function. MapReduce hasn’t been entirely replaced, however, as it still continues to gain momentum in traditional data warehousing and load applications.

Is your organization ready to take on big data, Hadoop, NoSQL, and analytics? Take advantage of this special limited offer! Discover the first Full Metal Data Lake as a Service in the world. Get 1TB free for life - limited to 100 applicants. Start here.