The Three V's in Big Data

Harnessing big data for business intelligence is the new catalyst driving enterprise organizations. E-commerce, the IoT, and the increasing digitization of societies in countries around the world have driven this phenomenon. All this data has given rise to data warehouses, data lakes, and data marts as holding tanks for the intelligence we can now capture.

But enterprise organizations are struggling to consolidate legacy platforms under the imperative to both manage and leverage big data. What are the frameworks that should guide the capture, analysis, and insight that will come from the next ten years of big data?

Big Data Architecture—The Three Vs

Gartner has defined big data as encompassing the three Vs—volume, velocity, and variety. What are the definitions and benefits of the Vs and which characteristic is leading the pack as the most important to CIOs in 2017?

Volume

The data capture that follows social media and e-commerce has yielded more transactional information than we can ever hope to mine. Couple that with the Internet of Things (IoT) sensor data and suddenly IT managers are drowning in a sea of information. While you can handle the data lake by increasing online storage, many enterprise organizations haven’t made the move to the cloud yet. Gartner suggests limiting data to business-case only or curbing data capture to a specific percentage of statistically relevant data. Eliminating redundancies can also curb the volume of data that’s weighing you down.

Velocity

Digitization means real-time data, and an increase in point-of-interaction (POI). But this rubs both ways; POI is now measured as an industry differentiator in the form of complaint or help desk response times or service delivery speeds. That’s why managing the velocity of data is just as important as managing the amount of data. This is more than bandwidth; Gartner is talking about changing the architecture to speed up processes. Establish a cache system that allows immediate access to transactions, P2P data integrations between apps and databases, and ODSs (operational data stores) that upgrade production data in real-time are all ways to increase the velocity of your response – while handling the increasing speed of the data that’s traveling in our direction.

Data managers must keep tabs on the volume, velocity, and variety of big data.

Big data
Data managers must keep tabs on the volume, velocity, and variety of big data.

Variety

This is where it gets really messy. Gartner suggests, “No greater barrier to effective data management will exist than the variety of incompatible data formats, non-aligned data structures, and inconsistent data semantics.” Handling data variety is the third great challenge faced by data managers and it will only continue as everything from e-commerce automation to M&A affect the variety of data flow. Certainly, advanced indexing can help incompatible data types, and management of metadata can help provide context. But query management software as an overlay can help sort and integrate “dumb” data. Data access middleware vendors can help connect apps and databases.

The NVP Big Data Executive Survey of 2016 suggested that of the three Vs the biggest challenge remains integration and usage of the variety of big data. In fact, concerns over volume and velocity are placing a distant second and third in the minds of data architects.

Learn more about us and how Bigstep can extend a life preserver to data managers still drowning in the three Vs.