Why Big Data will Always Need Gut Instincts Too

Big data is hailed as the next frontier of marketing. Some predict that in the not-so-distant future, big data will virtually drive business and marketing efforts, replacing the need for human managers, marketing professionals, and the like. But is this reality? Can big data do what P.T. Barnum and Steve Jobs did—revolutionize the business world with brilliant ideas and innovations?

No. Big data is powerful, indeed, but is no match for humans when it comes to one important factor: intuition. Consider Steve Jobs. He ignored market research completely, and ended up producing some of the most remarkable and world-altering products ever released. Could big data have predicted the enormous popularity of the pet rock or hula hoop? Likely not. So, where do human intuition and big data meet to mold the future?

Big Data Must Be Supplemented With Expert Knowledge

A human with experience in the field has intuition that a computer could never mimic.

One big data analyst was challenged to predict which buildings in a particular city would be most likely to succumb to fire. Data was entered, numbers were crunched, predictions were made. The data was then viewed by a group of experienced building inspectors, who immediately noticed flaws in the predictions based on things the data didn’t hold—like the look of a particular building’s bricks. This intuition can’t be programmed. It only comes from years and years of experience by a thinking human with gut instincts.

Human Input is Critical for Effective Big Data Analysis

Machines are excellent at crunching the numbers, but are terrible at determining what numbers need to be crunched. For example, a computer can make excellent predictive analysis on where to build a new facility or how to prevent shoplifting, but only if humans are smart about giving the computer access to the right information. Computers can never guess that adding additional security cameras in the parking lot or entering information on home sales in a particular area could improve analytics on building new locations or helping prevent theft. Only humans can determine what data and what sources of data need to be included in the analysis. What isn’t in the data is often more important than what is.

All Relevant Sources Need to Have Input in What Data is Included

Companies that are best able to leverage big data and predictive analysis do two critical things: blend the human and machine elements of big data analyisis and use all available sources for input. The companies making the best use of big data gather information and advice from production workers, suppliers, business partners, IT professionals, executives, and analytics specialists and use all of this information to develop sound strategies for using big data. Production workers and suppliers have knowledge about company operations that executives do not. Likewise, IT and analytics specialists can see the limitations of machines that are difficult for the executives to understand. It takes all the puzzle pieces to make a complete picture.

Big Data Has Limits, But So Does Intuition

Machine Analysis + Human Intuition = Better Outcome

Clearly, machines can’t do what’s needed without human assistance. Likewise, human intuition has limitations that are well balanced by big data. For instance, even marketing genius Steve Jobs sometimes released a big, fat flop. Having access to big data can often keep intuition in check, preventing some of the biggest marketing disasters. When people are there to guide the analysis, and big data can temper the worst of human intuition, the two can work in concert for a better overall result. In fact, one study of 600 American and British companies concluded that 75 percent of companies able to blend intuition with big data analysis were more effective than their counterparts that depended on predictive analysis alone.

The future of big data is never far separated from the human intuition that always has and always will be the driving force of innovation.

Sources

http://www.forbes.com/sites/gregsatell/2014/10/12/the-future-of-marketing-combines-big-data-with-human-intuition/

http://blogs.wsj.com/cio/2014/02/06/does-intuition-matter-in-a-big-data-world/

http://www.managementexchange.com/blog/big-data-big-intuition

http://www.ericsson.com/thinkingahead/the-networked-society-blog/2014/05/30/big-data-needs-human-touch/

http://www.wired.com/2013/09/putting-big-data-in-context/

http://www.nytimes.com/2012/12/30/technology/big-data-is-great-but-dont-forget-intuition.html?pagewanted=all&_r=0

http://www.informationweek.com/big-data/big-data-analytics/big-data-debate-do-analytics-trump-intuition/d/d-id/1269193

http://businessoverbroadway.com/improving-the-value-of-customer-experience-analytics