It’s hard to be successful with Big Data. I have seen numerous implementations of Big Data that either eternally remain in the early stages of production or do not produce the results they were supposed to. It’s become so widespread that there are three key questions commonly missed in these instances which, if not answered in the beginning, usually result in the project being a significant waste of time and funds.
With that said, it is a matter of asking the appropriate questions, getting the answers that correlate to those questions, and making the most of those responses. But there are the right questions to ask of your data, and then there are the right questions to ask of your overall investment in your data platform. Perhaps the most important questions of all are asked of your platform because they set the stage for everything else that follows.
The first question you should answer is: Why?
In many cases, you will inevitably reach the point in the initial briefing you have with a consultant or a customer where the consultant asks some variation of the same question: so what exactly are trying to do here? Are you trying to reverse a decline in sales? Are you attempting to get on top of increasing customer criticisms and displeasure? Do you want to create an innovative market or found a new channel and require an indication of interest and how it may perform in the future? If you look ahead a few years, what would this project have told you that you did not already know, and how would you have used that information to get to the desired outcome?
I’ve seen countless Big Data projects exist soley because the Big Data buzzword is exciting, new and all the rage. Executives across the board get on the Big Data bandwagon and begin approving massive investments of time and money to get up on a data platform. In many cases, this whole strategy is founded on the weak justification of “everyone else is doing it.” The thing is, data is unique to every business as are the individuals behind it. A wholistic evaluation of what you want to acheive through having access to this data, as well as a survey and record of the expertise and investments your organization needs to make, is required but far too often overlooked when authorizing a Big Data investment. This is problematic, to say the least.
Following this, all great questions asked in the world will not amount to anything without a solid, established timeframe for answering them. Asking “when” is a key moment, too.
Big Data projects are not immune to the hang ups associated with large corporate projects, such as scope creep, approvals, misstated or uncommunicated goals, absence of leadership and stakeholder buy in. Even once you’ve surpassed those hurdles, confirming concrete timelines for getting answers to the questions you have set out in advance is the only way to succeed. Without this information, you will have business analysts and data scientists performing queries ad infinitum, searching for a tidbit of wisdom in an endless sea of data.
Lastly, you need to ask: “Who?”
In essence, you have to make sure you have the appropriate people onboard. As hard as it is to admit, having individuals with business experience, as well as mathematical and statistics expertise, is a necessity for a successful Big Data program. Without these positions, you are bound to get interesting statistical analyses that hold little to no bearing on your real world business, including the marketing activities that go along with it. Data scientists and business analysts who can work together to tease the answers to your main questions out of the data are the key to success. This way you are able to decifer between what might be an actionable insight and what is merely a phenomenon with no clear action point.
Fair warning – without the answers to these three key questions, your Big Data project is condemned to mediocrity or failure. Learn them, know when to ask them and you’ll have a successful Big Data project underway.
Note: A similar version of this blog post appeared on DataSift’s blog at blog.datasift.com in Nov. 2015.