30/12/2014

By Charles Caldwell, Logi Analytics


Big data isn’t as big as big data hype. Yes, big data is doing some pretty cool things out there. But can big data cure cancer? Will big data destroy privacy? Will big data yield millions in revenues? It is easy to feel that you have to jump into big data now or you’re going to get left behind.

Here’s the rub: big data won’t do any of these things. Highly skilled clinicians and biochemists will cure cancer. Bad privacy policies and poor data security will destroy your privacy. And skilled business people will find ways to capture millions in revenues. Yes, big data will help. But the machines can’t do it alone.

Many large enterprises are trying to ride the big data wave by buying the technologies (focusing first on the storage capabilities, not the analytics capabilities) and taking a ‘let’s store all the data we can until we figure out what it might be good for’ approach. The myth that drives this behaviour is that somehow, magically, big data will sift through all that information and produce gold. The reality that drives this behaviour is that large companies know they don’t know what to do with that data and they can make the ROI numbers work by replacing other storage options, like tape backup.

SMBs don’t have the luxury of building data lakes and hoping. And that’s not a great strategy anyway. It’s analogous to owning a library full of books you never plan to read. Big data is a cool new tool, but it really doesn’t change how you apply analytics to your business. You can certainly use big data as an SMB, but only if you’re highly pragmatic in your approach.

Mind your own business

The big trick to using data to drive business performance comes down to one key factor: understanding your business model. Where are the key leverage points in your business? If you knew more about your customers’ preferences, what could you do about it? If you could monitor a key process, could you optimise it? Is there information you have that you could deliver to a customer or partner that would make your product or service more valuable? Answering these questions will help you identify where big data might generate an insight that you can act on to create value. If you can’t think of an interesting question to ask that could lead to value creation, you don’t need any data, much less big data.

Problem solving on a shoestring

Some of the best analytics projects have kicked-off with no budget. No new technology, no new staff… just get started with what you already have. Once you have identified an interesting question, start trying to answer it with the resources you have on hand. Find the person in the business who knows and cares most about the problem you’re working on. Find the most data-savvy person you can find inside your organisation. Don’t hire a Ph.D. data scientist just yet. Start to work on the problem. See if you can find the data you need to answer the questions. See how far you can get with the analytics tools you already have. Exhaust the first line of analysis before you try to use exotic techniques. Why? Because you’re likely to learn that you can find an answer without big data, that you had the wrong question and needed to think again, or you couldn’t get at the data and needed to implement a business process so you can access it.

Rent to own

If you had the right question but didn’t find the answer on the first round of analysis, now it is time to bring in the big data magic. Before you fill out the CapEx request for hardware, software and implementation costs, think ‘Cloud.’ Services like Amazon’s EC2 enable you to create your own analytic sandboxes, and you pay on a usage basis. You can fire up an environment to do some initial testing and turn it off when you’re done, paying just for the time you used. When your initial experiment is a success, you can scale that environment up to handle the full Big Data load. And, when the project is completed, you just turn it off. Capex becomes OpEx, total cost for your project goes way down and you can use a fail-fast model to test many possibilities before funding an initiative through to completion.

Even if you need help, it is still unlikely to be time to employ a data scientist. Because you defined your question well and have done the initial analysis, you’ll know enough to engage a targeted consultant. Not a big data generalist, but someone who has experience with the specific types of problems you are trying to solve. Options abound from big consulting shops to a graduate or doctoral student whose thesis is in your area of interest. Engage them on the specific problem and move from there.

Business is increasingly becoming a data-driven game. Even marketing departments are increasingly run by data-driven behavioural economist types rather than the creative copywriters of a decade ago. This shift isn’t about any given technology. It is really about a mind-set. Focus on problems that matter, define measurable methods for understanding and managing those problems and use analytics (big or small) to create value by solving those problems.