“Have Data, Will Predict (Something)”

The buzz-word instigated race to make money from data is creating 20140526-135108-49868207.jpg advanced analytical models that can’t be used in business while simple business problems with simple data solutions are left unattended. This is because the business problem has not been defined first.

At one end are organizations that have plenty of customer and transactional data that they have not even looked at yet, and at the other end are organizations that have plucked all the low-hanging insights from their data and are now ramping up advanced predictive analytics to squeeze out deeper meaning.

We find a depressingly larger number of organizations at the lower end of this maturity curve, and a very few that are pushing the frontier at the top end.

Kaiser Fung is mildly irritated when he asks, “Why websites still can’t predict exactly what you want?” The personalization he wants is simple but adequate, and just needs the organizations to use existing data about him. This is not happening because

…… these kinds of easy wins aren’t sexy enough for data scientists. And maybe they fear their effort would go unnoticed if we can get better personalization without teams of PhD’s spending three years to create hundreds of algorithms.

This is slightly unfair to the data scientists, who are after all supposed to be doing what the marketing and sales honchos are asking them to do. Even Kaiser admits that companies focus on the upsell instead of matching current users to current needs. So if the business wants the sexy algorithms the data scientists polish up their crystal balls and start gazing away.

Then there are some hapless data scientists who are being asked to ‘predict something’ so that we can make more money. This set does end up getting too deep into the analytical models that do seem unrelated to customers and indecipherable to business. Far too often this situation is a result of not thinking through the business context.

We need to ask, “What business decisions can make use of this additional insight obtained through applying advanced analytical techniques to a data set?” These questions are formally asked and answered through a Decision Model that ensures that business, customer and the data scientist are all on the same page. The Analytics will now be created and used in a business context for better decisions and consequent better outcomes.

Simple tools for simple problems and sophisticated tools for complex problems. Of course. But define the problem first.