“Have Data, Will Predict (Something)”

The buzz-word instigated race to make money from data is creating 20140526-135108-49868207.jpg advanced analytical models that can’t be used in business while simple business problems with simple data solutions are left unattended. This is because the business problem has not been defined first.

At one end are organizations that have plenty of customer and transactional data that they have not even looked at yet, and at the other end are organizations that have plucked all the low-hanging insights from their data and are now ramping up advanced predictive analytics to squeeze out deeper meaning.

We find a depressingly larger number of organizations at the lower end of this maturity curve, and a very few that are pushing the frontier at the top end.

Kaiser Fung is mildly irritated when he asks, “Why websites still can’t predict exactly what you want?” The personalization he wants is simple but adequate, and just needs the organizations to use existing data about him. This is not happening because

…… these kinds of easy wins aren’t sexy enough for data scientists. And maybe they fear their effort would go unnoticed if we can get better personalization without teams of PhD’s spending three years to create hundreds of algorithms.

This is slightly unfair to the data scientists, who are after all supposed to be doing what the marketing and sales honchos are asking them to do. Even Kaiser admits that companies focus on the upsell instead of matching current users to current needs. So if the business wants the sexy algorithms the data scientists polish up their crystal balls and start gazing away.

Then there are some hapless data scientists who are being asked to ‘predict something’ so that we can make more money. This set does end up getting too deep into the analytical models that do seem unrelated to customers and indecipherable to business. Far too often this situation is a result of not thinking through the business context.

We need to ask, “What business decisions can make use of this additional insight obtained through applying advanced analytical techniques to a data set?” These questions are formally asked and answered through a Decision Model that ensures that business, customer and the data scientist are all on the same page. The Analytics will now be created and used in a business context for better decisions and consequent better outcomes.

Simple tools for simple problems and sophisticated tools for complex problems. Of course. But define the problem first.

ManMachine – The Computer Human Synergy for Organizations

TARGIT CEO, @MortonSandlykke has a very topical post today, ‘Turning Emotion-based Decisions into Fact-based Decisions’.

Morton’s basic premise is that while information availability might have been a bottleneck for better decision-making in the past, that is not the case today.

SMIeyeTracking On Flickr

(c) SmiEyeTracking Flickr

Today the human is the bottleneck in the decision-making process.

We have been programmed to act on our own biases – built over time through our experiences and our learning paradigms; and colored by our own contexts, cultures and objectives. So, now when we are at the other end of the spectrum – drowning in information – our instincts kick in and we resort to ‘gut-reaction’ decision-making.

Business Intelligence is worth nothing if you don’t change your behavior……..Instead of letting the computer drown you in data, trust it to lead you to a conclusion.

Most of us recognize this truism and may truly believe that we are, in fact, making data-based (or, evidence-based) decisions. This is true in some cases at an individual level but extremely rare at an organizational level. Organizations are still largely structured for gut-based ‘leadership’ and ‘executive’ decisions. There is very little scope for learning from patterns in the data or for running experiments to choose the best strategy.

Q: Why can’t organizations step up to fact-based decision-making?

A: Because they cannot describe the organizational decision-making process.

While most organizations have systematically described their processes, structures and data-stores, the decision making is still in the managers’ heads. Good managers make good decisions and if there are no managers, decisions don’t get made. This is a serious handicap in being scalable and being consistent in business.

Decision modeling using the new DMN Standard and the DecisionsFirst tool are good starting points to start describing decisions.

For Man to truly trust the Machine the decision-making responsibilities between the two have to be explicitly described first and judiciously partitioned next.

 

A (Data) Revolution bogged down in Organizing Busywork

Bentley_BoggedDown

via CarScoops.com, 06/10

Sure, we need to organize our world to make sense out of it and to manage it.

We started with standardizing and labeling physical goods – everything that came out of the first factories in industrial Britain. When that started paying for itself in efficiencies, Frederick Taylor decided to take a closer look at the things we do – the processes in an enterprise. The famous (or infamous) Time and Motion studies started the business process improvement movement leading to the modern Business Process Management with other associated streams of quality management.

Now that processes have been organized and (mostly) automated, the search is on for the next quantum of improvement -the next revolution. After dealing with and settling the management of physical goods and the physical actions, what is left to improve?

People? Yes, social media has empowered them but creativity and innovation have a difficult time dodging the giant wheels of automated processes and do get crushed far too often. In any case, human resource management and Catbert have been trying to ‘improve’ people for a while now. And we would rather not talk about all that stuff for now.

After physical goods, physical actions – and the people – we are left with the virtual stuff, the data and information that we have been collecting over the years as our processes spewed them out. The byproduct of automation is data. Lots and lots of it. Not surprisingly, data is being seen as the next frontier for competitive advantage.

And being the good little organizers that we are, we have set out to organize all our data with a vengeance. Shiny, new dashboards with pretty colored graphs and scientifically researched user interfaces; brand new ‘big data’ infrastructure that can keep data ‘in memory’ and in the ‘cloud’; and quite a few of us are mining this data now to get to that rare insight that the silly fallible humans miss.

All of this data (or Big Data) activity seems to be useful – and in some cases it does move the ball a few yards forward. But the promise of revolution is still out there – a promise. There is no doubt that better insights are being served up to the human decision makers and even if there is no evidence that it is resulting in better business decisions, let us stipulate that decisions are indeed getting better. This still leaves open the question of scaling the human decision making to an industrial level. Remember the silly humans are squishy and unpredictable. So, data is being organized but not being used.

The missing element to unite them all is Decision Management. This is a formal mechanism to automate operational decisions using (big and small) data analytics and (human) subject-matter defined business rules. You use data and people to automate decisions required by your automated processes. Automated decisions embedded in your operations (processes) enable you to scale tremendously to attain the promise of the data revolution.

Decisions-First thinking forces you to organize decisions formally. Data organization follows naturally, now with an actionable purpose behind it. All revolutions need a purpose and now the data revolution has one.

Happy revolutioning!

 

Taming the Machine (Learning)

The one thing everyone agreed on was the need for Decision Velocity. manmachine.jpg

Increasing it, that is. Make decisions faster.

Chris Taylor says that there is simply too much data out there for humans to use effectively. So bring in the machines. Machine Learning is where raw computing power is let loose on stacks of data in order to find useful patterns there.

Fair point. The machines are coming.

Opher Etzion points out the issues with relying on Machine Learning alone – cannot be done in real-time; cannot scale since every use case needs its own ‘learning'; and, finally it needs to be supplemented by humans who can provide a future vision that machines that learn from past data cannot.

Sound assessment of the current situation. No doubt the technology to support real time machine learning will improve – using Event Processing infrastructure as pointed out by Opher. Better tools will emerge that allow generalized learning across multiple use cases – promoting reusability and thereby providing scalability.

The open question is how to get humans to participate in machine learning.

This is where Decision Management Technology comes in. It includes Advanced Analytics like Machine Learning as well as Business Rules Management Systems (BRMS). The latter is where Human Expertise can be explicitly stated and managed. So, ideally we need to create a Decision Service black box that holds the Machine Learning model surrounded by Human Expertise expressed as business rules. And viola, this Decision Service can now automate most operational decisions – and you have increased Decision Velocity. Building Decision Management systems does require explicit decision modeling and the need to start there first.

Let us tame the Machine by giving it business rule Prime Directives.

Why won’t Dilbert Retweet?

One of the abiding tenets of ‘Social Business’ has been the use of employees to spread the company’s marketing message. This is the problem of employee advocacy where the first order thinking is the following.

Dilbert

…. if your company prospers [the employees] will also, so it’s to their benefit to support your brand. The result is that if employees retweet or share content published by their marketing department, they could save their company thousands, or hundreds of thousands of dollars in advertising costs.

From a pure business perspective this is perfectly rational thinking – optimizing scarce resources in support of business goals. The missing element is the inherently different mechanics of social interaction – in person or via digital media. First, this is not a one-way broadcast and a reasonable opportunity for a dialog is assumed. Most Dilbertesque organizations have strict controls over who can engage on behalf of the company and who cannot. Second, a public retweet or ‘like’ is a personal endorsement. Employees have the inside view on how things are put together within an organization – and they are likely to assume that their product is not good enough. Trusting their own organization comes first. Public endorsements, next.

Getting employees to engage in social business is more than making it easier for them to ‘share’ approved marketing content. Successful social businesses will have to rethink organization culture and organization. It is a tightrope walk between too much flexibility and too little. Still, Enterprise 2.0 does not add window dressing on the periphery of industrial-age assembly lines but organizes itself as a knowledge network that it really is.

Dilbert needs open, social interaction within his company before he retweets some of it.