What is no longer enough – we need to know why

Depending on who you believe, data is the new gold… or the new oil… or the new currency. Whichever way, it’s what businesses run on, and it’s what businesses have been grappling for decades to harness, manipulate and make sense of.

Business Intelligence (BI) came along in the 1990s and promised a world where anyone could take all the data they needed and cut through to the insights that could consign decision-making by one person to the history books.

Yet, sadly, many people using BI tools – even the modern ones – still find that the intelligence part is lacking. They are trapped in a state of continually trying to gather data efficiently from numerous sources and inputs, into one central place. But today’s tools are simply not up to the challenge of delivering the relevant and impactful insights that are needed immediately, to show what’s actually going on in the business. Eventually you might get there. But who can wait weeks when the board is asking for it yesterday?

And even if you can find out what happened, that’s no longer enough. We need to know why.

But how?

The data explosion means organisations are scrambling for fast, trustworthy, actionable intelligence. Making sense of data from a myriad of sources and spotting unusual behaviour is incredibly tough, with data spread – and constantly moving – across increasingly complex IT infrastructures and a potentially huge number of internet-connected things. Data is created at a pace that’s hard to keep track of, making it nearly impossible for businesses to understand why behavioural abnormalities and issues happen when they do. Yet, at the same time, we’re under more pressure than ever to understand what’s going on in our department or organisation.

Businesses have, of course, long tried to address this challenge: any organisation with more than a handful of employees will have some form of tool they use to attempt to show what’s going on. Enterprises have dozens if not hundreds. Add in all the partners, suppliers and agencies that form the hyperconnected nature of modern business, and the data landscape (and the tools trying to make sense of it) becomes an overwhelmingly complex picture.

So overwhelming in fact, that current business intelligence tools are simply not up to the task – they’re too slow and cumbersome for today’s demands. Too much time is wasted needlessly processing and analysing old or unverified data and creating visualisations that have nothing to say. Businesses cannot afford to continue wading through the labyrinth of data for every ad-hoc request they get, particularly when the initial results need refining. By the time an answer emerges, it’s no longer relevant. Yes, after much wrangling with the data, you can get some powerful insights. And yes, there are products that can produce really nice visualisations to help illustrate the findings. But to find out why things are happening, and allow businesses to act on it immediately, a new approach is required.

Enter AI

Augmented intelligence and machine learning are changing every industry. And few areas are more suitable than data management to exploit the power of machine learning. There’s a huge potential to put machine learning to work: crunching large volumes of data across complex models, finding hidden insights and alerting employees to take action, is tailor-made for algorithms to augment human intelligence.

In business analytics, AI’s principal benefit is to help business leaders understand issues as they arise and, in the future, predict trends from which they will benefit. And, crucially, to do this quickly, so that businesses can immediately act on insights.

But where AI really comes into its own is by adding the why to the what. Because however much data traditional BI tools can analyse, finding the root cause is beyond them. The limit of traditional tools has been reached.

Whether it’s used to get a deeper insight into breakdowns in customer journeys, unearth previously unexplained anomalies in financial reports, or find regional blips in product launches, AI is required to know what’s unusual and automatically flag it, ensuring actionable insights at speed.

The future of analytics

The future of analytics lies in the smart interaction between man and machine. By exploiting machine learning to automatically surface insight, employees can devote time to focusing on making informed decisions that require the intelligence that – for now at least – only humans can bring.

So let’s challenge ourselves not to be happy with what happened. We know we can find that; but so can everyone else. What’s really going to set organisations apart in the hunt for those slivers of competitive advantage, is the ability to find out why.