Unlocking Data, the New “Liquid Gold”, with AI

Co-authored by Andreas Freakley (primary author) and Pawel Krzysztofik

Some call data the new oil due to its relevance in today’s business environments. It fuels any endeavour to generate insights through advanced analytics as well as artificial intelligence and automation. But just as an oil company would not lower an oil rig into the ocean at random, companies should scout their data reservoirs and assess them depending on accessibility, quantity and quality. Taking these factors into account, the effort for tapping into a new data reservoir can be analyzed and considered when building a business case for investing into AI capabilities.

Accessibility can be influenced by many factors. Even with data being digitized, it may not be usable initially. Data could be stored in binary formats such as old office formats or nestled within documents in complex structures with high variation leading to only partially labelled data points. These and other factors can make leveraging a company’s data challenging.

 

Data Driven, Explorative Approach

 

The core idea behind any successful artificial intelligence and automation (AI&A) project is that it’s about business problems worth solving.

AI&A projects rely heavily on the data available but come with the added challenge that there isn’t always a clear view on the data readily available. By default, this gives AI&A projects an explorative nature and requires an iterative and agile approach to successfully generate sustainable value. There is a need to combine both a top down business driven approach with a bottom up data driven analytics to sketch out potential options and assess their validity.

Our building block AI&A framework (see figure below) can match both views and identify multiple potential algorithms to tackle specific challenges.

With this approach, non-value generating scenarios can be identified early in the process and resources allocated to sustainable tracks of exploration. Early design phases aim to build minimum viable products to generate trust within organizations which can then be subsequently scaled to production levels.

 

Unlocking the right data the right way

 

Technology is changing at a rapid pace and the amount of data being produced is growing exponentially. This should be considered carefully when looking for the right data to unlock business insights.

When unlocking data sources there are multiple questions that should be asked and considering the following points will help to define a way forward and ensure critical alignment with other organizational initiatives.

1. Historic data: may provide data for longer timeframes and is already at least physically available. There may be challenges in terms of data quality, such as accuracy or completeness, which could be due to dated methodology. This means that unlocking the data requires advanced machine learning methods, as well as initial supervision by subject matter experts to ensure data quality.
2. Current data: that an organization produces may not be captured yet but could provide an organization with better insights in terms of quality and quantity. Creating new data sources also requires investment but gives an organisation the ability to capture fully digitized, structured data directly at the source, thus negating the need to unlock data with a complex cognitive solution.

AI&A algorithms are highly specific. This means that they are very good at doing one specific task but cannot perform others. The process to unlock an organization’s data can be very individual and is influenced by multiple factors, such as data complexity and structure, volume, source type and the current level of human interaction.

In part 2 of this series, we will explore the various phases of the data unlocking process.

Click here for Part 2

Co-authored by Andreas Freakley (primary author) and Pawel Krzysztofik