Data Governance – the key to AI and Machine Learning implementation

Building on Jaime Reid’s recent blog on Data Interoperability – The missing ingredient in exceptional enterprise design and delivery?, Lee Almond our Vice President Chief Data Officer follows on with this blog setting out the data governance fundamentals needed to deliver AI Machine Learning implementations successfully and achieve the key business strategy benefits desired. Look out for her next blog on data minimisation and how to achieve process optimisation and process productivity savings while achieving data compliance.

Artificial Intelligence and Generative AI is such a hot topic because of the potential efficiency and process optimisation opportunities it can offer for organisations. However, this can only work if you have accurate, optimised and well governed data being fed into the AI or any Machine Learning software – otherwise it will only exacerbate the data issues and potentially cause more inefficiency, avoidable cost to organisations and risk of reputational damage. Damage that will be hard for organisations to recover from where customers have high expectations on the pace and accuracy in which their transactions should be dealt with.

Data Governance – how to do it!

I have seen and experienced through my career how organisations jump to a technology solution when it comes to data governance – hoping this will create that magic solution to a number of their data governance issues. This can be curation issues, limited organisation knowledge on the data lifecycle, value of their data and unclear triage/prioritisation approaches to data configuration to limited core compliance obligations built into the handling of their data.

Fundamentally, mature enterprise data governance approaches are built around people, policy, and process. Technology is the final answer to help automate and optimise some of the data governance approaches. If your core ‘people and process’ elements of your data governance are well defined, understood and agreed – this will enable the technology solution of choice to be effectively optimised and bring the intended return on investment.

I have spent much of the last 10 years designing and delivering data governance models for several public sector organisations, operating in complex, highly regulated environments. Organisations who are managing a technology estate utilising a mixture of cloud and legacy platforms and applications. Here are some of my top tips I have learnt over the years:

How mature is your data?

Key to any data governance approach is to start from a position of fundamentally understanding the maturity of your data, the processes around it, and what governance is in place now. This can be easily understood through conducting a data maturity assessment assessing each of the key data governance criteria – subject to the outcome of this assessment determines how centralised or federated the data governance operating model your organisation needs.

People are at the core of any data governance approach

The next key steps are around People – who are the Senior Product Owners or Information Asset Owners for the core organisation datasets. Do they know they are the owners of this data and have key delegated accountabilities from the Head of the Organisation? Do they know what their responsibilities are and any key risks they need to mitigate? How do they manage the key interfaces, transformations, and updates to the data they own? How is this governed in that data’s lifecycle including within one sole or multiple systems (e.g. monitored, updated, which is the master record that all updates are configured around? Is this documented in key data flow documentation etc?). Is this regularly reviewed by the key experts across the product set who handle that data? Who reviews the exceptions where data configurations don’t easily match or the core identity of that data [or customer] cannot be matched, or duplicated data removed? Where is the data stored – in the UK, offshore? This is not exhaustive but key for any organisation to derive the most value from their data and any technology it implements to support this.

Cloud – Data Governance still applies!

An enterprise data governance approach is key to ensure any migration to the cloud is successful and achieves the business and data ambitions originally set out. There are so many benefits to the use of Cloud technology, including more advanced at pace data processing capabilities that enable more on demand interactions with customers, with a joined-up view of their journey with your organisation.

With the rush to the Cloud over the last decade, organisations saw clearly these benefits to their business operating model, solving a number of perennial issues they were experiencing with their products and interactions with customers. However, issues were found in some of these implementations due to not investing in the right data governance arrangements around the use of data in the cloud. I have seen how this leads to multiple copies of the same data being stored in their cloud environment, or data migrated that wasn’t needed and adds no value to the key products and the handling processes around it.

This fast-paced approach comes with extensive storage and processing costs. It also places organisations in a position of compliance risk – this is where data governance can really help to identity duplicate/dormant data that can be removed. This creates cost savings, helps meet their compliance obligations but also from a sustainability perspective - reduces CO2 emissions from keeping the Cloud processing to its most optimum. 

AI – Data Governance stills applies…

In the case of AI, I cannot emphasise how important it is to have a mature, enterprise data governance approach in place before implementing this kind of technology. It doesn’t have to be extensive or onerous in process and it can operate using a fairly agile approach but it is a key component that needs to be in place from the start of any technology delivery…put another way, if AI is implemented without these fundamental steps put in place, the true value of this technology will not be achieved and could cause a number of negative downstream consequences.

Data governance and the right data handling approaches are a key passion of mine. I always say People are the most important asset to any organisation, but second to this is Data – investing in both is key to creating high performing organisations, delivering the core business strategies they set out and driving true innovation.

If you would like to know any more on these topics – do reach out.