There are plenty of opportunities to use generative artificial intelligence (GenAI) to boost business productivity. But in regulated sectors like financial services, the output from the AI needs to be explainable and error-free. Lloyds Banking Group has been working with Aveni, to help it develop FinLLM, an AI language model specifically tuned for financial services. The tech startup, which has been supported by Edinburgh Innovations, the University of Edinburgh’s commercialisation service, has developed a language model for the financial services sector.

Instead of training models on internet-scale text with no regard for context, FinLLM is tuned specifically for the language, regulation, and products of financial services. That means when it analyses affordability, reviews a call for conduct, or drafts a suitability letter, it draws on a model designed to interpret the nuances of financial services, rather than offer a best guess.

In this podcast, Computer Weekly looks into how Lloyds Banking Group has been using GenAI and specialist AI models like FinLLM to broaden access to financial advisors in a way that scales.

The bank has been testing FinLLM in its audit team where an audit chatbot virtual assistant developed by the group audit & conduct investigations (GA&CI) at Lloyds Banking Group is  transforming how auditors access and interact with audit intelligence. The chatbot integrates generative AI with the group’s internal documentation system, Atlas, making information retrieval faster, smarter, and more intuitive.

Ranil Boteju, chief data and analytics officer at Lloyds Banking Group, says the bank effectively trained the chatbot using FinLLM and its knowledge of audits, based on all the audit data it has collected.

Generative AI can get things wrong, which is clearly unacceptable in any business. Recognising that many of the large language AI models provided by the likes of OpenAI and Google tend to be generalists, the bank has worked with Aveni on how to combine the breadth of knowledge of generative AI models trained on public internet data with the highly nuanced financial services information the bank has access to.

The project has been used to show how GenAI and specialist AI models like FinLLM can be deployed to support key banking processes. The way it has been engineered shows how the bank could apply GenAI to power customer-facing chatbots.

To handle a conversation a customer may have with the bank, the chatbot needs to understand the conversation, translating its understanding of the customer query into a series of orchestrated tasks. This is what agentic AI looks like – each of the tasks is passed onto a specialist AI system to handle.

Joseph Twigg, CEO at Aveni says: “The emergence of AI agents undertaking direct customer interactions at scale is  inevitable. That requires a new class of assurance to govern their behaviour, ensuring AI models provide the right advice and responses.”

Along with orchestrating specialist AI models, there also needs to be a way to ensure that the responses delivered by the various AI systems involved make sense and are correct.

Boteju describes the approach Lloyds Banking Group has taken to reduce errors as “agent as a judge”. “You may have a specific model or agent that comes up with a specific outcome,” he says, “then we’ll develop different models and different agents that review those outcomes and effectively score them.” The bank has been working closely with Aveni to develop the approach of using AI agents as judges to assess the output of other AI models.

Each outcome is independently assessed by a set of different models. The review of the outputs from the AI models enables Lloyds Banking Group to ensure they are aligned with regulator, Financial Conduct Authority (FCA) guidelines and check if they are also aligned with the bank’s internal regulations.

As Boteju points out, checking the outputs of AI models is a really good way to double check that the customer is not being given bad advice. He says: “We’re in the process of refining these guardrails and these. It’s imperative that we have [this process] in place.”