When machine learning goes bad: the threat of AI-driven fraud

Posted 30 May 2023 by Tom Redmayne

In January 2022, I wrote about my vision for the future of advice and how technology can be leveraged to assist financial advisers to become more efficient and enhance the human aspect of the job.

With the recent explosion of Artificial Intelligence and free access to chatbots such as Chat GPT, my Twitter feed has been inundated with top tips on how to leverage AI in your life, profession, or business.

I, like many others, have been swept up with the possibilities of this emerging technology.

AI is well positioned to replace the jobs that do not benefit from direct human input, such as automation of manual processes, data entry, document/form creation, workflow optimisation, virtual assistants, calendar management and appointment scheduling.

I also remain hopeful that one day we’ll be able to get an AI to sit on hold to product providers for hours on end – an even better outcome would be that AI will get rid of the archaic information request systems all together.

The above are positive examples of how AI can be employed to streamline business processes, which could result in a slicker, more cost-effective service offering.

What I hadn’t considered, until recently, are the negative consequences AI could have on the industry and our clients.

The dangers

AI isn’t currently more intelligent than us, but it may be soon, so what happens when we invite something more intelligent than humans into our lives?

Some believe it’s akin to our ancestral cousins inviting homo sapiens to join them at their campfire - before long we’d wiped them all out, leaving us as the sole human species roaming the earth.

Now, I must admit, I’m not currently concerned about the machines killing us off. I’m not sleeping with one eye open, waiting for Chat GPT to garrotte me with an ethernet cable; however, I am concerned about what happens when you take existing financial scams and insert an AI that is smarter than humans into the mix.

AI is learning new skills at a fascinating rate. One of the skills, which I find particularly concerning, is the art of persuasion. Soon we may have AI that is not only more intelligent than us, but also a master manipulator. Combine this new skill with deep fakes, in which AI can now perfectly mimic a voice with less than 60 seconds of input, and you have a recipe for the perfect financial scam.

Fraudsters are already deploying AI copying a person’s voice that holds a position of trust with their target, whether that be a loved one, an accountant or financial adviser, to extract information or money from them – this is known as ‘spear phishing’. Soon we may not be able to trust who we are speaking to.

Criminals are about to receive an uncontrollably powerful tool and there isn’t much we can do about it. If the rate of progress keeps pace, then advanced AI scams will widen the net of who we deem ‘vulnerable’.

With the advent of this new threat, which will take many forms, there will need to be a conversation within the financial services industry about how best to protect our clients and their hard-earned money.

For me, the first line of defence is education

It will become even more important to educate clients as to what sort of contact they should expect from their adviser and general threats to be aware of. Clear guidelines for engagement will help to maintain trust within the relationship, while also encouraging a healthy dose of scepticism.

I’ve been ‘chatting’ to Chat GPT about the dangers of AI within our industry and it’s always keen to remind me that AI can also be used to combat fraud, so I asked it to concisely explain this for the article:

“AI combats financial fraud by analysing data for patterns, detecting anomalies, and generating real-time alerts. It assesses risk, identifies suspicious activities, and adapts to emerging fraud techniques, enabling proactive prevention and safeguarding client assets.”

How that’ll work in practice, remains to be seen, but I think it’s only fair that it gets a chance to say its piece!

From a selfish point of view, I don’t believe the role of a financial planner is under threat, as it’s a human-centric role. If anything, there will be a premium put on a human-based service. Financial planners will continue to have job security until such a time that we cannot distinguish man from machine.

Image
author image

Tom Redmayne

Hubspot form settings

Change the portalId and formId numbers in the code below

Portal id: 316077

Form id: 73339447-8d8e-4274-8ff9-7150e18ea5a3

Subscribe via email

Sign up to get weekly illuminate content directly to your inbox.

Image
author image

Tom Redmayne