Google DeepMind Builds an AI That Catches Scam Conversations On-Device

Google DeepMind says it has worked with the Singapore government to build an AI system that spots scams as they unfold in a conversation. The model is a version of Gemma, Google’s small and lightweight model, given additional training on scam conversations seen in Singapore. The goal is to embed it in apps so it can flag manipulation attempts before they expose users to financial risk or compromise their credentials.

What kind of scam conversations does it learn?

It learns the scam tactics actually in use in Singapore. Google DeepMind explained that widely used general-purpose models often fail to catch deceptions specific to a given region. So the training data included scam conversations involving impersonation of public agencies, banks, and local public figures. Scam techniques currently circulating locally were also folded into the training.

The model is built to tell two things apart: ordinary everyday conversation on one side, and conversation designed to manipulate the other person on the other. Google DeepMind says this additional training lets the model, once embedded in an app, distinguish between the two types. For users, it amounts to a tool that helps gauge the intent of whoever is on the other end.

Where does my conversation get processed?

On the user’s own device. The existing approach to threat detection that Google DeepMind offered as a point of comparison sends messages to cloud servers for inspection. This model was designed to look for scam attempts on the device itself, without sending conversation content to any external server.

Image: Google DeepMind

Google DeepMind explained that a model embedded in an app can analyze the flow of a conversation more quickly. The idea is to help users respond before they are exposed to financial risk or their credentials are leaked.

App rollout will come in stages

The work of embedding the model in apps will proceed in stages, together with the Singapore government. The model can see conversations on the device only when the user has given consent under the terms of service of the app in question. Because the model runs entirely on the device, Google DeepMind says that neither Google nor the Singapore government can access a user’s conversations through this embedded model.

Image: Google DeepMind

Two things are worth watching from here: how well the system distinguishes everyday chatter from actual scam attempts in real apps, and how the user consent process is designed.