Beware: How Generative AI Is Empowering Fraudulent Schemes - Protect Yourself with Essential Tips

Quick Links

• The Democratization of Fake Images
• AI Voices Lend Credibility to Video
• Dating App Scammers Cast Wider Nets
• AI Video Is On the Horizon
• AI Can Even Fake Reviews
• Things Can Only Get Better (Worse)

Key Takeaways

• Scammers are getting more creative with fake products and AI-generated voiceovers to hook unsuspecting buyers.
• Dating app scammers use AI chatbots to target victims; fake reviews become easier with large language models.
• As AI technology improves, it will be harder to distinguish real content from fakes; always research before purchasing online.

The term “scam” applies to everything from a sophisticated targeted attack to a dodgy pre-roll ad on your favorite video streaming site. The bad news is that engineering these scams is getting easier.

The Democratization of Fake Images

There’s a fine yet distinct line between scam and spam, but the two realms regularly overlap. This is especially true when it comes to online advertising, whether you’re browsing the web or scrolling through social media. Advertisers don’t approve every advert they show, instead relying on user reports to weed out the bad actors.

If you spend a lot of time online, you’re probably scrolling past countless dodgy advertisements every day. Perhaps the most interesting of these are the products that don’t exist and are instead fabrications either made by hand or using generative AI like DALL-E or Midjourney.

Hooking would-be buyers with fake products is nothing new, but once upon a time, you’d need the vision, some Photoshop skills, and a chunk of spare time. With the arrival of text-to-image generators, you can spend as little or as much time as you want feeding prompts into an image generator until you get the results you’re looking for. The sky is the limit when it comes to the sorts of products you can come up with.

In April 2024 Snopes reported that a large number of “Cat’s Eye Dazzle” scams were hitting social media like a storm. The scam involved generating fake flowers that look like cats, to draw in users on social networks like Facebook. eBay listings popped up for the seeds, while other “vendors” used self-hosted shopping portals to cash out.

For some, the fact that these flowers are AI-generated fakes is immediately obvious. Other similar scams aren’t quite as far-fetched. YouTube channel Atomic Shrimp created a video documenting the phenomenon, exploring more reasonable-sounding listings for “true” blue roses and sunflowers, neither of which exist.

Flowers that look like cats and plants with impossible colorings are just two examples of scams that appear to have caught hundreds if not thousands of people out. These images took a few seconds to generate, and a few more to post on Facebook.

AI Voices Lend Credibility to Video

You’re probably well aware of the rise of text-to-voice features, with social platforms like TikTok making it easy to generate a voiceover that sounds natural but is also easy to understand. By now, the “TikTok lady” voice is as much about TikTok’s branding as it is a useful tool for content creators.

But AI voices are so much more than that. ChatGPT can generate voiceovers if you pay for the premium tier, and services like ElevenLabs have free tiers available. I’ve seen smaller YouTube channels run by creators who aren’t comfortable speaking on camera turn to AI-generated voices. This can be jarring but is at least grounded in good intentions.

The main hook is that a voiceover lends credibility to your video. A few years ago you’d need to put time and effort or funding into adding a voiceover to your video. You’d need equipment and practice or a hired voice actor. Now it’s as simple as writing a script you’re happy with and picking the voice you’d like to use.

This makes scammy advertisements for get-rich-quick schemes and dropshipped products all the more credible. If the advert sounds like the real deal, you’re probably inclined to put more faith in the product. The people running these adverts know this, that’s why there’s been an explosion in this practice over the last year.

For now, it’s not too hard to identify AI-generated voiceover. Inflections and pronunciation can sound a bit robotic, and the natural rhythm of human speech isn’t quite there. Buckle up, because these tools are only going to improve and eventually you won’t be able to tell the difference.

Dating App Scammers Cast Wider Nets

It’s one thing being baited with fake products and a robotic voiceover, but what about a scam that personally targets you? That’s exactly what’s happening within the online dating space on a massive scale, thanks to the power of large language models that can converse like real people (well, close enough).

Using AI-powered chatbots, scammers can cast a wide net to snare their victims using software that can effectively hold down a conversation long enough to grab someone’s attention. There are a few ways this is being used, and all of them feel invasive.

Some are using chatbots to message vast swathes of people, and then follow up in the hope of meeting someone they genuinely like. In early 2024, there were countless reports of a man who met his wife using this technique. Whether you believe it or not, there are a huge number of mobile apps and services that promise to chat potential partners up for you (for a fee, of course).

Things take a decidedly more sinister turn when these tools are used to find targets for financial scams. Online scammers have long exploited the basic human desire for companionship in a bid to extract money from their marks. Now they can put chatbots to work for them, taking over when they feel it’s necessary.

Couple this with an AI image generator that gives you access to as many original pretty faces as you want, and scammers don’t even need to steal someone else’s dating profile pictures anymore.

AI Video Is On the Horizon

Convincing text-to-video isn’t broadly available at the time of writing, but with tools like OpenAI SORA on the horizon it’s set to become a force to be reckoned with. With only a prompt, OpenAI’s tool can generate realistic-looking moving pictures (though if you look carefully you can spot the flaws ).

OpenAI

If nothing more, AI-generated video will only exacerbate the problems posed by other AI-generated content. Through the use of ever-improving artificial intelligence, it’s not hard to imagine a future where you can generate fake product images, video of that product being used in relevant locations by “real” people, and a voiceover for the video that you can use to advertise it.

If you’re feeling especially enterprising, you can then use a large language model to boost your product’s standing…

AI Can Even Fake Reviews

Fake reviews are nothing new. We’ve all seen them and many of us have fallen for them (whether we know it or not). But until only a few years ago, fake reviews required some effort. With the advent of large language models, fake reviews can quite literally write themselves on a massive scale.

The misuse of large language models to generate reviews and promote products or businesses is one thing, but the technology works both ways. What about using this technology to craft reviews that unfairly criticize a rival product? It’s never been easier to generate a negative screed, even if you have to copy-paste it yourself.

Couple AI-generated reviews with more traditional brushing scams and you’ve got the believability of a “verified purchase” label to boot. Ultimately it’s up to platform holders to combat this new wave of misleading online content.

Things Can Only Get Better (Worse)

These tools will get better over time, making it more difficult to tell the real from the fantastical. You’ll need to be wise to the technology if you want to stay ahead of scammers. If you understand what is possible, you’re better armed to understand what’s real and what’s not.

Facebook is particularly rife with AI-generated images , so understanding how to spot AI-generated fakes is a valuable skill. You should always be wary of online advertisements, whether you spot them on your favorite websites or a social network.

DALL-E

Always do your research before you buy something, whether it’s a piece of technology or a packet of seeds you saw on Facebook. Ironically, you could also ask your favorite chatbot that sources its claims.