AI-powered scam calls are becoming increasingly sophisticated and difficult for the average person to detect, posing a significant threat to less tech-savvy individuals. The accessibility of this technology has led to a surge in its use for fraudulent purposes.
Content creators have begun experimenting with ways to combat these scams. YouTube creator Kitboga discovered that simple prompt injection can cause AI scam callers to enter an infinite loop, repeatedly vocalizing phrases such as "kooga-ooga-amen, Albuquerque, New Mexico." This vulnerability appears to stem from the AI's difficulty in processing certain inputs or commands.
Another content creator, Jim Browning, has focused on exposing a different, more insidious form of AI fraud: deepfake video scams. These scams utilize fabricated video and audio to impersonate individuals, aiming to deceive victims. The effectiveness of these deepfakes is a growing concern, as they are becoming easier and faster to produce.
Simon Mylius, a researcher at MIT and collaborator with the AI Incident Database, noted the alarming ease with which AI-generated fake content can now be produced. He stated that the capabilities have advanced to a point where almost anyone can create such content, with virtually no barrier to entry. Bobby Ford, chief strategy and experience officer at the social engineering defense company Doppel, echoed this sentiment, adding that AI scammers are increasing their output due to the reduced time and resources required to generate deepfakes. This acceleration in production means a higher volume of scams are being deployed.
While individuals who are already knowledgeable about technology may be able to distinguish between legitimate communications and AI-spoofed ones, the broader public remains vulnerable. The rise of these scams highlights the need for increased awareness and education, particularly among those who may be more susceptible to falling victim to such schemes. Sharing examples of these AI scam breakdowns, as popularized by creators like Kitboga and Browning, could serve as a cautionary tool for friends and family who may be less aware of these emerging threats.