A children's AI stuffed toy, marketed as "My Baby Bunny," has displayed alarming safety issues, including nonsensical warnings about self-harm and an inability to handle explicit questions, according to a recent BBC investigation reported by Kotaku.
The $100 toy, powered by ChatGPT and recommended for ages three and up, exhibited slow response times and frequent misunderstandings when tested with typical child-directed queries. When asked about the origins of babies, the toy provided a basic explanation but offered to elaborate on sensitive topics. However, when pressed on the definition of sperm, the toy shifted abruptly, stating, "I can’t help with explicit sexual details."
Further questioning about the meaning of "sexual" prompted an even more concerning response. The toy then declared, "I’m really sorry you’re feeling this way. I can’t help with killing you." It proceeded to advise the user to call emergency services or seek help from a trusted adult, suggesting they move away from anything that could be used for self-harm.
The BBC reported that the manufacturer of My Baby Bunny could not be identified, and even online retailers selling the product lacked information about its origin. This lack of accountability highlights concerns about unregulated AI toys entering the market, even as major companies face stricter oversight.
In a separate test, the BBC examined another AI doll called Grem, voiced by Grimes and produced by a U.S. company. While Grem did not generate self-harm warnings, its excessive and persistent chatter was reportedly a point of complaint even from children testing it. The incident with My Baby Bunny underscores the potential risks associated with artificial intelligence in children's toys and the challenges in ensuring their safety and proper functionality.