Following Google's recent achievement in fully mapping the brain and central nervous system of an adult male fruit fly into a 3D neural model, software engineers have demonstrated its capability to play several video games. The process involved using AI to reconstruct over 166,000 neurons from millions of 2D images, a feat Google described as a major milestone in neuroscience.
Shortly after the map's creation, Alex Wormuth, a software engineer at Coinbase, began training the digital fruit fly brain to play the original Doom. This was achieved by mapping each Doom frame to stimulate sensory neurons, with neural activity then translated into game controls. The system incorporates reinforcement learning, where damage in the game triggers a stimulus to specific dopamine cells, encouraging learning. A real-time display of the fly's gameplay is available, showing both the in-game perspective and a third-person render of the fly navigating Doom's maps. As of its latest update, the fly's performance has been described as less than optimal.
Beyond Doom, the fruit fly's capabilities have been extended to other titles. Lyra Bubbles, using X, has trained the fly to play Beat Saber. This involved using brain data and replays of the game to teach the fly where to strike and why, employing reinforcement learning. While the current iteration relies on external signals, further training aims to make its gameplay fully reactive without input data.
Jessica Paquette also shared progress on X regarding the digital fly brain's performance in Super Mario 64, noting a shift in focus to "fly engineering." Separately, YouTuber Ro0oney utilized Flywire.AI, a similar project developed in collaboration with Princeton University, to place a digital fly brain into Minecraft. The Flywire project analyzed a female fruit fly brain, contrasting with Google's analysis of a male brain, but applied a similar methodology.
The development raises questions about future possibilities, particularly concerning the replication of human cognitive functions and the potential timeline for such advancements.