Sundar Pichai, in an X post, has announced the launch of DolphinGemma, a large language model (LLM) in collaboration with Wild Dolphin Project (WDP) to help scientists decipher dolphin communication and potentially talk back to them.
The model is capable of learning the structure of dolphin vocalisations and creating unique dolphin-like sound sequences.
What is DolphinGemma
Developed by Google, DolphinGemma was trained on 40 years of underwater audio-visual data from recording Atlantic spotted dolphins in the Bahamas, sourced from WDP.
Using Google’s SoundStream technology, DolphinGemma processes natural dolphin sounds and predicts subsequent sounds, much like how LLMs predict the next word in human text.
WDP is not just listening to the marine animals but also exploring ways to establish a two-way communication with them.
This effort has led to the formation of CHAT (Cetacean Hearing Augmentation Telemetry) system in partnership with Georgia Institute of Technology.
CHAT is an underwater computer designed to establish a shared vocabulary.
The idea is to associate distinct synthetic whistles with different objects that dolphins like to play with. Then leading them to mimic the corresponding sound to request a specific item.
As more natural dolphin sounds are decoded, those can be added in the system as well. A shared vocabulary with the dolphins can be established for interactive communication.
DolphinGemma’s predictive model can help CHAT identify and forecast mimics faster, making interactions easier.
Using Pixel phones to analyse real-time dolphin sounds
At around 400 million parameters, the model is lightweight enough to run on Pixel phones used in field research. This enables real-time underwater analysis with quick benefits.
Using Google smartphones cuts down the need for custom hardware, making the system easier to maintain, lowers power consumption, and reduces device's cost and size.
The next generation, built around Google Pixel 9, expected to launch in summer 2025, adds speaker and mic features, and uses the phone’s advanced processing power to run deep learning and template matching together.
Pichai also announced rolling out DolphinGemma as an open model in the summer of 2025.
It intends to aid research across other cetacean species, like bottlenose or spinner dolphins. While different species may need fine-tuning, the model’s open design makes it easy to adapt.