Pratyusha Sharma listens to sperm whales speak. She hopes to understand what their short bursts of clicking sounds – called codas – mean.

She is laser-focused on unmasking their complex linguistic patterns. That is her work; the PhD student at the MIT Computer Science and Artificial Intelligence Laboratory is collaborating with the nonprofit Project CETI (Cetacean Translation Initiative) that is applying artificial intelligence to decode the language of sperm whales.

The knowledge that whales communicate with each other is not new. In a 1971 Science magazine article, CETI’s principal advisor, the late Roger Payne, had documented that humpback whales can ‘sing’.

“However, sperm whale calls are distinct from the melodic ‘songs’ recorded by Dr Payne,” Pratyusha Sharma tells The Telegraph Online. “Instead, they resemble rhythmic beatboxing, with intricate patterns and variations that form a complex acoustic language.”

She explains how the project ended up in her research portfolio.

“In my PhD, I was thinking about algorithms that will allow us to do ‘structure-discovery’ from unstructured data – how we can verify if we have recovered the right structure. In the case of whales all we have are their unstructured calls and behaviour and what we want to do is understand how it is structured and what purpose it serves. The project was a wonderful fit.”

She says being part of the Project CETI Machine Learning Team has been nothing short of a privilege. “When I heard about the project from my advisor, I just loved what the team was exploring, and joined in.”

What sparked her interest in non-human communication? “The linguistic diversity in India made me realise that language shapes how we perceive our world and the kind of things we think about,” replies Pratyusha. “This has influenced how I think about whale communication and the role whale calls might play in shaping their lives, activities and thoughts.”

Pratyusha, who graduated from IIT Delhi, interned in Mumbai with Deutsche Bank and worked in Tokyo with Honda Innovations before she joined MIT, knows four languages – Hindi, Telugu, Japanese and English.

How to understand what sperm whales say

To study whale communication, Pratyusha and her team use Digital Acoustic Recording Tags (DTAGs) – advanced non-invasive suction-cup devices that are attached to the whales temporarily.

Sperm whale calls resemble rhythmic beatboxing, with intricate patterns and variations that form a complex acoustic language (Wikipedia)

“These devices help in mapping whale behaviour and positions while simultaneously recording vocalisations using microphones to capture underwater vocalisations, accelerometers, gyroscopes and magnetometers to record movement and orientation, and pressure sensors to track diving depth,” Pratyusha explains.

Codas help whales coordinate with each other, inform about individual entities and also express clan loyalty when required, she says. And while hunting, they use a different type of “rapid codas”, which are referred to as echolocation.

Pratyusha and her team’s recent work published in Nature Communications 2024 (‘Contextual and Combinatorial Structure in Sperm Whale Vocalizations’) suggests that their communication system operates as a “combinatorial coding system”.

“Each call is constructed from features such as rhythm, tempo, rubato, and ornamentation,” she explains.

The role of AI

Pratyusha credits her undergraduate studies at the robotics lab at IIT Delhi as a turning point in her life.

“The experience critically shaped my understanding of what I wanted to do and the types of questions I wanted to ponder growing up. It also was the place where I learned everything I know about how research is done and what it means to practise science.”

For the CETI project, she has incorporated machine learning – a subfield of AI that uses algorithms to find and predict patterns – across multiple stages of the process.

“Unsupervised machine learning helps to uncover patterns and structures in the data. Self-supervised machine learning to develop sequence models that capture the nuances of whale calls and associated behaviours. Supervised machine learning to create automated pipelines for detecting and segmenting whale calls within audio and video recordings,” Pratyusha explains.

While AI is the driving force behind the project, it alone is not sufficient to crack the whale codas.

“CETI also draws on expertise from bioacoustics, marine biology and computational linguistics to ensure utmost accuracy and alignment with biological realities and communication theories,” Pratyusha says.

Can animals talk to humans?

A captive gorilla named Koko gained social media attention when she was seen in a 2015 video using sign language to apparently send a message to humans to ‘fix’ ‘help’ and ‘protect’ the Earth. The credibility of the inference is debatable; but is such communication at all possible?

Pratyusha does believe in the possibility of a similar kind of establishment of communication between humans and whales.

“I think every animal develops a unique communication protocol tailored to its environment, and I would love to learn more about how the plethora of unique communication systems are structured – their similarities and differences. However, any efforts must be conducted sustainably, ensuring minimal disruption to the whales and their natural habitat,” she adds.

Asked whether Project CETI intends to go beyond whales with the research, she says: “It would be great to study the communication of many other animals – including birds, elephants – and other types of cetaceans. Our goal is to apply the tools and techniques we’ve developed to study other species. By doing so, we hope to deepen our understanding of the diverse forms of communication used by the many creatures we share this planet with.”