Lucknow, Oct. 15: At least 20 women and four men died in a stampede near Varanasi city as members of the Jai Gurudev religious cult headed to a congregation, with devotees claiming a police baton-charge on a narrow bridge led to the tragedy.

Another 50 people were injured, five critically, and over two dozen women and children were missing from the site, about 10km from the temple town and within Prime Minister Narendra Modi's constituency.

Eyewitnesses said many devotees had jumped into the Ganga off the Rajghat Bridge to save themselves from the stampede, which the police have blamed on overcrowding on the bridge and rumours of an impending collapse.

"We had given permission for an assembly of only 3,000 people for the congregation, but 3 lakh turned up," Varanasi district magistrate Vijay Kiran Anand said.

The occasion was a two-day " mahasamagam" from today in memory of Jai Gurudev, the Mathura-based cult's founder who died in 2012. The venue was Dumri, a village in Chandauli district about 2km from the bridge on the other bank, where makeshift camps had been built.

Led by Pankaj Baba, the cult's principal guru, a huge procession of buses, SUVs, cars and 50,000-odd pedestrians approached the bridge from the Varanasi side around noon. Eyewitnesses claimed more than 25,000 people were on the bridge when the panic rush broke out.

A police officer in Lucknow said the force knew that an excessively large number of people had gathered in Varanasi but were helpless because of the cult's close ties with the ruling Samajwadi Party's first family.

"They were on every crossroad, every street. Officers didn't dare stop them because they knew the clout of Pankaj Baba," he said.

Lalji Chauhan of Sitapur, whose wife Rajwanti died in the crush, blamed a police baton charge. "The devotees tried to protect themselves from the sudden attack and the stampede happened," he said.

Additional director-general of police Daljit Chaudhary evaded the question whether there was a baton charge but said a man had died of suffocation on the bridge because of overcrowding, leading to rumours that set the stampede off.

A PTI report quoted state police chief Javeed Ahmed as saying a buzz that the bridge was collapsing led to the stampede. Security has been strengthened for tomorrow's mahasamagam.

Modi tweeted he was "deeply saddened" and offered "condolences to the bereaved families". He spoke to local officials and asked them to take care of the injured.

Chief minister Akhilesh Yadav has announced a compensation of Rs 5 lakh for each bereaved family and Rs 50,000 for the injured. He has offered free treatment to the injured and announced a probe.

Two of Varanasi's superintendents of police, for city and traffic, have been suspended along with the circle officer (kotwali) and the officers in charge of the Ramnagar and Mughalsarai police stations for "gross negligence leading to deaths".

The Jai Gurudev cult had made news in June when the supporters of one of its gurus, Ram Briksh Yadav, who had captured the Jawahar Bagh park in Mathura, fired on police during an eviction bid. Two officers and 27 followers of Ram Briksh died.

Ram Briksh and Pankaj Baba had been fighting a succession battle since Jai Gurudev's death. Pankaj Baba grabbed the ashram and declared himself the heir thanks to his closeness to a Samajwadi bigwig, who later allegedly incited Ram Briksh to set up his own ashram at Jawahar Bagh.

Born in a Yadav family in Etawah, Jai Gurudev began preaching in Varanasi in 1952 and eventually established his ashram in Mathura after founding the cult in the 1970s. A trust looks after the cult's Rs 4,000 crore worth of properties.

Jai Gurudev had declared sometime in the early 1980s that he would bring back Netaji Subhas Chandra Bose. He later called a meeting of his followers in Kanpur and declared that he himself was Bose.

Many among the crowd that had gathered to see Netaji grew angry and attacked Jai Gurudev, whom the local police somehow managed to whisk away to safety.

In October 2013, a stampede at a religious gathering in Madhya Pradesh killed more than 110 people.